DNN结果略有不同

I am wondering if there is any randomness in the DNN training recipe such as steps/nnet2/train_tahn.sh. I’m asking this because I noticed that when I run the same script twice, I got slightly different results. What could be the randomness in the script and how can I get exactly the same result each time I run the script? 我想知道DNN训练脚本中是否存在任何随机性，例如steps / nnet2 / train_tahn.sh。 我之所以这样问，是因为我注意到当我两次运行相同的脚本时，得到的结果略有不同。脚本中的随机性是什么？每次运行脚本时如何获得完全相同的结果？

Probably the most important source of randomness that can't be removed is that if you are using the CPU-based training, there are multiple threads and they are not synchronized (asynchronous SGD), so depending on the order of execution the result gets slightly different. Some authors, when publishing results with DNNs, run the same thing a few times and show the average. 可能无法消除的最重要的随机性来源是，如果您使用的是基于CPU的训练，则有多个线程并且它们不同步（异步SGD），因此根据执行顺序，结果会略有不同。不同。一些作者在发布具有DNN的结果时，会多次运行同一件事并显示平均值。

Thank you. I am using GPU for the nnet2 recipe, does the randomness still occur then? 谢谢。我将GPU用于nnet2脚本，那么是否还会出现随机性？

There are a lot of places where rand() is called in the code... mostly it should still be deterministic if you run on the same hardware and OS, but there are probably some cases where it's not deterministic. And maybe some GPU computations can give slightly different results because of scheduling issues. I'm not sure exactly what the source of the randomness is; I generally haven't viewed being deterministic as a super-important feature that we should spend a lot of effort to preserve. 在很多地方，在代码中调用rand（）…在大多数情况下，如果您在相同的硬件和操作系统上运行，仍应是确定性的，但是在某些情况下，它不是确定性的。也许由于调度问题，某些GPU计算可能会给出略有不同的结果。我不确定随机性的确切来源是什么；通常，并没有将确定性视为一项非常重要的功能，我们应该花费很多精力来保留它。