mkgraph时AddSelfLoops函数断言错误

最近被fst搞的头疼，本文的fstinfo和fst初始状态理解值得一看，也注意IRSTLM比LM好。

 

Hi all! I was trying to use the voxforge recipe under cygwin environment. The default LM tool is MITLM. But I cannot compile it with my machine. So I use the IRSTLM package. Then other things were fine except when tried to make the monophone decode graph. I got this error:大家好！我试图在cygwin环境下使用voxforge方案。默认的LM工具是MITLM。但是我不能用我的机器编译它。因此，我使用IRSTLM软件包。然后其他事情都很好，除了尝试制作单声道解码图时。我收到此错误：

utils/mkgraph.sh --mono data/lang_test exp/mono exp/mono/graph
add-self-loops --self-loop-scale=0.1 --reorder=true exp/mono/final.mdl
assertion "fst->Start() != fst::kNoStateId" failed: file "hmm-utils.cc", line 576, function: void kaldi::AddSelfLoops(const kaldi::TransitionModel&, const std::vector<int>&, kaldi::BaseFloat, bool, fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*)
utils/mkgraph.sh: line 107: 9576 Aborted (core dumped) add-self-loops --self-loop-scale=$loopscale --reorder=true $model < $dir/HCLGa.fst > $dir/HCLG.fst

I don't know what's going and what might cause. Any clues?我不知道怎么回事，可能会导致什么。有什么线索吗？
How big is the HCLGa.fst file?What does the output of the following command show?fstinfo HCLGa.fstHCLGa.fst文件的大小是多少？以下命令的输出显示什么？
It's only 66-byte long. And the output of "fstinfo HCLGa.fst" comes to be an error它只有66字节长。并且“ fstinfo HCLGa.fst”的输出出现错误

Seems like one of the previous graph building steps failed, track back through the script and look at the output of the previous steps. The intermediate fsts should be pretty big and readable with fstinfo. I'm not familiar with cygwin, it is limited to 32bit? The graph building can be very memory heavy and you might be running out of memory to perform the determinization.似乎先前的图形构建步骤之一失败了，请追溯脚本并查看先前步骤的输出。中间的fsts应该很大，并且可以被fstinfo读取。我对cygwin不熟悉，它仅限于32bit？图形的建立可能会占用大量内存，并且您可能用尽了内存来执行确定。
After a careful check on the procedure, I found there were errors in the format data step(local/voxforge_format_data.sh). When doing arpa2fst, I got:在仔细检查过程之后，我发现格式数据步骤（local / voxforge_format_data.sh）中存在错误。在做arpa2fst时，我得到了：

--- Preparing the grammar transducer (G.fst) for testing ...
arpa2fst -
\data\ Processing 1-grams
Processing 2-grams
Processing 3-grams
Connected 0 states without outgoing arcs.
remove_oovs.pl: removed 1 lines.
fstisstochastic data/lang_test/G.fst
0 0

The G.fst is bad. Right now I still don't know the reason. I am suspecting that the problem may lie in the LM I generated. As I said, the MITLM won't compile on my machine(because the version I checked out seemed to have lost a lot of files), so I used IRSTLM.G.fst不好。现在我仍然不知道原因。我怀疑问题可能出在我生成的LM上。就像我说的那样，MITLM无法在我的机器上编译（因为我签出的版本似乎丢失了很多文件），所以我使用了IRSTLM。
In my experience IRSTLM is not very reliable - in fact all of the FOSS LM toolkits seem to be quirky. I would suggest to try SRILM if you can get it to compile in your environment.以我的经验，IRSTLM并不是很可靠-实际上，所有FOSS LM工具包似乎都是古怪的。如果可以在环境中进行编译，我建议您尝试使用SRILM。
Hi Vassil. It's good to have your reply.嗨，瓦西尔。收到您的回复真是太好了。

Right now I cannot conclude that it is the LM which caused those errors. But I've got more information.In "voxforge_format_data.sh" the code to generate G.fst is like this:现在，我无法断定是由LM造成了这些错误。但是我有更多信息。在“ voxforge_format_data.sh”中，生成G.fst的代码如下：

cat $lmdir/lm.arpa | \ grep -v ' ' | \ grep -v ' ' | \ grep -v ' ' | \ arpa2fst - | fstprint | \ utils/remove_oovs.pl $tmpdir/oovs.txt | \ utils/eps2disambig.pl | utils/s2eps.pl | fstcompile --isymbols=$test/words.txt \ --osymbols=$test/words.txt --keep_isymbols=false --keep_osymbols=false | \ fstrmepsilon > $test/G.fst

The last command is fstrmspsilon. I just removed fstrmepsilon and got G1.fst. Then I checked it's information by fstinfo G1.fst and got:最后命令是fstrmspsilon。我只是删除fstrmspsilon，就得到了G1.fst，用fstinfo检查看到：

fst type vector
arc type standard
input symbol table none
output symbol table none

of states 95481
of arcs 203158
initial state 0

of final states 0
of input/output epsilons 2
of input epsilons 2
of output epsilons 95481
of accessible states 95480
of coaccessible states 0
of connected states 0
of connected components 1
of strongly conn components 3
input matcher n
output matcher n
input lookahead n
output lookahead n
expanded y
mutable y
error n
acceptor n
input deterministic y
output deterministic n
input/output epsilons y
input epsilons y
output epsilons y
input label sorted n
output label sorted n
weighted y
cyclic y
cyclic at initial state n
top sorted n
accessible n
coaccessible n
string n

But if I do fstrmepsilon to G1.fst, then the resulting G.fst will be 66-byte long which only contains the fst file header. the fstrmepsilon remove all lines in G1.fst. I don't know why.但是，如果我对g1.fst执行fstrmepsilon，则生成的G.fst将为66字节长，其中仅包含fst文件头。 fstrmepsilon删除G1.fst中的所有行。我不知道为什么

This Fst doesn't have any final states, so the epsilon removal is removing all the states and transitions该Fst没有任何最终状态，因此去除epsilon的过程是去除了所有状态和转换。
Check what happens if you pipe the output of arpa2fst into fstinfo If this also doesn't have final states there is a mismatch between the arpa lm and what arpa2fst expects检查如果将arpa2fst的输出通过管道传输到fstinfo中会发生什么情况，如果这也没有最终状态，则说明arpa lm与arpa2fst期望的状态不匹配
seems that the lm.arpa lost it's "< /s >" tag.似乎lm.arpa丢失了它的“ </s>”标签。
By the way I think IRSTLM requires to "manually" add "<s>" and "</s>" at the start/end of the utterances using "add-start-end.sh < your-text-file" (https://sourceforge.net/apps/mediawiki/irstlm/index.php?title=User_Manual).I haven't look at the arpa2fst's internals, but if it relies on the </s> tags and you missed the above step ...

顺便说一句，我认为IRSTLM要求在发声的开头/结尾“手动”添加“ <s>”和“ </ s>”，通过使用“ add-start-end.sh <your-text-file”（https） ：//sourceforge.net/apps/mediawiki/irstlm/index.php？title = User_Manual）来添加。我没有看过arpa2fst的内部结构，但是如果它依赖</ s>标记，则您错过了上面的步骤。 ..
Thank you all! The problem is solved!谢谢你们！问题已经解决了！