Newnnet3#1

Open

mrsrikanth wants to merge 197 commits into

Commits on Nov 4, 2019

replicating mini_librispeech/s5 to mini_librispeech/s5b
Srikanth MADIKERI
committed
adding run_tdnn_2a.sh, data_pre_common.sh and get_model_context.sh to egs/mini_librispeech
Srikanth MADIKERI
committed
egs/mini_librispeech/s5b: removing num-clusters options while calling build_tree.sh from run_tdnn_2a.sh
Srikanth MADIKERI
committed
adding many new source files from PR:
Srikanth MADIKERI
committed
train2.sh: chaning chaina references to chain
Srikanth MADIKERI
committed
egs/wsj5/steps/nnet3/chain/get_model_context.sh: removed bottom nnet related variables
Srikanth MADIKERI
committed
steps/nnet3/chain/get_raw_egs.sh: removed long-key option to nnet3-chain-get-egs
Srikanth MADIKERI
committed
steps/nnet3/chain/get_raw_egs.sh: added cmvn and online_ivector_dir options
Srikanth MADIKERI
committed
egs/wsj5/steps/nnet3/chain/get_raw_egs.sh: working version with minilibrespeech recipe
Srikanth MADIKERI
committed
modifications to process_egs.sh
Srikanth MADIKERI
committed
in egs/mini_librespeech
Srikanth MADIKERI
committed
steps/nnet3/chain/train2.sh: working version of the new chain model training script
Srikanth MADIKERI
committed
egs/wsj/s5/steps/nnet3/chain/compute_preconditioning_matrix.sh: compute LDA for chain models
Srikanth MADIKERI
committed
steps/libs/nnet3/train/dropout_schedule.py: compute dropout schedule for chain model scripts
Srikanth MADIKERI
committed
steps/nnet3/chain/get_raw_egs.sh: added optoins for cmvn, online-ivectors. looks for transition model in two places.
Srikanth MADIKERI
committed
steps/nnet3/chain/internal/choose_egs_to_merge.py: now an executable
Srikanth MADIKERI
committed
steps/nnet3/chain/internal/get_train_schedule.py: now an executable
Srikanth MADIKERI
committed
steps/nnet3/chain/process_egs.sh: an intermediate version with lots of commented code. but committing it because it works and the code is moving towards what we want
Srikanth MADIKERI
committed
steps/nnet3/chain/process_egs.sh: changed chaina references to chain
Srikanth MADIKERI
committed
steps/nnet3/chain/validate_processed_egs.sh: chaina references changed to chain
Srikanth MADIKERI
committed
steps/nnet3/chain/validate_randomized_egs.sh: changed chaina references to chain
Srikanth MADIKERI
committed
steps/nnet3/chain/validate_raw_egs.sh: changed chaina references to chain
Srikanth MADIKERI
committed
steps/nnet3/chain/get_raw_egs.sh: working version of get_raw_egs. num_input_frames is now obtained from data folder
Srikanth MADIKERI
committed
steps/nnet3/chain/process_egs.sh: working version of process_egs.sh
Srikanth MADIKERI
committed
steps/nnet3/chain/randomize_egs.sh: working version of randomize_egs
Srikanth MADIKERI
committed
Idiap related changes to files in egs/mini_librispeech/s5b:
Srikanth MADIKERI
committed
again, Idiap related changes in egs/wsj/s5/steps/nnet3/chain/train.py. Don't merge this
Srikanth MADIKERI
committed
minilibrespeech recipe: changes to run.sh and decode.sh.
Srikanth MADIKERI
committed
the commit to updated code in src/nnet3
Srikanth MADIKERI
committed
nnet3-chain-get-egs.cc: handles long keys
Srikanth MADIKERI
committed
egs/minilibrespeech/s5b/local/chain/tuning/run_tdnn_2c.sh : the working tdnn recipe for this egs.
Srikanth MADIKERI
committed
adding RESULTS file to egs/mini_librispeech/s5b
Srikanth MADIKERI
committed
adding swbd/s5c/local/chain/tuning/run_tdnn_8k.sh to test it with new kaldi recipe
Srikanth MADIKERI
committed
egs/swbd/s5c/local/chain/tuning/run_tdnn_8k.sh: updated tdnn script to run train2.sh
Srikanth MADIKERI
committed
commiting the version of run_tdnn_8k.sh that finish training finally!
Srikanth MADIKERI
committed
removed train.py from run_tdnn_8k.sh
Srikanth MADIKERI
committed
modifying egs/wsj/s5/steps/nnet3/chain/train2.sh to include lrate. I
Srikanth MADIKERI
committed
running v3 of egs/swbd/s5c/local/chain/tuning/run_tdnn_8k.sh which uses
Srikanth MADIKERI
committed
small modification to egs/swbd/s5c/local/chain/tuning/run_tdnn_8k.sh:
Srikanth MADIKERI
committed
fixed a stupid bug while in run_tdnn_8k.sh while checking if common_egs_dir is empty (I was doing it the other way round)
Srikanth MADIKERI
committed
fixed another bug in run_tdnn_8k.sh: wasn't really using common_egs_dir after setting it correctly.
Srikanth MADIKERI
committed
starting run_tdnn_8k.sh from stage 17
Srikanth MADIKERI
committed
starting run_tdnn_8k.sh from stage 12
Srikanth MADIKERI
committed
fixed another place in run_tdnn_8k.sh that didn't use the right egs dir
Srikanth MADIKERI
committed
silly mistake in the previous commit
Srikanth MADIKERI
committed
train2.sh: adding model_dir_in explicitly to the train command
Srikanth MADIKERI
committed
train2.sh: fixing --lrate to --learning-rate
Srikanth MADIKERI
committed
removing --raw=true options to nnet3-am-copy in train2.sh
Srikanth MADIKERI
committed
train2.sh: nnet3-am-copy is creating troubles becaues nne3-chain-train2 doesn't write models. it writes raw nnets. so I'm now using nnet3-am-init as a workaround
Srikanth MADIKERI
committed
nnet-chain-training2.cc: TrainInternal call updated
Srikanth MADIKERI
committed
committing soft links for swbd/s5d recipe
Srikanth MADIKERI
committed
s5d/RESULTS added
Srikanth MADIKERI
committed
stable version of tdnn_7k and tdnn_8k committed
Srikanth MADIKERI
committed
src/nnet3/nnet-chain-training2.h: removed const qualifier for chain example
Srikanth MADIKERI
committed
egs/wsj/s5/steps/diagnostic/analyze_lats.sh: disabling analyze lats. crashes on the grid at Idiap
Srikanth MADIKERI
committed
egs/wsj/s5/steps/nnet3/chain/compute_preconditioning_matrix.sh: stats file clean-up added
Srikanth MADIKERI
committed
egs/wsj/s5/steps/nnet3/chain/internal/get_train_schedule.py: different frame_shift computation
Srikanth MADIKERI
committed
egs/wsj/s5/steps/nnet3/chain/randomize_egs.sh: shuffles and splits into a number of scp files to randomize the egs
Srikanth MADIKERI
committed
egs/wsj/s5/steps/nnet3/chain/train2.sh: updated with the following changes
Srikanth MADIKERI
committed
adding nnet3/nnet-chain-diagnostics2.h and chainbin/nnet3-chain-combine2.cc
Srikanth MADIKERI
committed
moved code for the new training scripts from steps/nnet3/chain/ to steps/nnet3/chain2/
Srikanth MADIKERI
committed
added soft link to steps/nnet3/chain2 in steps/
Srikanth MADIKERI
committed
modified paths to other scripts in steps/nnet3/chain2/train2.sh
Srikanth MADIKERI
committed
renamed steps/nnet3/chain2/train2.sh to steps/nnet3/chain2/train.sh
Srikanth MADIKERI
committed
added new script to local/chain2/tuning. earlier this recipe was in s5b/local/.., but there is no need to have a new recipe
Srikanth MADIKERI
committed
added softlink to best tdnn recipe for minilibrespeech with new kaldi scripts
Srikanth MADIKERI
committed
moved local/chain/tuning/run_tdnn_8k.sh to local/chain2/tuning/run_tdnn_7k.sh
Srikanth MADIKERI
committed
reverting back steps/diagnostic/analyze_alignments.sh
Srikanth MADIKERI
committed
reverting analyze_lats
Srikanth MADIKERI
committed
removing steps/nnet3/chain/data_prep_common.sh. it doesn't belong in steps
Srikanth MADIKERI
committed
removing s5d
Srikanth MADIKERI
committed
RESULTS for swbd added
Srikanth MADIKERI
committed
minilibrespeech RESULTS integrated into s5 recipe
Srikanth MADIKERI
committed
- added data_prep_common.sh to local/chain2
Srikanth MADIKERI
committed
removing the configuration section in local/chain/tuning/run_tdnn_7k.sh
Srikanth MADIKERI
committed
removed safe-checking graphs in local/chain/tuning/run_tdnn_7k.sh. It is now the same as what is there in master
Srikanth MADIKERI
committed
setting graph_dir value (reverting the code in parts)
Srikanth MADIKERI
committed
one more rollback to local/chain/tuning/run_tdnn_7k.sh: removing graph_dir from options
Srikanth MADIKERI
committed
modifying local/chain2/tuning/run_tdnn_7k.sh to be identical to local/chain2/tuning/run_tdnn_7k.sh
Srikanth MADIKERI
committed
reverting the following files to master (to reduce the number of files to be reviewed for PR). these files originally had changes adapted to the Idiap environment:
Srikanth MADIKERI
committed
reverting 2 more files to match master:
Srikanth MADIKERI
committed
removing egs/wsj/s5/steps/nnet3/chain/internal/choose_egs_to_merge.py from the commits, it was there in the chaina branch, but it is no longer useful
Srikanth MADIKERI
committed

Commits on Nov 7, 2019

[scripts] Change make_rttm.py to read/write files with UTF-8 encoding (#3705)

entn-at
authored and
danpovey
committed

Commits on Nov 9, 2019

[src] Removing non-compiling paranoid asserts in nnet-computation-graph. (#3709)

RuABraun
authored and
danpovey
committed
[build] Fix gfortran package name for centos (#3708)

rezame
authored and
danpovey
committed

Commits on Nov 11, 2019

[scripts] Change the Python diagnostic scripts to accept non-ASCII UTF-8 phone set (#3711)

vdp
authored and
danpovey
committed

Commits on Nov 12, 2019

[scripts] Fix some issues in #3653 in split_scp.pl (#3710)

RuABraun
authored and
danpovey
committed
[scripts] Fix 2 issues in nnet2->nnet3 model conversion script (#886)(#3713)

jfainberg
authored and
danpovey
committed

Commits on Nov 14, 2019

Removing changes to split_scp.pl (#3717)
danpovey
authored

Commits on Nov 15, 2019

[scripts] Improve how combine_ali_dirs.sh gets job-specific filenames (#3720)

KimJeongSun
authored and
danpovey
committed

Commits on Nov 16, 2019

[src] Add --debug-level=N configure option to control assertions (#3690) (#3700)

iskunk
authored and
danpovey
committed

Commits on Nov 28, 2019

[src] Fix NVCC compilation errors on Windows (#3741)

cloudhan
authored and
danpovey
committed
[build] Add CMake Build System as alternative to current Makefile-based build (#3580)

cloudhan
authored and
danpovey
committed

Commits on Nov 30, 2019

[scripts] Modify split_data_dir.sh and split_scp.pl to use utt2dur if present, to balance splits

desh2608
authored and
danpovey
committed
[scripts] fix slurm.pl error (#3745)

naxingyu
authored and
danpovey
committed

Commits on Dec 1, 2019

Revert "[scripts] Modify split_data_dir.sh and split_scp.pl to use utt2dur if present, to balance splits" (#3746)
danpovey
authored

Commits on Dec 2, 2019

Commits on Dec 3, 2019

[scripts] Add scripts to create combine fmllr-tranform dirs(#3752)

marvin-nj
authored and
danpovey
committed

Commits on Dec 5, 2019

[src] CUDA decoder: fix invalid-lattice error that happens in corner cases (#3756)

hugovbraun
authored and
danpovey
committed
[egs] Add Chime 6 baseline system (#3755)

aarora8
authored and
danpovey
committed
[scripts] Fix issue in copy_lat_dir.sh affecting combine_lat_dirs.sh (missing phones.txt) (#3757)

marvin-nj
authored and
danpovey
committed

Commits on Dec 6, 2019

[src] Add missing #include, needed for CUDA decoder compilation on some platforms (#3759)

luitjens
authored and
danpovey
committed

Commits on Dec 8, 2019

[scripts] fix bug in steps/data/reverberate_data_dir.py (#3762)

HuangZiliAndy
authored and
danpovey
committed

Commits on Dec 10, 2019

replicating mini_librispeech/s5 to mini_librispeech/s5b
Srikanth MADIKERI
committed
adding run_tdnn_2a.sh, data_pre_common.sh and get_model_context.sh to egs/mini_librispeech
Srikanth MADIKERI
committed
egs/mini_librispeech/s5b: removing num-clusters options while calling build_tree.sh from run_tdnn_2a.sh
Srikanth MADIKERI
committed
adding many new source files from PR:
Srikanth MADIKERI
committed
train2.sh: chaning chaina references to chain
Srikanth MADIKERI
committed
egs/wsj5/steps/nnet3/chain/get_model_context.sh: removed bottom nnet related variables
Srikanth MADIKERI
committed
steps/nnet3/chain/get_raw_egs.sh: removed long-key option to nnet3-chain-get-egs
Srikanth MADIKERI
committed
steps/nnet3/chain/get_raw_egs.sh: added cmvn and online_ivector_dir options
Srikanth MADIKERI
committed
egs/wsj5/steps/nnet3/chain/get_raw_egs.sh: working version with minilibrespeech recipe
Srikanth MADIKERI
committed
modifications to process_egs.sh
Srikanth MADIKERI
committed
in egs/mini_librespeech
Srikanth MADIKERI
committed
steps/nnet3/chain/train2.sh: working version of the new chain model training script
Srikanth MADIKERI
committed
egs/wsj/s5/steps/nnet3/chain/compute_preconditioning_matrix.sh: compute LDA for chain models
Srikanth MADIKERI
committed
steps/libs/nnet3/train/dropout_schedule.py: compute dropout schedule for chain model scripts
Srikanth MADIKERI
committed
steps/nnet3/chain/get_raw_egs.sh: added optoins for cmvn, online-ivectors. looks for transition model in two places.
Srikanth MADIKERI
committed
steps/nnet3/chain/internal/choose_egs_to_merge.py: now an executable
Srikanth MADIKERI
committed
steps/nnet3/chain/internal/get_train_schedule.py: now an executable
Srikanth MADIKERI
committed
steps/nnet3/chain/process_egs.sh: an intermediate version with lots of commented code. but committing it because it works and the code is moving towards what we want
Srikanth MADIKERI
committed
steps/nnet3/chain/process_egs.sh: changed chaina references to chain
Srikanth MADIKERI
committed
steps/nnet3/chain/validate_processed_egs.sh: chaina references changed to chain
Srikanth MADIKERI
committed
steps/nnet3/chain/validate_randomized_egs.sh: changed chaina references to chain
Srikanth MADIKERI
committed
steps/nnet3/chain/validate_raw_egs.sh: changed chaina references to chain
Srikanth MADIKERI
committed
steps/nnet3/chain/get_raw_egs.sh: working version of get_raw_egs. num_input_frames is now obtained from data folder
Srikanth MADIKERI
committed
steps/nnet3/chain/process_egs.sh: working version of process_egs.sh
Srikanth MADIKERI
committed
steps/nnet3/chain/randomize_egs.sh: working version of randomize_egs
Srikanth MADIKERI
committed
Idiap related changes to files in egs/mini_librispeech/s5b:
Srikanth MADIKERI
committed
again, Idiap related changes in egs/wsj/s5/steps/nnet3/chain/train.py. Don't merge this
Srikanth MADIKERI
committed
minilibrespeech recipe: changes to run.sh and decode.sh.
Srikanth MADIKERI
committed
the commit to updated code in src/nnet3
Srikanth MADIKERI
committed
nnet3-chain-get-egs.cc: handles long keys
Srikanth MADIKERI
committed
egs/minilibrespeech/s5b/local/chain/tuning/run_tdnn_2c.sh : the working tdnn recipe for this egs.
Srikanth MADIKERI
committed
adding RESULTS file to egs/mini_librispeech/s5b
Srikanth MADIKERI
committed
adding swbd/s5c/local/chain/tuning/run_tdnn_8k.sh to test it with new kaldi recipe
Srikanth MADIKERI
committed
egs/swbd/s5c/local/chain/tuning/run_tdnn_8k.sh: updated tdnn script to run train2.sh
Srikanth MADIKERI
committed
commiting the version of run_tdnn_8k.sh that finish training finally!
Srikanth MADIKERI
committed
removed train.py from run_tdnn_8k.sh
Srikanth MADIKERI
committed
modifying egs/wsj/s5/steps/nnet3/chain/train2.sh to include lrate. I
Srikanth MADIKERI
committed
running v3 of egs/swbd/s5c/local/chain/tuning/run_tdnn_8k.sh which uses
Srikanth MADIKERI
committed
small modification to egs/swbd/s5c/local/chain/tuning/run_tdnn_8k.sh:
Srikanth MADIKERI
committed
fixed a stupid bug while in run_tdnn_8k.sh while checking if common_egs_dir is empty (I was doing it the other way round)
Srikanth MADIKERI
committed
fixed another bug in run_tdnn_8k.sh: wasn't really using common_egs_dir after setting it correctly.
Srikanth MADIKERI
committed
starting run_tdnn_8k.sh from stage 17
Srikanth MADIKERI
committed
starting run_tdnn_8k.sh from stage 12
Srikanth MADIKERI
committed
fixed another place in run_tdnn_8k.sh that didn't use the right egs dir
Srikanth MADIKERI
committed
silly mistake in the previous commit
Srikanth MADIKERI
committed
train2.sh: adding model_dir_in explicitly to the train command
Srikanth MADIKERI
committed
train2.sh: fixing --lrate to --learning-rate
Srikanth MADIKERI
committed
removing --raw=true options to nnet3-am-copy in train2.sh
Srikanth MADIKERI
committed
train2.sh: nnet3-am-copy is creating troubles becaues nne3-chain-train2 doesn't write models. it writes raw nnets. so I'm now using nnet3-am-init as a workaround
Srikanth MADIKERI
committed
nnet-chain-training2.cc: TrainInternal call updated
Srikanth MADIKERI
committed
committing soft links for swbd/s5d recipe
Srikanth MADIKERI
committed
s5d/RESULTS added
Srikanth MADIKERI
committed
stable version of tdnn_7k and tdnn_8k committed
Srikanth MADIKERI
committed
src/nnet3/nnet-chain-training2.h: removed const qualifier for chain example
Srikanth MADIKERI
committed
egs/wsj/s5/steps/diagnostic/analyze_lats.sh: disabling analyze lats. crashes on the grid at Idiap
Srikanth MADIKERI
committed
egs/wsj/s5/steps/nnet3/chain/compute_preconditioning_matrix.sh: stats file clean-up added
Srikanth MADIKERI
committed
egs/wsj/s5/steps/nnet3/chain/internal/get_train_schedule.py: different frame_shift computation
Srikanth MADIKERI
committed
egs/wsj/s5/steps/nnet3/chain/randomize_egs.sh: shuffles and splits into a number of scp files to randomize the egs
Srikanth MADIKERI
committed
egs/wsj/s5/steps/nnet3/chain/train2.sh: updated with the following changes
Srikanth MADIKERI
committed
adding nnet3/nnet-chain-diagnostics2.h and chainbin/nnet3-chain-combine2.cc
Srikanth MADIKERI
committed
moved code for the new training scripts from steps/nnet3/chain/ to steps/nnet3/chain2/
Srikanth MADIKERI
committed
added soft link to steps/nnet3/chain2 in steps/
Srikanth MADIKERI
committed
modified paths to other scripts in steps/nnet3/chain2/train2.sh
Srikanth MADIKERI
committed
renamed steps/nnet3/chain2/train2.sh to steps/nnet3/chain2/train.sh
Srikanth MADIKERI
committed
added new script to local/chain2/tuning. earlier this recipe was in s5b/local/.., but there is no need to have a new recipe
Srikanth MADIKERI
committed
added softlink to best tdnn recipe for minilibrespeech with new kaldi scripts
Srikanth MADIKERI
committed
moved local/chain/tuning/run_tdnn_8k.sh to local/chain2/tuning/run_tdnn_7k.sh
Srikanth MADIKERI
committed
reverting back steps/diagnostic/analyze_alignments.sh
Srikanth MADIKERI
committed
reverting analyze_lats
Srikanth MADIKERI
committed
removing steps/nnet3/chain/data_prep_common.sh. it doesn't belong in steps
Srikanth MADIKERI
committed
removing s5d
Srikanth MADIKERI
committed
RESULTS for swbd added
Srikanth MADIKERI
committed
minilibrespeech RESULTS integrated into s5 recipe
Srikanth MADIKERI
committed
- added data_prep_common.sh to local/chain2
Srikanth MADIKERI
committed
removing the configuration section in local/chain/tuning/run_tdnn_7k.sh
Srikanth MADIKERI
committed
removed safe-checking graphs in local/chain/tuning/run_tdnn_7k.sh. It is now the same as what is there in master
Srikanth MADIKERI
committed
setting graph_dir value (reverting the code in parts)
Srikanth MADIKERI
committed
one more rollback to local/chain/tuning/run_tdnn_7k.sh: removing graph_dir from options
Srikanth MADIKERI
committed
modifying local/chain2/tuning/run_tdnn_7k.sh to be identical to local/chain2/tuning/run_tdnn_7k.sh
Srikanth MADIKERI
committed
reverting the following files to master (to reduce the number of files to be reviewed for PR). these files originally had changes adapted to the Idiap environment:
Srikanth MADIKERI
committed
reverting 2 more files to match master:
Srikanth MADIKERI
committed
removing egs/wsj/s5/steps/nnet3/chain/internal/choose_egs_to_merge.py from the commits, it was there in the chaina branch, but it is no longer useful
Srikanth MADIKERI
committed
added copyright headers to code modified in mini_librespeech recipe
Srikanth MADIKERI
committed
added copyright header to recipe in swbd
Srikanth MADIKERI
committed
copyright added to source code in nnet3 and chainbin
Srikanth MADIKERI
committed
added run_tdnn.sh for chain2 in egs/babel
Srikanth MADIKERI
committed
maded egs/babel/s5d/local/chain2/run_tdnn.sh an executable
Srikanth MADIKERI
committed
tdnn recipe for babel added
Srikanth MADIKERI
committed
merging changes after adding copyright headers
Srikanth MADIKERI
committed