[runtime] upgrade openfst to v1.8.5 and WeTextProcessing to v1.2.0#2817
Open
pengzhendong wants to merge 1 commit into
Open
[runtime] upgrade openfst to v1.8.5 and WeTextProcessing to v1.2.0#2817pengzhendong wants to merge 1 commit into
pengzhendong wants to merge 1 commit into
Conversation
1562bb6 to
62bcc1b
Compare
- openfst: switch from kkm000/openfst 1.7.2.1 to csukuangfj/openfst v1.8.5, remove obsolete patch files - WeTextProcessing: pin to v1.2.0 with SOURCE_SUBDIR runtime - gflags/glog: bump to v2.3.0/v0.7.1 for CMake >=3.31 compatibility - Add fst_types.h for missing int32/uint64 type aliases - kaldi-error.h, utils/flags.h: restore DECLARE_* macros after glog's #undef (FST_FLAGS_ prefix for openfst) - kaldi API: MapFst->ArcMapFst, Map->ArcMap, FLAGS_->FST_FLAGS_ - kaldi/CMakeLists.txt: explicit STATIC to avoid .dylib circular dependency on macOS - post_processor.cc: string::npos to std::string::npos - context_graph.cc: remove streaming CHECK incompatible with new openfst Fixes #2816
62bcc1b to
ef00759
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SOURCE_SUBDIR runtimeFixes #2816
Details
Why upgrade
WeTextProcessing v1.2.0 changed
StringCompiler/StringPrinterconstruction to use the defaultTokenType::BYTEparameter. This requires openfst >= 1.8.5 (csukuangfj fork). wenet previously used kkm000/openfst 1.7.2.1 which has no default for the first constructor argument, causing a compile error.openfst upgrade (openfst.cmake)
Switch from kkm000/openfst 1.7.2.1 to csukuangfj/openfst v1.8.5-2026-04-11. The new fork has a proper root CMakeLists.txt, so
FetchContent_MakeAvailableworks directly. Removed the old PATCH_COMMAND since the patched CMakeLists.txt/log.h/flags.h files are no longer needed — the new openfst ships its own CMake build system and usesFST_FLAGS_prefix for flag variables (avoiding gflags conflict).WeTextProcessing v1.2.0 (wetextprocessing.cmake)
Added
SOURCE_SUBDIR runtimesoFetchContent_MakeAvailablefinds CMakeLists.txt in theruntime/subdirectory (the repo root has only Python code). Removed manualadd_subdirectorycalls — the runtime CMakeLists.txt handles them viaadd_subdirectory(utils)andadd_subdirectory(processor).gflags/glog version bumps (gflags.cmake, glog.cmake)
gflags v2.2.2 uses
cmake_minimum_required(VERSION 2.8.12)which is rejected by CMake >= 3.31. Bumped to v2.3.0 and v0.7.1, matching WeTextProcessing v1.2.0's dependency versions.Removed old openfst patches (6 files, -619 lines)
The patch files (log.h, flags.h, flags.cc, CMakeLists.txt) were designed for kkm000/openfst. They commented out openfst's LOG/CHECK macros and included glog/gflags to avoid naming conflicts. No longer needed with the new openfst.
Compatibility layer
int32,int64,uint64etc. The old openfst provided these infst/types.hwhich no longer exists.fst_types.hinstead of<fst/types.h>.google::LogMessage(was previously obtained transitively through the patched openfst). RestoreDECLARE_*macros after glog's logging.h#undefs them at end of file. These macros must useFST_FLAGS_prefix for openfst's own internal flags.DECLARE_*macros same as above. Note:DEFINE_*comes from gflags withFLAGS_prefix, completely separate — wenet's own flag passing is unaffected.kaldi API updates
New openfst renamed several types and uses
FST_FLAGS_prefix for flag variables:MapFst→ArcMapFst,MapFstOptions→ArcMapFstOptions,Map()→ArcMap()FLAGS_fst_field_separator→FST_FLAGS_fst_field_separator,FLAGS_fst_weight_separator→FST_FLAGS_fst_weight_separatorBuild system
STATICforkaldi-utilandkaldi-decoder. The new openfst setsBUILD_SHARED_LIBS=ONon non-Windows, causing kaldi-decoder to build as.dylib. But kaldi-decoder depends onContextGraph::GetNextStatefrom thedecoderlibrary (which links back to kaldi-decoder), creating a circular dependency on macOS.Small fixes
string::npos→std::string::npos(old openfst'susing std::stringis gone).CHECK(...) << "msg"syntax — new openfst'sCHECKreturns void, not a stream.