Summary
colgrep settings --fp32 appears to be ineffective on non-CUDA builds.
The command prints that FP32 is selected, but the implementation clears the persisted fp32 field. On non-CUDA builds, a missing fp32 field means INT8, not FP32.
This makes it hard to force model.onnx instead of model_int8.onnx on CoreML/CPU builds.
Environment
colgrep 1.5.4
- Observed while debugging a CoreML-enabled macOS build:
cargo install colgrep --features "accelerate,coreml" --force
Current Behavior
Running:
prints:
✅ Set model precision to FP32 (full-precision, default)
But the implementation calls config.clear_fp32():
// colgrep/src/commands/config.rs
if fp32 {
config.clear_fp32();
println!("✅ Set model precision to FP32 (full-precision, default)");
changed = true;
}
On non-CUDA builds, Config::use_fp32() treats None as false:
// colgrep/src/config.rs
#[cfg(not(feature = "cuda"))]
{
self.fp32.unwrap_or(false)
}
So after settings --fp32, Config::use_fp32() still resolves to false, which means the index/search path uses INT8:
let quantized = !config.use_fp32();
Expected Behavior
colgrep settings --fp32 should persist a value that makes Config::use_fp32() return true on all builds.
That should force model.onnx instead of model_int8.onnx until the user changes precision again.
Suggested Fix
Change the --fp32 branch to persist Some(true):
if fp32 {
config.set_fp32(true);
println!("✅ Set model precision to FP32 (full-precision)");
changed = true;
}
Keep clear_fp32() only for an explicit reset-to-default action.
If FP32 should be the default on some build profiles and INT8 on others, the reset command can still clear the option. But --fp32 should not clear it, because clearing is not equivalent to forcing FP32 on non-CUDA builds.
Why This Matters
This surfaced while debugging a separate CoreML model-load failure. The misleading --fp32 behavior made it look like precision had been changed when the effective config still selected INT8.
Summary
colgrep settings --fp32appears to be ineffective on non-CUDA builds.The command prints that FP32 is selected, but the implementation clears the persisted
fp32field. On non-CUDA builds, a missingfp32field means INT8, not FP32.This makes it hard to force
model.onnxinstead ofmodel_int8.onnxon CoreML/CPU builds.Environment
colgrep 1.5.4cargo install colgrep --features "accelerate,coreml" --forceCurrent Behavior
Running:
prints:
But the implementation calls
config.clear_fp32():On non-CUDA builds,
Config::use_fp32()treatsNoneasfalse:So after
settings --fp32,Config::use_fp32()still resolves tofalse, which means the index/search path uses INT8:Expected Behavior
colgrep settings --fp32should persist a value that makesConfig::use_fp32()returntrueon all builds.That should force
model.onnxinstead ofmodel_int8.onnxuntil the user changes precision again.Suggested Fix
Change the
--fp32branch to persistSome(true):Keep
clear_fp32()only for an explicit reset-to-default action.If FP32 should be the default on some build profiles and INT8 on others, the reset command can still clear the option. But
--fp32should not clear it, because clearing is not equivalent to forcing FP32 on non-CUDA builds.Why This Matters
This surfaced while debugging a separate CoreML model-load failure. The misleading
--fp32behavior made it look like precision had been changed when the effective config still selected INT8.