Skip to content

colgrep settings --fp32 clears precision config on non-CUDA builds #130

@RBozydar

Description

@RBozydar

Summary

colgrep settings --fp32 appears to be ineffective on non-CUDA builds.

The command prints that FP32 is selected, but the implementation clears the persisted fp32 field. On non-CUDA builds, a missing fp32 field means INT8, not FP32.

This makes it hard to force model.onnx instead of model_int8.onnx on CoreML/CPU builds.

Environment

  • colgrep 1.5.4
  • Observed while debugging a CoreML-enabled macOS build:
cargo install colgrep --features "accelerate,coreml" --force

Current Behavior

Running:

colgrep settings --fp32

prints:

✅ Set model precision to FP32 (full-precision, default)

But the implementation calls config.clear_fp32():

// colgrep/src/commands/config.rs
if fp32 {
    config.clear_fp32();
    println!("✅ Set model precision to FP32 (full-precision, default)");
    changed = true;
}

On non-CUDA builds, Config::use_fp32() treats None as false:

// colgrep/src/config.rs
#[cfg(not(feature = "cuda"))]
{
    self.fp32.unwrap_or(false)
}

So after settings --fp32, Config::use_fp32() still resolves to false, which means the index/search path uses INT8:

let quantized = !config.use_fp32();

Expected Behavior

colgrep settings --fp32 should persist a value that makes Config::use_fp32() return true on all builds.

That should force model.onnx instead of model_int8.onnx until the user changes precision again.

Suggested Fix

Change the --fp32 branch to persist Some(true):

if fp32 {
    config.set_fp32(true);
    println!("✅ Set model precision to FP32 (full-precision)");
    changed = true;
}

Keep clear_fp32() only for an explicit reset-to-default action.

If FP32 should be the default on some build profiles and INT8 on others, the reset command can still clear the option. But --fp32 should not clear it, because clearing is not equivalent to forcing FP32 on non-CUDA builds.

Why This Matters

This surfaced while debugging a separate CoreML model-load failure. The misleading --fp32 behavior made it look like precision had been changed when the effective config still selected INT8.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions