Skip to content

fix(sft): expose filtered row reasons - #468

Open
lsr-05 wants to merge 1 commit into
inclusionAI:mainfrom
lsr-05:codex/sft-filter-diagnostics
Open

fix(sft): expose filtered row reasons#468
lsr-05 wants to merge 1 commit into
inclusionAI:mainfrom
lsr-05:codex/sft-filter-diagnostics

Conversation

@lsr-05

@lsr-05 lsr-05 commented Aug 11, 2026

Copy link
Copy Markdown

Summary

SFT now reports why rows were filtered when no trainable rows remain.

  • count empty prompts/responses, all-prompt rows, and prompt/response budget violations separately
  • include the counts in the existing ValueError and filter log
  • cover empty datasets and mixed all-invalid datasets in the CPU suite
  • keep _record_to_train_sequence behavior unchanged for existing callers

This is a follow-up to #90.

Validation

  • python -m pytest -q tests/test_trainer_dataset_utils_cpu.py tests/test_qwen35_multimodal_cpu.py (26 passed, 11 skipped)
  • ruff check areno/api/trainers/sft.py tests/test_trainer_dataset_utils_cpu.py

@lsr-05

lsr-05 commented Aug 11, 2026

Copy link
Copy Markdown
Author

#79

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant