Implement PPL evaluator for qwen 3 0.6B

[zhenchaoni]

Add teacher-forced perplexity evaluation for genai causal LMs

Summary

Adds a text-generation eval task to winml eval that computes teacher-forced, disjoint fixed-length perplexity (PPL) for autoregressive causal LMs. The same evaluator scores both onnxruntime-genai bundles (ONNX, on-device) and PyTorch fp32 baselines through a single model-agnostic contract, so ONNX-vs-baseline accuracy regression can be tracked in the existing e2e harness.

Validated on Qwen/Qwen3-0.6B against wikitext-2-raw-v1 (test split):

• ONNX genai bundle PPL = 25.20
• PyTorch fp32 baseline PPL = 23.65

What's included

Evaluator (eval/text_generation_evaluator.py)

• WinMLTextGenerationEvaluator: model-agnostic, zero-branch. Builds a token corpus, splits it into non-overlapping seqlen blocks, and computes PPL via a numerically stable log-sum-exp NLL accumulator.
• Relies on a uniform scoring contract — encode(text) -> list[int] and forward(ids) -> out where out.logits is (1, len(ids)-1, vocab) with row i predicting token i+1. No per-model conditionals.

Model layer (models/winml/genai_causal_lm.py)

• WinMLGenaiCausalLM: causal-LM inference over an onnxruntime-genai bundle directory (loaded as-is, compile=False — building/compiling is winml build's job).
• HFCausalLM: PyTorch fp32 adapter implementing the same contract (lazy torch/transformers import, add_special_tokens=False to reproduce POC tokenization). Lives in src so the baseline script and the evaluator share one code path.
• CausalLMOutput shared output dataclass.

Wiring

• eval/evaluate.py: registers the text-generation evaluator, a default wikitext-2-raw-v1 dataset, and _load_genai_causal_lm (validates the bundle dir has genai_config.json + .onnx files before loading).
• commands/eval.py: _resolve_model_path now routes an existing local directory in -m to model_path (so a genai bundle dir is read from disk instead of being treated as a Hub id).
• utils/eval_utils.py: _TEXT_GENERATION_SCHEMA with input_column (default text), num_tokens (default 8192), seqlen (default 2048).

Baseline / accuracy harness

• run_pytorch_baseline.py: text-generation branch uses the shared HFCausalLM adapter.
• accuracy.py: adds a perplexity compare strategy (delta_relative, warn 0.05 / fail 0.10, lower-is-better).
• testsets/models_with_acc.json + cache/baseline_cache.json: Qwen/Qwen3-0.6B entry with cached fp32 baseline (PPL 23.645425).

Tests

• test_text_generation_evaluator.py (14): block-chunking protocol, corpus loading, uniform-logits → PPL = vocab-size sanity, NLL cross-entropy reference.
• test_genai_causal_lm.py (9): CausalLMOutput, encode/forward contract, float32 conversion, last-row trimming.