Skip to content

feat: add SVD model with normalization, RMSE improved to 0.965#15

Merged
harmandeep2993 merged 5 commits into
mainfrom
feat-loader
Apr 26, 2026
Merged

feat: add SVD model with normalization, RMSE improved to 0.965#15
harmandeep2993 merged 5 commits into
mainfrom
feat-loader

Conversation

@harmandeep2993
Copy link
Copy Markdown
Owner

Summary

Significant model improvement by implementing SVD matrix factorization
with rating normalization, reducing RMSE from 2.52 to 0.965.

Changes

src/models/train.py

  • Added train_svd_model() → SVD with configurable latent factors
  • Added _evaluate_svd() → fast SVD evaluation
  • Updated train_pipeline() → trains both ItemKNN and SVD
  • Compares 6 models total via MLflow experiment tracking

src/models/evaluate.py

  • Added calculate_rmse_svd() → full test set evaluation for SVD
  • Added calculate_rmse_knn() → sampled evaluation for ItemKNN
  • Updated evaluate_pipeline() → handles both model types

src/models/predict.py

  • Added load_svd_artifacts() → loads SVD model + user means
  • Added recommend_movies_svd() → instant SVD recommendations
  • Updated predict_pipeline() → uses SVD, falls back to ItemKNN

src/features/build_features.py

  • Added normalize_matrix() → subtracts user mean from ratings
  • Added save_normalized_matrix() → saves normalized matrix + user means
  • Updated build_features_pipeline() → returns normalized matrix

main.py

  • Updated pipeline to pass normalized_matrix and user_means
  • Clean up debug prints and unused imports

Model Comparison (MLflow)

Model RMSE
ItemKNN K=10 2.72
ItemKNN K=20 2.82
ItemKNN K=50 2.88
SVD 50 factors 1.005 ← best ✅
SVD 100 factors 1.020
SVD 200 factors 1.023

Final Evaluation

  • Full test set (199,708 rows)
  • RMSE: 0.965 ✅
  • Under 1.0 star error on 1-5 scale

@harmandeep2993 harmandeep2993 merged commit 569d34f into main Apr 26, 2026
2 checks passed
@harmandeep2993 harmandeep2993 deleted the feat-loader branch April 26, 2026 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant