Skip to content

feat: add job ingestion to index the courses embedding into Qdrant #112

Merged
mhd-hi merged 11 commits into
devfrom
feat/109_job-indexation
Jun 4, 2026
Merged

feat: add job ingestion to index the courses embedding into Qdrant #112
mhd-hi merged 11 commits into
devfrom
feat/109_job-indexation

Conversation

@mhd-hi
Copy link
Copy Markdown
Member

@mhd-hi mhd-hi commented Jun 3, 2026 •
edited
Loading

⁉️ Related Issue

closes #109

📖 Description

  • Pin node to version 22.22.3 for more reliable setup
  • Fix Dockerfile to not expose devDependencies in the image
  • Add job ingestion to index the view courses embedding inside qrant using transformers.js
  • Add batch courses, 30 per batch
  • Add hash to cache courses, making subsequent jobs instant and resource-free
  • Add error handling to handle backend silencly failing

🧪 How Has This Been Tested?

  • tests added

☑️ Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.

🖼️ Screenshots (if useful)

@mhd-hi mhd-hi marked this pull request as ready for review June 3, 2026 20:51
@mhd-hi mhd-hi changed the base branch from main to dev June 3, 2026 20:55
@mhd-hi mhd-hi marked this pull request as draft June 3, 2026 20:55
@mhd-hi mhd-hi added the enhancement New feature or request label Jun 4, 2026
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Jun 4, 2026

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@mhd-hi mhd-hi marked this pull request as ready for review June 4, 2026 18:22
@mhd-hi mhd-hi requested a review from Copilot June 4, 2026 18:27
Copilot AI reviewed Jun 4, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@mhd-hi mhd-hi changed the title test embedding feat: add job ingestion to index the courses embedding into Qdrant Jun 4, 2026
@mhd-hi mhd-hi merged commit 9b3d339 into dev Jun 4, 2026
8 checks passed
@mhd-hi mhd-hi deleted the feat/109_job-indexation branch June 4, 2026 21:11
@mhd-hi mhd-hi mentioned this pull request Jun 4, 2026
Closed
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

Assignees

@mhd-hi mhd-hi

@Jung500 Jung500

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature]: Job d'indexation (index time : PostgreSQL → Qdrant)

3 participants

@mhd-hi @Jung500