Feature: Auto-extract QA sheet data from uploaded image using AI Vision

[dwarakavikraman]

Problem

Quality inspectors fill out a detailed paper Quality Approval Sheet during inspections. They photograph it and upload to the upload_quality_approval_sheet field in Essdee Quality Inspection. However, only summary data gets manually entered into the form:

• Total major/minor defect counts — but not the individual defect breakdown (e.g., "Uncut Thread: 7 major, Open Seam: 5 major")
• Brief remarks — truncated, missing measurement deviations and specific findings
• Measurement deviations — rarely filled in the form, but clearly written on the paper

The paper sheet contains 3× more data than what reaches the system. This means we cannot do defect pattern analysis, supplier-specific defect tracking, or measurement trend analysis from the digital data alone.

Example: QI Sheet vs Digital Data

Data Point	On Paper Sheet	In System
Defect breakdown (Uncut Thread: 7, Open Seam: 5, Holes: 2...)	✅ Full list with counts	❌ Only total: "Major: 19"
Measurement deviations (Chest: -1cm, Neck: +1.5cm)	✅ Specific measurements	❌ Empty or partial
SPI (Stitches Per Inch)	✅ Noted on sheet	❌ Field exists, rarely filled
Colour / sizes inspected	✅ Listed	❌ Child table sparse

Proposed Solution

When an image is uploaded to upload_quality_approval_sheet, automatically extract structured data from the image using an AI Vision API and populate the form fields.

Flow

Inspector uploads photo of QA sheet
  → on_change hook on upload_quality_approval_sheet field
  → Send image to AI Vision API
  → Parse response into structured fields
  → Auto-populate:
     - New: QI Defect Detail child table
     - Existing: measurement_deviation field
     - Existing: remarks_and_suggestions (full AI summary)
     - Existing: spi_found
  → Inspector reviews auto-populated data → submits

What needs to be built

1. New Master Doctype: Defect Type

Predefined, extensible list of defect categories:

• Uncut Thread
• Open Seam
• Broken Stitch
• Skip Stitch
• Raw Edge / Bottom Raw Edge
• Holes
• Oil Stain
• Bottom Shape Out / Hiking
• Neck Shape Out
• Printing Mistake / Slanting
• Fusing Sticker Slanting
• Shade Variation
• Puckering
• Measurement Deviation
• (extensible — add more as needed)

2. New Child Doctype: Essdee Quality Inspection Defect Detail

Fields:

Field	Type	Description
defect_type	Link → Defect Type	Category of defect
major_count	Int	Number of major defects of this type
minor_count	Int	Number of minor defects of this type

Parent: Essdee Quality Inspection, parentfield: defect_details

3. Server-side hook on image upload

• Trigger: when upload_quality_approval_sheet is set/changed (client script or server hook)
• Action: Call AI Vision API with the image
• Prompt: Extract all defect types with major/minor counts, measurement deviations, SPI, and generate a summary
• Parse the AI response and populate:
  • defect_details child table (new)
  • measurement_deviation (existing Small Text field)
  • remarks_and_suggestions (existing Small Text field — AI-generated summary)
  • spi_found (existing Int field)
• Show a "Review & Confirm" step before saving (optional but recommended)

4. AI API Integration

• Use OpenAI Vision API (gpt-4o) or Claude Vision API
• Single API call per image
• Structured JSON output prompt for reliable parsing
• Estimated cost: ~$0.01 per image (low resolution sufficient)

What does NOT change

• Inspector workflow: take photo → upload → submit (same as today)
• Paper form: no changes needed
• Existing fields: all preserved, just auto-populated instead of manual
• Zero extra data entry for inspectors

Expected Impact

• Defect pattern analysis becomes possible — track which defect types are increasing across suppliers
• Supplier quality scoring with defect-type granularity
• Measurement trend tracking — catch systematic pattern/cutting issues
• Time saved: ~2-3 minutes per QI on manual data entry

Reference

• Doctype: production_api/production_api/doctype/essdee_quality_inspection/
• Image field: upload_quality_approval_sheet (Attach Image)
• Existing fields that should be auto-filled: measurement_deviation, remarks_and_suggestions, spi_found
• Sample QI images available in production data (e.g., EQI-2526-00396, EQI-2526-00395, EQI-2526-00406)