feat: integrate local ONNX inference for offline plant disease detection#67
feat: integrate local ONNX inference for offline plant disease detection#67vedant-kawale-27 wants to merge 2 commits into
Conversation
|
@vedant-kawale-27 is attempting to deploy a commit to the karan3431's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
🎉 Thanks for your contribution, @vedant-kawale-27! Please make sure CI passes and the checklist in the PR template is complete. A maintainer will review this soon. — The AgroNavis team |
|
Warning Review limit reached
More reviews will be available in 37 minutes and 48 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughA backend script exports a fine-tuned ResNet18 PyTorch model to ONNX (with optional 8-bit quantization). The frontend gains ChangesOn-Device ONNX Inference Pipeline
Sequence Diagram(s)sequenceDiagram
participant Farmer
participant CropScanTab
participant diagnoseImage as diagnoseImage (Server)
participant runLocalONNXInference as runLocalONNXInference (WASM)
participant addOfflineCropScan as IndexedDB
Farmer->>CropScanTab: Upload image & tap Scan
CropScanTab->>CropScanTab: Check navigator.onLine
alt Online
CropScanTab->>diagnoseImage: POST image
diagnoseImage-->>CropScanTab: DiagnosisResult (scanSource=server)
else Offline or server error
CropScanTab->>runLocalONNXInference: file
runLocalONNXInference-->>CropScanTab: LocalDiagnosisResult (scanSource=local)
CropScanTab->>addOfflineCropScan: OfflineCropScan record
addOfflineCropScan-->>CropScanTab: saved to IndexedDB
end
CropScanTab-->>Farmer: Display result + source badge
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (1)
frontend/src/utils/onnxInference.ts (1)
18-46: ⚡ Quick winPrevent duplicate ONNX session creation during concurrent first calls.
sessionis cached only aftercreate(...)resolves. Two parallel calls before resolution can initialize multiple sessions.Proposed patch
let session: any = null; +let sessionPromise: Promise<any> | null = null; @@ async function getInferenceSession(): Promise<any> { if (session) { return session; } + if (sessionPromise) { + return sessionPromise; + } @@ - try { - const ort = await import('onnxruntime-web'); + try { + sessionPromise = (async () => { + const ort = await import('onnxruntime-web'); @@ - session = await ort.InferenceSession.create('/model/plant_disease_resnet18.onnx', { - executionProviders: ['wasm'] - }); - console.log('[LocalInference] ONNX Inference Session loaded successfully.'); - return session; + session = await ort.InferenceSession.create('/model/plant_disease_resnet18.onnx', { + executionProviders: ['wasm'] + }); + console.log('[LocalInference] ONNX Inference Session loaded successfully.'); + return session; + })(); + return await sessionPromise; } catch (error) { + sessionPromise = null; console.error('[LocalInference] Failed to load ONNX session:', error); throw new Error('Could not initialize the local disease scanner model.'); } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/src/utils/onnxInference.ts` around lines 18 - 46, The race condition occurs because the `session` variable is only assigned after the async create call completes, allowing concurrent calls to getInferenceSession to all see session as null and each initiate their own session creation. Instead of caching only the resolved session, you need to cache the promise itself. Create a new variable to hold the session creation promise (separate from the resolved session variable), store the promise from ort.InferenceSession.create() immediately before awaiting it, and return the cached promise on subsequent calls so they wait for the same in-flight creation rather than starting new ones.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/export_to_onnx.py`:
- Around line 39-55: Replace the else block that provides fallback behavior when
fine-tuned weights are missing at MODEL_PATH with a hard-fail guard that raises
an exception or exits the program. Instead of attempting to load torchvision
pretrained weights or exporting with random weights, immediately raise an
informative error when the fine-tuned checkpoint is not found. This ensures the
export process terminates early rather than silently proceeding with weights
that may produce incorrect predictions in production.
In `@frontend/next.config.mjs`:
- Around line 23-37: The Workbox configuration for caching ONNX and WASM files
uses a CacheFirst strategy with hardcoded model paths and a 30-day expiration,
which prevents model updates from being served to users. To fix this, either
modify the model asset filenames to include version numbers or hashes (such as
changing the path pattern to include versioning like v1, v2, or a hash suffix)
in the urlPattern matching logic, or add a cacheVersion property within the
options object of the Workbox configuration that can be incremented whenever
models are updated to force cache invalidation. The cacheVersion mechanism in
Workbox automatically appends a version identifier to the cache name, ensuring
old cached entries are abandoned when the version changes.
In `@frontend/package.json`:
- Line 11: The postinstall script in package.json currently only warns when the
onnxruntime-web WASM files are missing, allowing the build to succeed without
the required WASM files. Since the application hardcodes the WASM path at
runtime, this causes failures that are difficult to debug. Replace the warning
console.warn with a fatal error that exits the process with a non-zero code
using process.exit(1) when the source directory is not found, ensuring the build
fails early and prevents incomplete deployments.
In `@frontend/src/utils/diseaseData.ts`:
- Around line 236-242: The extractCropType function incorrectly parses
classnames with diseased_ prefixes and multi-token crop names. The function
currently returns "Diseased" for diseased_cucumber and "Bell" for
bell_pepper_bacterial_spot instead of the actual crop types. Refactor the
function to properly handle both healthy_ and diseased_ prefixes by extracting
the crop name portion that follows these prefixes, and for multi-token crop
names (like bell_pepper), split by underscore and capitalize each word properly
to return the correct crop type that gets displayed in CropScanTab.
In `@frontend/src/utils/onnxInference.ts`:
- Around line 138-172: Replace the hardcoded input and output names in the feeds
object construction and results retrieval with dynamic resolution using
activeSession.inputNames and activeSession.outputNames properties to decouple
from exporter configuration. Additionally, add validation after retrieving the
outputTensor to ensure the logits array length (output.length) matches
CLASS_NAMES.length, throwing a descriptive error if they do not match to prevent
silent data integrity failures.
---
Nitpick comments:
In `@frontend/src/utils/onnxInference.ts`:
- Around line 18-46: The race condition occurs because the `session` variable is
only assigned after the async create call completes, allowing concurrent calls
to getInferenceSession to all see session as null and each initiate their own
session creation. Instead of caching only the resolved session, you need to
cache the promise itself. Create a new variable to hold the session creation
promise (separate from the resolved session variable), store the promise from
ort.InferenceSession.create() immediately before awaiting it, and return the
cached promise on subsequent calls so they wait for the same in-flight creation
rather than starting new ones.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 6a19da11-4b9b-43b8-a9fe-58eeff05d91d
⛔ Files ignored due to path filters (5)
frontend/package-lock.jsonis excluded by!**/package-lock.jsonfrontend/public/wasm/ort-wasm-simd-threaded.asyncify.wasmis excluded by!**/*.wasmfrontend/public/wasm/ort-wasm-simd-threaded.jsep.wasmis excluded by!**/*.wasmfrontend/public/wasm/ort-wasm-simd-threaded.jspi.wasmis excluded by!**/*.wasmfrontend/public/wasm/ort-wasm-simd-threaded.wasmis excluded by!**/*.wasm
📒 Files selected for processing (7)
backend/export_to_onnx.pyfrontend/next.config.mjsfrontend/package.jsonfrontend/src/components/CropScanTab.tsxfrontend/src/lib/offlineStorage.tsfrontend/src/utils/diseaseData.tsfrontend/src/utils/onnxInference.ts
|
hee @jpdevhub check it |
Summary
Integrates the fine-tuned PyTorch ResNet18 model into the Next.js frontend using onnxruntime-web, enabling instant, offline crop disease scanning directly on the farmer's device.
Related Issue
Closes #51
Changes
onnxruntime-weband configured apostinstallscript in package.json to copy the WebAssembly binaries (.wasm) topublic/wasm/for offline CDN-independent load.import.metacompilation constraints) when packaging WASM..onnxand.wasmfiles locally on first visit (agronavis-ml-assets).⚡ On-Devicevs☁️ Server), and persist local scan entries inside IndexedDB using offlineStorage.ts.Testing
npm run build. Rannpm run test --prefix frontend(all 10 Jest tests successfully passed). Checked local browser fallback scan behavior under offline devtool network emulation.Checklist
.envvalues are committedSummary by CodeRabbit