KernelScope Implementation Plan

Decisions Log

Question	Decision
Build tooling	Vite
State management	React Context + hooks (add library only if needed)
Rendering	Three.js for main canvas (pixels as texture on plane)
Pixel inspection	Bottom bar (auto-tracks current pixel, click to pin)
Sample images	CDN-hosted at `kernel-scope.ammaralam.me`
Default state	Pre-loaded with first sample image
Sobel magnitude	`sqrt(Gx² + Gy²)` (accurate)
Worker granularity	Step-by-step capable (supports playback/slo-mo)
Timeline scrubber	Linear (every pixel position)
"Instant" mode	Process all remaining pixels immediately

Phase 1: Project Setup

Initialize Vite + React + TypeScript project
Configure ESLint and Prettier

Set up folder structure:

src/
  components/     # React components
  hooks/          # Custom hooks
  workers/        # Web Worker files
  core/           # Pure algorithms (convolution, kernels)
  types/          # TypeScript interfaces
  constants/      # Kernel presets, config values
  utils/          # Helper functions

Install dependencies:
- three (Three.js)
- @react-three/fiber (React renderer for Three.js)
- @react-three/drei (useful Three.js helpers)
Create basic App shell with placeholder layout

Phase 2: Core Algorithms (No UI)

2.1 Convolution Engine

Define Kernel type (3×3 number matrix + optional divisor + name)
Implement applyKernelToPixel(image, x, y, kernel, edgeMode) → single output value
Implement edge handling modes:
- Zero padding: out-of-bounds → 0
- Mirror: reflect at boundary
- Clamp: repeat edge pixel
Unit tests for applyKernelToPixel with all edge modes

2.2 Preset Kernels

Define Sobel Gx kernel: [[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]]
Define Sobel Gy kernel: [[1, 2, 1], [0, 0, 0], [-1, -2, -1]]
Implement Sobel magnitude: sqrt(Gx² + Gy²) (clamp to 0-255)
Define Gaussian blur kernel: [[1, 2, 1], [2, 4, 2], [1, 2, 1]] with divisor 16
Define Sharpen kernel: [[0, -1, 0], [-1, 5, -1], [0, -1, 0]]
Unit tests for each preset kernel

2.3 Image Utilities

loadImage(url | File) → returns ImageData (grayscale, clamped to 1024×1024)
rgbToGrayscale(imageData) → grayscale ImageData
resizeIfNeeded(imageData, maxDim) → resized ImageData
imageDataToUint8Array(imageData) → flat grayscale array
uint8ArrayToImageData(array, width, height) → ImageData
Unit tests for image utilities

Phase 3: Web Worker

Create convolution.worker.ts

Define message protocol:

// Main → Worker
type WorkerCommand =
  | { type: 'INIT'; image: Uint8Array; width: number; height: number; kernel: Kernel; edgeMode: EdgeMode }
  | { type: 'STEP' }           // Compute next pixel
  | { type: 'STEP_N'; n: number }  // Compute next N pixels
  | { type: 'RUN_TO_END' }     // Compute all remaining pixels
  | { type: 'SEEK'; position: number }  // Jump to position (recompute from start)
  | { type: 'RESET' }          // Go back to position 0

// Worker → Main
type WorkerResponse =
  | { type: 'PROGRESS'; position: number; totalPixels: number; outputPixel: { x: number; y: number; value: number } }
  | { type: 'BATCH_COMPLETE'; position: number; pixels: Array<{ x: number; y: number; value: number }> }
  | { type: 'COMPLETE'; outputImage: Uint8Array }
  | { type: 'READY' }

Implement worker state machine:
- Stores source image, output image (in progress), current position
- On STEP: compute one pixel, send PROGRESS
- On STEP_N: compute N pixels, send BATCH_COMPLETE
- On RUN_TO_END: compute all remaining, send COMPLETE
- On SEEK: recompute from 0 to position (for scrubbing backward)
Create useConvolutionWorker hook:
- Manages worker lifecycle
- Exposes: init(), step(), stepN(), runToEnd(), seek(), reset()
- Exposes state: position, totalPixels, isRunning, outputImage

Phase 4: Three.js Canvas

4.1 Basic Renderer

Create ImageCanvas component using @react-three/fiber
Set up orthographic camera for 2D view
Create plane geometry sized to image dimensions
Apply grayscale image as DataTexture (LUMINANCE format)
Configure texture for pixelated rendering (NearestFilter, no mipmaps)

4.2 Zoom & Pan

Implement zoom via scroll wheel (continuous)
- Store zoom level in state
- Update camera zoom
- Zoom toward mouse cursor position
Implement pan via click-drag
- Track drag state
- Update camera position
Add zoom preset buttons (1×, 4×, 16×, 64×)
Add pixel grid overlay when zoom > threshold (e.g., 8×)
- Thin lines between pixels
- Use LineSegments or shader

4.3 Dual Texture Support

Maintain two textures: originalTexture, outputTexture
Update outputTexture as convolution progresses
Support switching between textures for display

Phase 5: View Modes

5.1 Side-by-Side View

Render two planes: original (left), processed (right)
Sync zoom/pan between both views
Add visual separator between views
Toggle button to switch to single view

5.2 Single View (In-Place Progressive)

Single plane with custom shader:
- Input: originalTexture, outputTexture, currentPosition (raster index)
- For each pixel: if pixelRasterIndex < currentPosition, sample from outputTexture; else sample from originalTexture
- Raster index = y * width + x
Visual boundary at current processing row (horizontal line + highlight on current pixel)
Toggle button to switch to side-by-side

5.3 Sliding Window Highlight

Render 3×3 border overlay at current kernel position
Position updates with convolution progress
Visible in both view modes
Style: semi-transparent colored border

Phase 6: Playback Controls

6.1 Core Controls

Play/Pause button
- Play: start interval calling step() or stepN() based on speed
- Pause: clear interval
Speed slider
- Range: 1 px/sec to 10000 px/sec (log scale)
- "Instant" option at max
Step forward button: advance 1 pixel
Step backward button: seek to position - 1

6.2 Timeline Scrubber

Horizontal slider from 0 to totalPixels
Drag to seek to any position
Show current position / total (e.g., "524,288 / 1,048,576")
Visual progress indicator

6.3 Follow Mode

Toggle checkbox for follow mode
When enabled:
- Camera centers on current kernel position
- Zoom set to show ~50×50 pixel region (adjustable)
Smooth camera transitions (lerp)

Phase 7: UI Layout & Controls Panel

7.1 Layout Shell

Header bar with app name "KernelScope"
Left sidebar (controls panel, ~280px width)
Main canvas area (fills remaining space)
Bottom bar for playback controls

7.2 Image Selection

Dropdown/grid of sample images (thumbnails)
"Upload" button → file picker
- Accept: image/*
- Validate size ≤ 1024×1024 (or auto-resize with warning)
Show current image info: dimensions, filename

7.3 Kernel Selection

Dropdown or button group:
- Sobel Gx
- Sobel Gy
- Sobel Magnitude
- Gaussian Blur
- Sharpen
- Custom
When "Custom" selected, show editable 3×3 grid
Display current kernel matrix (read-only for presets)

7.4 Custom Kernel Editor

3×3 grid of number inputs
Each cell accepts decimal values
"Apply" button to use custom kernel
Optional: "Reset" to clear values

7.5 Edge Handling Selector

Radio buttons or dropdown:
- Zero Padding (default)
- Mirror
- Clamp
Brief tooltip explaining each option

7.6 Options

Checkbox: "Color-coded feedback"
Checkbox: "Follow mode"
View mode toggle: "Side-by-side" | "Single"

Phase 8: Pixel Inspection Panel

Persistent panel (below kernel display or collapsible section)
Click any pixel on canvas → update panel
Display:
- Clicked position (x, y)
- Original value at that position
- Output value (if computed)
- 3×3 neighborhood grid with source values
- 3×3 kernel grid with weights
- 3×3 grid showing element-wise products
- Sum calculation with formula
Visual formatting: aligned grids, clear labels
"No pixel selected" placeholder when empty

Phase 9: Color-Coded Feedback

Track previous and current value for each pixel
When pixel is computed:
- If new > old: flash green
- If new < old: flash red
- If new == old: no flash
Implementation options:
- Shader-based: pass delta values, animate in shader
- Overlay-based: temporary colored squares
Flash duration: ~200ms fade out
Toggle via checkbox in options

Phase 10: Export

"Download" button in sidebar
Convert output texture to canvas
Export as PNG via canvas.toBlob()
Filename: kernelscope-{kernel}-{timestamp}.png
Disabled until convolution complete (or allow partial export)

Phase 11: Educational Content

Tooltips for each kernel option explaining what it does
Brief description panel that updates with selected kernel
Example text:
- Sobel Gx: "Detects vertical edges by computing horizontal gradient"
- Gaussian: "Smooths the image by averaging neighboring pixels with weighted distribution"
Help button (?) in header → modal with overview

Phase 12: Polish & Edge Cases

12.1 Error Handling

Invalid image upload → show error toast
Image too large → auto-resize with notification
Worker crash → show error, allow retry

12.2 Performance

Profile Three.js render loop, optimize if needed
Ensure 60fps during zoom/pan
Test with 1024×1024 images
Throttle scrubber seek operations

12.3 Responsive Behavior

Sidebar collapses on narrow screens
Canvas resizes to fit container
Touch support for mobile (basic pan/zoom)

12.4 Visual Polish

Consistent color scheme
Loading states during image load/processing
Smooth transitions for UI state changes
Cursor changes (grab/grabbing for pan, pointer for buttons)

Phase 13: Testing

Phase 14: Pixel Inspection Relocation & Live Tracking

14.1 Move Pixel Inspection to Bottom Bar

Redesign bottom bar layout: calculation grids on the left, playback controls (scrubber, buttons, speed) on the right
Remove pixel inspection panel from the sidebar
Create compact BottomBar component that houses both calculation display and playback controls side-by-side
Ensure no vertical scrolling is needed — everything fits in a fixed-height bottom bar at standard desktop resolutions
Calculation display shows three compact 3×3 grids inline: neighborhood, kernel weights, element-wise products
Show summary line: pixel position (x, y), original value → output value, sum formula

14.2 Auto-Track Current Convolution Pixel

Pixel inspection auto-updates to show the calculation for the pixel currently being processed (driven by convolution position, not clicks)
When convolution advances (play, step, seek), the calculation display updates to the current pixel
Click-to-pin: clicking a pixel on the canvas pins the inspection to that pixel; a "pin" indicator and "unpin" button are shown
When pinned, inspection stays on the pinned pixel even as convolution continues
Unpinning returns to auto-tracking the current convolution pixel

Phase 15: Fix Color-Coded Feedback (Pixel Flashing)

Debug and fix the color-coded feedback system — flashing does not currently work despite being wired up
Verify that the worker sends previousValue alongside value in PROGRESS and BATCH_COMPLETE responses
Verify that pixelUpdatesRef is populated correctly in useConvolutionWorker and drained each frame by the renderer
Verify the feedback overlay/shader correctly reads deltas and applies green (increase) / red (decrease) coloring
Ensure flash decay (~200-300ms fade out) is visible and smooth
Test color feedback at various playback speeds (1 px/sec, 100 px/sec, instant)
Ensure the "Color-coded feedback" checkbox properly toggles the feature on/off

Sample Images

CDN Base URL: https://kernel-scope.ammaralam.me

Image	URL	Size	Purpose
House	`https://kernel-scope.ammaralam.me/house.png`	512×512	Architecture - clear horizontal/vertical edges for Sobel
Peppers	`https://kernel-scope.ammaralam.me/peppers.png`	512×512	Classic test image - varied textures and curves
Einstein	`https://kernel-scope.ammaralam.me/einstein.png`	490×600	Portrait - edge detection on facial features
Blobs	`https://kernel-scope.ammaralam.me/blobs.png`	512×512	Simple geometric shapes - beginner-friendly
Circuit	`https://kernel-scope.ammaralam.me/circuit.png`	800×263	Fine patterns - shows detail preservation

Default image on load: House (clean edges, good for first-time demonstration)

Dependency Summary

{
  "dependencies": {
    "react": "^18.x",
    "react-dom": "^18.x",
    "three": "^0.160.x",
    "@react-three/fiber": "^8.x",
    "@react-three/drei": "^9.x"
  },
  "devDependencies": {
    "typescript": "^5.x",
    "vite": "^5.x",
    "@vitejs/plugin-react": "^4.x",
    "vitest": "^1.x",
    "@testing-library/react": "^14.x",
    "eslint": "^8.x",
    "prettier": "^3.x"
  }
}

Estimated Task Count

Phase	Tasks
1. Project Setup	5
2. Core Algorithms	14
3. Web Worker	5
4. Three.js Canvas	10
5. View Modes	8
6. Playback Controls	9
7. UI Layout	12
8. Pixel Inspection	4
9. Color Feedback	4
10. Export	3
11. Educational	4
12. Polish	10
13. Testing	8
14. Pixel Inspection Relocation	8
15. Fix Color-Coded Feedback	7
Total	~111 tasks

FilesExpand file tree

IMPLEMENTATION_PLAN.md

Latest commit

History

IMPLEMENTATION_PLAN.md

File metadata and controls

KernelScope Implementation Plan

Decisions Log

Phase 1: Project Setup

Phase 2: Core Algorithms (No UI)

2.1 Convolution Engine

2.2 Preset Kernels

2.3 Image Utilities

Phase 3: Web Worker

Phase 4: Three.js Canvas

4.1 Basic Renderer

4.2 Zoom & Pan

4.3 Dual Texture Support

Phase 5: View Modes

5.1 Side-by-Side View

5.2 Single View (In-Place Progressive)

5.3 Sliding Window Highlight

Phase 6: Playback Controls

6.1 Core Controls

6.2 Timeline Scrubber

6.3 Follow Mode

Phase 7: UI Layout & Controls Panel

7.1 Layout Shell

7.2 Image Selection

7.3 Kernel Selection

7.4 Custom Kernel Editor

7.5 Edge Handling Selector

7.6 Options

Phase 8: Pixel Inspection Panel

Phase 9: Color-Coded Feedback

Phase 10: Export

Phase 11: Educational Content

Phase 12: Polish & Edge Cases

12.1 Error Handling

12.2 Performance

12.3 Responsive Behavior

12.4 Visual Polish

Phase 13: Testing

Phase 14: Pixel Inspection Relocation & Live Tracking

14.1 Move Pixel Inspection to Bottom Bar

14.2 Auto-Track Current Convolution Pixel

Phase 15: Fix Color-Coded Feedback (Pixel Flashing)

Sample Images

Dependency Summary

Estimated Task Count