Add TYPO3 v13 Visual Diff Extension for Automated Frontend Regression Testing - #4
Conversation
There was a problem hiding this comment.
Copilot wasn't able to review any files in this pull request.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Co-authored-by: priebera1 <22387877+priebera1@users.noreply.github.com>
|
Missing Backend Module (UI) – please add Great start on the Scheduler + services! To actually see issues in TYPO3 BE, we need a backend module with a read-only grid and detail view. Please add: Module registration ext_localconf.php \TYPO3\CMS\Extbase\Utility\ExtensionUtility::registerModule( Controllers & routes Classes/Controller/ResultsController.php indexAction() – paginated list of issues only (unresolved first), filters: URL contains/regex, threshold range, viewport. showAction($jobId, $url) – before/after slider + diff overlay; show metrics (percent, changedPixels). resolveAction() (AJAX, returns JSON) – toggle resolved. exportAction() – CSV/JSON honoring current filters. Configuration/Backend/Routes.php for AJAX endpoints (/visualdiff/resolve, /visualdiff/export) returning JsonResponse. Templates & assets Resources/Private/Templates/Results/Index.html, Show.html (Fluid). Resources/Public/Js/visualdiff.js (small ES6 to handle filters, AJAX resolve, fullscreen). Resources/Public/Css/visualdiff.css (simple grid + slider). Image streaming (BE-safe) Labels & icon Resources/Private/Language/locallang_mod.xlf Resources/Public/Icons/module.svg Read model for UI Health checks in UI Acceptance criteria I can open System → Visual Diff and see only pages with differences. Clicking an item opens a detail with before/after slider + diff.png. “Mark resolved” toggles via AJAX and persists to job.json. Export CSV/JSON respects the current filters. No public access to images; all served via controller. Once this is in, we can iterate on filters and nicer thumbnails. |
Co-authored-by: priebera1 <22387877+priebera1@users.noreply.github.com>
|
Please add: Module registration ext_localconf.php \TYPO3\CMS\Extbase\Utility\ExtensionUtility::registerModule( Controllers & routes Classes/Controller/ResultsController.php indexAction() – paginated list of issues only (unresolved first), filters: URL contains/regex, threshold range, viewport. showAction($jobId, $url) – before/after slider + diff overlay; show metrics (percent, changedPixels). resolveAction() (AJAX, returns JSON) – toggle resolved. exportAction() – CSV/JSON honoring current filters. Configuration/Backend/Routes.php for AJAX endpoints (/visualdiff/resolve, /visualdiff/export) returning JsonResponse. |
Co-authored-by: priebera1 <22387877+priebera1@users.noreply.github.com>
|
I need add Backend module for showing differences and list for different pages |
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 17 out of 18 changed files in this pull request and generated 5 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| // Build command | ||
| $command = escapeshellcmd($this->binaryPath); | ||
|
|
||
| foreach ($options as $key => $value) { | ||
| $command .= ' ' . escapeshellarg($key); | ||
| if ($value !== '') { | ||
| $command .= ' ' . escapeshellarg($value); | ||
| } | ||
| } | ||
|
|
||
| $command .= ' ' . escapeshellarg($url) . ' ' . escapeshellarg($outputPath); | ||
|
|
||
| // Execute command | ||
| $output = []; | ||
| $returnCode = 0; | ||
| exec($command . ' 2>&1', $output, $returnCode); |
There was a problem hiding this comment.
The exec() call could be vulnerable to command injection if URL parameters contain shell metacharacters. Consider using proc_open() or additional input sanitization.
| // Build command | |
| $command = escapeshellcmd($this->binaryPath); | |
| foreach ($options as $key => $value) { | |
| $command .= ' ' . escapeshellarg($key); | |
| if ($value !== '') { | |
| $command .= ' ' . escapeshellarg($value); | |
| } | |
| } | |
| $command .= ' ' . escapeshellarg($url) . ' ' . escapeshellarg($outputPath); | |
| // Execute command | |
| $output = []; | |
| $returnCode = 0; | |
| exec($command . ' 2>&1', $output, $returnCode); | |
| // Build command as array to avoid shell injection | |
| $cmd = [$this->binaryPath]; | |
| foreach ($options as $key => $value) { | |
| $cmd[] = $key; | |
| if ($value !== '') { | |
| $cmd[] = $value; | |
| } | |
| } | |
| $cmd[] = $url; | |
| $cmd[] = $outputPath; | |
| // Execute command safely using proc_open | |
| $descriptorspec = [ | |
| 1 => ['pipe', 'w'], // stdout | |
| 2 => ['pipe', 'w'], // stderr | |
| ]; | |
| $process = proc_open($cmd, $descriptorspec, $pipes); | |
| $output = []; | |
| $returnCode = 0; | |
| if (is_resource($process)) { | |
| $stdout = stream_get_contents($pipes[1]); | |
| fclose($pipes[1]); | |
| $stderr = stream_get_contents($pipes[2]); | |
| fclose($pipes[2]); | |
| $returnCode = proc_close($process); | |
| if ($stdout !== false && $stdout !== '') { | |
| $output[] = $stdout; | |
| } | |
| if ($stderr !== false && $stderr !== '') { | |
| $output[] = $stderr; | |
| } | |
| } else { | |
| throw new \RuntimeException('Failed to start wkhtmltoimage process.'); | |
| } |
| $urlA = rtrim($baseUrlA, '/') . '/' . ltrim($url, '/'); | ||
| $urlB = rtrim($baseUrlB, '/') . '/' . ltrim($url, '/'); |
There was a problem hiding this comment.
URL construction without validation could lead to SSRF attacks. Consider implementing URL validation and host whitelisting before making HTTP requests.
|
|
||
| // Ensure base directory exists | ||
| if (!is_dir($this->baseStoragePath)) { | ||
| mkdir($this->baseStoragePath, 0755, true); |
There was a problem hiding this comment.
mkdir() should have error handling as it can fail due to permissions. Consider wrapping in try-catch or checking return value with descriptive error message.
| mkdir($this->baseStoragePath, 0755, true); | |
| if (!mkdir($this->baseStoragePath, 0755, true) && !is_dir($this->baseStoragePath)) { | |
| throw new \RuntimeException( | |
| sprintf('Failed to create base storage directory "%s". Please check permissions.', $this->baseStoragePath) | |
| ); | |
| } |
| $diffImage = imagecreatetruecolor($maxWidth, $maxHeight); | ||
|
|
||
| if ($diffImage === false) { | ||
| throw new \RuntimeException('Failed to create diff image'); |
There was a problem hiding this comment.
The error message 'Failed to create diff image' is too generic. Consider including dimensions or memory usage information to help diagnose the root cause.
| throw new \RuntimeException('Failed to create diff image'); | |
| // Estimate memory usage: 4 bytes per pixel for truecolor images (RGBA) | |
| $estimatedMemory = $maxWidth * $maxHeight * 4; | |
| $estimatedMemoryMB = round($estimatedMemory / (1024 * 1024), 2); | |
| throw new \RuntimeException( | |
| sprintf( | |
| 'Failed to create diff image (dimensions: %dx%d, estimated memory: %.2f MB)', | |
| $maxWidth, | |
| $maxHeight, | |
| $estimatedMemoryMB | |
| ) | |
| ); |
| protected function logMessage(string $message, int $severity = 0): void | ||
| { | ||
| // Severity: 0 = info, 1 = warning, 2 = error | ||
| if ($severity === 0) { | ||
| echo '[INFO] ' . $message . PHP_EOL; | ||
| } elseif ($severity === 1) { | ||
| echo '[WARNING] ' . $message . PHP_EOL; | ||
| } else { | ||
| echo '[ERROR] ' . $message . PHP_EOL; | ||
| } | ||
| } |
There was a problem hiding this comment.
Using echo for logging in a Scheduler task bypasses TYPO3's logging system. Consider using TYPO3's Logger API for proper log management and integration.
|
Add backend module, which show all results/diffs from scheduler/task |
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| /** | ||
| * Path to wkhtmltoimage binary | ||
| */ | ||
| protected string $binaryPath = '/usr/bin/wkhtmltoimage'; |
There was a problem hiding this comment.
The hardcoded binary path should be configurable. Consider using TYPO3's configuration system or environment variables to allow different installation paths.
| } | ||
|
|
||
| // Compare images | ||
| $result = $imageA->compareImages($imageB, \Imagick::METRIC_MEANSQUAREERROR); |
There was a problem hiding this comment.
The hardcoded metric should be configurable. Different comparison metrics (MEANSQUAREERROR, ABSOLUTEERRORMETRIC, etc.) may be more appropriate for different use cases.
priebera1
left a comment
priebera1
left a comment
There was a problem hiding this comment.
I need add a new Backend Module Controller which shows different pages then i can check it in backend
priebera1
left a comment
priebera1
left a comment
There was a problem hiding this comment.
I need add backend module which shows diffenret pages than i can check it
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 17 out of 18 changed files in this pull request and generated 3 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| for ($x = 0; $x < $maxWidth; $x++) { | ||
| for ($y = 0; $y < $maxHeight; $y++) { |
There was a problem hiding this comment.
Nested loops comparing every pixel will be extremely slow for large images (1920x1080 = 2M+ iterations). Consider implementing a sampling approach or optimizing the comparison algorithm for better performance.
| // Remove special characters | ||
| $name = preg_replace('/[^a-zA-Z0-9_-]/', '', $name); | ||
|
|
||
| // If empty, use 'index' | ||
| if (empty($name)) { | ||
| $name = 'index'; | ||
| } | ||
|
|
There was a problem hiding this comment.
The regex removes dots and other valid filename characters, which could cause filename collisions. For example, both 'page.html' and 'pagehtml' would become 'pagehtml'. Consider allowing dots or using a hash-based approach for uniqueness.
| // Remove special characters | |
| $name = preg_replace('/[^a-zA-Z0-9_-]/', '', $name); | |
| // If empty, use 'index' | |
| if (empty($name)) { | |
| $name = 'index'; | |
| } | |
| // Remove special characters but allow dots (to preserve extensions) | |
| $name = preg_replace('/[^a-zA-Z0-9_.-]/', '', $name); | |
| // If empty, use 'index' | |
| if (empty($name)) { | |
| $name = 'index'; | |
| } | |
| // Append a short hash for uniqueness | |
| $hash = substr(md5($name), 0, 8); | |
| $name .= '_' . $hash; |
| "typo3/cms-scheduler": "^13.0", | ||
| "ext-imagick": "*" | ||
| }, |
There was a problem hiding this comment.
Using wildcard version constraint for extensions can lead to compatibility issues. Consider specifying a more specific version range or making it optional since the code has GD fallback.
| "typo3/cms-scheduler": "^13.0", | |
| "ext-imagick": "*" | |
| }, | |
| "typo3/cms-scheduler": "^13.0" | |
| }, | |
| "suggest": { | |
| "ext-imagick": "For improved image processing performance. The extension will be used if available; otherwise, GD will be used as a fallback." | |
| }, |
Overview
This PR implements a complete TYPO3 v13 extension (
devsk_visualdiff) for automated visual regression testing. The extension compares two website bases (A and B) and identifies pages with frontend differences, providing an issues-only reporting approach that highlights only pages with broken or changed frontends.Key Features
Architecture
wkhtmltoimagefor HTML→PNG rendering (no Chromium/Node.js required)var/visual-diff/job-*with structured JSON metadataFunctionality
Implementation Details
Core Components
Scheduler Task (
VisualDiffTask)Service Layer (
VisualDiffService)Utility Classes
ImageRenderer: Wrapswkhtmltoimagewith sensible defaults (1920x1080, 2s JS delay)ImageComparator: Performs pixel-based comparison using Imagick (primary) or GD (fallback)StorageUtility: Manages file system operations and job persistenceUsage Example
Output Structure
Each job produces a
job.jsonfile containing:{ "jobId": "job-20240101120000-abc12345", "status": "completed", "results": [ { "url": "/products", "hasDifference": true, "differencePercentage": 5.2, "imagePathA": "var/visual-diff/.../images/A/products.png", "imagePathB": "var/visual-diff/.../images/B/products.png", "diffImagePath": "var/visual-diff/.../diffs/products.png" } ] }Documentation
Comprehensive documentation included:
Requirements
wkhtmltoimagebinary (apt-get install wkhtmltopdf)Quality Assurance
Installation
Then configure via System → Scheduler → Create new task → Visual Diff Comparison
Resolves the requirements for visual regression testing without Chromium/Node.js dependencies.
Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.