Go to v2.3.28#256
Merged
Merged
Conversation
#250) * feat: Implement Analyzer and AnalyzerMapping models with associated serializers and views - Added Analyzer model with fields for name, type, enabled status, config, and description. - Created AnalyzerMapping model with foreign key to Analyzer and fields for mapping_from and mapping_to. - Developed serializers for both models to handle validation and representation. - Introduced viewsets for Analyzer and AnalyzerMapping, including custom actions for testing connections and retrieving vulnerability choices. - Updated event retesting functionality to support analyzer mappings, allowing for dynamic selection of analyzers during event retests. - Enhanced frontend component to utilize dropdown for selecting analyzers when retesting events. - Added filters for Analyzer and AnalyzerMapping models to improve query capabilities. - Implemented migrations for new models and relationships, ensuring database integrity. - Created unit tests for Analyzer and AnalyzerMapping models to validate functionality and constraints. Co-authored-by: Copilot <copilot@github.com> * Potential fix for pull request finding 'CodeQL / Information exposure through an exception' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * Update ngen/analyzers/kintun.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update frontend/src/views/analyzer/components/FormAnalyzer.jsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * feat: Make analyzer selection required in Create and Edit Analyzer Mapping forms; update KintunAdapter to disable redirects and improve error logging Co-authored-by: Copilot <copilot@github.com> * feat: Refactor analyzer configuration to include port and SSL options; update translations and form handling --------- Co-authored-by: Copilot <copilot@github.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: agrega filtro select url * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
#254) * avoid raise condition con sending email inside atomic transaction * better traceback * tests fix for transaction on_commit
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces first-class “Analyzer” support across the backend and frontend (models, API, UI), updates event retesting to run through analyzer adapters/mappings, and adds evidence-added email notifications. It also includes localization cleanup/updates and increases Nginx upload limits.
Changes:
- Add Analyzer/AnalyzerMapping-backed adapter architecture (models, migrations, serializers, viewsets, filters) and UI screens to manage analyzers/mappings.
- Update event retest flow to select an analyzer mapping and execute via the configured analyzer adapter.
- Add “new evidence added” email templates/notifications, update async email dispatch behavior, and adjust tests; bump Nginx
client_max_body_sizeto 100M.
Reviewed changes
Copilot reviewed 60 out of 62 changed files in this pull request and generated 16 comments.
Show a summary per file
| File | Description |
|---|---|
| project/urls.py | Registers the new analyzer API route. |
| ngen/views/case.py | Extends event retest endpoint to accept an analyzer mapping reference and pass it to Celery. |
| ngen/views/analyzer.py | New Analyzer CRUD API plus connection test and vuln-choices endpoints. |
| ngen/views/analyzer_mapping.py | Tightens permissions and updates search fields for new analyzer FK. |
| ngen/views/init.py | Exposes new Analyzer viewset module. |
| ngen/tests/models/test_announcement.py | Updates expectations for increased email messages in channels. |
| ngen/tests/models/test_analyzer.py | Adds unit tests for Analyzer/AnalyzerMapping validation and adapter behavior. |
| ngen/tests/api/test_events.py | Adds multipart evidence upload tests; updates notes text. |
| ngen/tests/api/test_emailmessage.py | Ensures on-commit callbacks execute in tests for async email dispatch. |
| ngen/tests/api/test_communication_channel_communicate.py | Ensures on-commit callbacks execute in communicate tests. |
| ngen/templates/reports/new_evidence_added_to_event.html | New email template for evidence added to an event. |
| ngen/templates/reports/new_evidence_added_to_case.html | New email template for evidence added to a case. |
| ngen/tasks.py | Generalizes retest task to use analyzer adapters/mappings; improves async email logging. |
| ngen/serializers/analyzer.py | Adds Analyzer serializer with schema-based validation. |
| ngen/serializers/analyzer_mapping.py | Adds derived analyzer name/type fields to mapping serialization. |
| ngen/serializers/init.py | Exposes new Analyzer serializer module. |
| ngen/models/case.py | Triggers case communications when new evidence is created. |
| ngen/models/analyzer.py | New Analyzer model with registry-based type/config validation. |
| ngen/models/analyzer_mapping.py | Replaces analyzer_type with FK to Analyzer and updates __str__. |
| ngen/models/init.py | Exposes new Analyzer model module. |
| ngen/migrations/0024_analyzer_model.py | Creates Analyzer table. |
| ngen/migrations/0025_analyzermapping_add_analyzer_fk.py | Adds nullable analyzer FK to mappings and removes analyzer_type. |
| ngen/migrations/0026_analyzermapping_cascade_analyzer.py | Deletes orphaned mappings and enforces CASCADE/non-null analyzer FK. |
| ngen/mailer/email_handler.py | Adds new evidence templates, improves invalid template error, and defers Celery send to on_commit. |
| ngen/locale/es/LC_MESSAGES/django.po | Updates Spanish catalog with new strings and line shifts. |
| ngen/filters.py | Adds AnalyzerFilter; updates mapping filter to use analyzer FK fields. |
| ngen/analyzers/registry.py | New adapter registry + schema/choices helpers. |
| ngen/analyzers/kintun.py | New Kintun adapter implementation (scan + polling + vuln choice fetch). |
| ngen/analyzers/cortex.py | New Cortex adapter skeleton (connection test). |
| ngen/analyzers/base.py | Introduces shared analyzer adapter base class. |
| frontend/src/views/tools/lookup/FormLookup.jsx | Fixes translation key namespace for lookup errors. |
| frontend/src/views/template/ListTemplate.jsx | Adds controlled filter state for feed/taxonomy filters. |
| frontend/src/views/taxonomyGroup/components/ButtonView.jsx | Updates taxonomy detail i18n key usage. |
| frontend/src/views/taxonomy/ListTaxonomies.jsx | Adds controlled filter state for multiple taxonomy filters. |
| frontend/src/views/taxonomy/components/ButtonView.jsx | Updates taxonomy detail i18n key usage. |
| frontend/src/views/priority/components/FormPriority.jsx | Updates i18n key for “time to solve”. |
| frontend/src/views/network/ListNetwork.jsx | Adds controlled filter state for entity/type filters. |
| frontend/src/views/event/components/SmallRetestTable.jsx | Adds mapping selection dropdown and passes mapping reference to retest API. |
| frontend/src/views/case/ModalListCase.jsx | Resets controlled filter selections when modal closes/filters cleared. |
| frontend/src/views/case/ListCase.jsx | Adds controlled filter state for state filter. |
| frontend/src/views/analyzerMapping/ListAnalyzerMappings.jsx | Forces refresh after delete via refresh key. |
| frontend/src/views/analyzerMapping/EditAnalyzerMapping.jsx | Switches mapping to selecting an analyzer instance; adds vuln choices select. |
| frontend/src/views/analyzerMapping/CreateAnalyzerMapping.jsx | Switches mapping to selecting an analyzer instance; adds vuln choices select. |
| frontend/src/views/analyzerMapping/components/TableAnalyzerMapping.jsx | Displays analyzer name and triggers list refresh after delete. |
| frontend/src/views/analyzer/ListAnalyzers.jsx | New analyzers listing page. |
| frontend/src/views/analyzer/EditAnalyzer.jsx | New analyzer edit page. |
| frontend/src/views/analyzer/CreateAnalyzer.jsx | New analyzer create page. |
| frontend/src/views/analyzer/components/TableAnalyzer.jsx | New analyzer table with detail modal, delete, and connection test. |
| frontend/src/views/analyzer/components/FormAnalyzer.jsx | New analyzer form with schema-driven config fields + sensitive input handling. |
| frontend/src/routes.jsx | Adds analyzer routes with permission guards. |
| frontend/src/menu-items.jsx | Adds “Analyzers” menu entry. |
| frontend/src/config/constant.jsx | Adds analyzer API base path constant. |
| frontend/src/components/Filter/FilterSelectUrl.jsx | Updates selection state handling for optional value/setValue. |
| frontend/src/api/services/eventAnalysis.jsx | Fetches more retests and posts analyzer mapping reference with retest request. |
| frontend/src/api/services/analyzerMapping.jsx | Updates mapping create/update to use analyzer FK and improves duplicate detection. |
| frontend/src/api/services/analyzer.jsx | New analyzer API client service (CRUD + test + vuln choices). |
| frontend/public/locales/es/translation.json | Cleans up keys and adds analyzer/analyzer-mapping strings. |
| frontend/public/locales/en/translation.json | Cleans up keys and adds analyzer/analyzer-mapping strings. |
| frontend/nginx-prod/nginx.conf.template.ssl | Increases client_max_body_size to 100M. |
| frontend/nginx-prod/nginx.conf.template.no-ssl | Increases client_max_body_size to 100M. |
| .gitignore | Ignores CLAUDE.md. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+174
to
180
| retest_event_kintun.delay( | ||
| event_id=event.id, | ||
| analyzer_mapping_id=analyzer_mapping_id, | ||
| ) | ||
| return Response( | ||
| {"message": gettext_lazy(f"Task retest event for {event.pk} launched")}, | ||
| status=status.HTTP_200_OK, |
Comment on lines
+31
to
+39
| def clean(self): | ||
| from ngen.analyzers.registry import ADAPTER_REGISTRY | ||
|
|
||
| if self.type in ADAPTER_REGISTRY: | ||
| adapter_class = ADAPTER_REGISTRY[self.type] | ||
| instance = adapter_class(self) | ||
| errors = instance.validate_config() | ||
| if errors: | ||
| raise ValidationError(errors) |
Comment on lines
+8
to
+20
| class AnalyzerSerializer(AuditSerializerMixin): | ||
| type = serializers.ChoiceField(choices=ANALYZER_TYPE_CHOICES) | ||
| config = serializers.JSONField(default=dict) | ||
|
|
||
| class Meta: | ||
| model = models.Analyzer | ||
| fields = "__all__" | ||
|
|
||
| def validate(self, attrs): | ||
| analyzer_type = attrs.get("type") or (self.instance.type if self.instance else None) | ||
| config = attrs.get("config", {}) | ||
|
|
||
| schema = get_config_schema().get(analyzer_type, {}) |
Comment on lines
+183
to
+187
| sleep(poll_interval) | ||
| poll_interval = min(poll_interval * 2, POLL_INTERVAL_MAX) | ||
| response = requests.get( | ||
| scan_url, headers=self._headers(), auth=self._auth(), timeout=10 | ||
| ) |
Comment on lines
+10
to
+15
| {% block content_body %} | ||
| <p class="lead"> | ||
| {% blocktranslate trimmed with id=case.uuid %} | ||
| A new evidence has been added to the event {{ id }} | ||
| {% endblocktranslate %} | ||
| </p> |
| setSelect={setSelectedAnalyzer} | ||
| options={analyzerOptions} | ||
| value={selectedAnalyzer} | ||
| placeholder={t("ngen.analyzer_mapping.analyzer_type")} |
| setSelect={setSelectedAnalyzer} | ||
| options={analyzerOptions} | ||
| value={selectedAnalyzer} | ||
| placeholder={t("ngen.analyzer_mapping.analyzer_type")} |
Comment on lines
75
to
92
| <tr> | ||
| <th>{t("ngen.analyzer_mapping.mapping_from")}</th> | ||
| <th>{t("ngen.analyzer_mapping.mapping_to")}</th> | ||
| <th>{t("ngen.analyzer_mapping.analyzer_type")}</th> | ||
| <th>{t("ngen.date.created")}</th> | ||
| <th>{t("ngen.options")}</th> | ||
| </tr> | ||
| </thead> | ||
| <tbody> | ||
| {list.map((mapping, index) => { | ||
| const parts = mapping.url.split("/"); | ||
| let itemNumber = parts[parts.length - 2]; | ||
| return ( | ||
| <tr key={index}> | ||
| <td>{taxonomyNames[mapping.mapping_from]}</td> | ||
| <td>{mapping.mapping_to}</td> | ||
| <td>{mapping.analyzer_type}</td> | ||
| <td>{mapping.analyzer_name || "-"}</td> | ||
| <td> |
Comment on lines
+11
to
+15
| <p class="lead"> | ||
| {% blocktranslate trimmed with id=case.uuid %} | ||
| A new evidence has been added to the case {{ id }} | ||
| {% endblocktranslate %} | ||
| </p> |
Comment on lines
+11
to
+14
| <p class="lead"> | ||
| {% blocktranslate trimmed with id=case.uuid %} | ||
| A new evidence has been added to the event {{ id }} | ||
| {% endblocktranslate %} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request primarily updates the Nginx configuration and makes extensive changes to the English translation JSON file, mainly cleaning up unused or redundant translation keys and adding new ones for analyzer-related features. The most significant changes are grouped below.
Nginx Configuration Updates:
client_max_body_sizeto 100M in bothnginx.conf.template.no-sslandnginx.conf.template.sslto allow larger client uploads. [1] [2]Frontend Localization (English) Changes:
frontend/public/locales/en/translation.json, streamlining the translation file and potentially removing dead UI elements. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21]Frontend Localization (Spanish) Changes:
frontend/public/locales/es/translation.jsonto keep it consistent with the English version.These changes help improve maintainability by removing obsolete translations and enhance functionality by supporting new analyzer features and increasing upload limits.