feat: LibreNMS location mapping and location string parsing#302
Open
bonzo81 wants to merge 11 commits into
Open
feat: LibreNMS location mapping and location string parsing#302bonzo81 wants to merge 11 commits into
bonzo81 wants to merge 11 commits into
Conversation
Add a LocationMapping model that maps a LibreNMS location string to a NetBox Region/Site/Location/Rack/Tenant via a GenericForeignKey, plus two LibreNMSSettings fields (location_parse_pattern, location_parse_is_regex) for parsing the LibreNMS location string into tokens. Includes the model/form definitions, parser and mapping-resolver helpers in utils.py, and the supporting migrations.
Wire the LocationMapping model into the UI and API: list/detail/edit views, django-tables2 table, filterset, REST serializer/endpoints, URL routes, and a navigation menu item. Add a contrib seed YAML and document it in the contrib README.
Use the configured parse pattern (with non-destructive fallback to whole-string matching) and LocationMapping aliases when resolving the site and location during device import validation and creation. Add the Location Parsing settings card and link the matched site to its NetBox object in the import validation modal.
LibreNMS 26.5.0 returns each device's location as a relationship object (id, location, lat, lng, ...) instead of a flat name string. Flatten it to the location name in list_devices so the import flow matches sites correctly, mirroring the existing get_device_info handling.
Add test_location_mapping.py and test_location_parse.py covering the model, parser, settings-aware wrapper (including non-destructive fallback and dict location normalization), and form validation. Update test_utils.py for the find_matching_site LocationMapping fallback.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThis pull request adds LocationMapping support: a new LocationMapping model and migrations, parsing settings on LibreNMSSettings, parsing/resolver utilities, integration into device import (exact-name lookup with mapping fallback), full CRUD views/URLs/navigation, REST API serializer/viewset, forms with validation and import helpers, table/templates including bulk YAML export, and comprehensive tests. Example configuration template and README entry are included. Sequence Diagram(s)sequenceDiagram
participant DeviceImport
participant ParseService
participant Settings
participant LocationMapping
participant NetBoxDB
DeviceImport->>ParseService: parse_location_for_import(librenms_location)
ParseService->>Settings: get_location_parse_settings()
Settings-->>ParseService: pattern, is_regex
ParseService->>ParseService: parse_librenms_location(...)
alt parsed site found
ParseService-->>DeviceImport: site token
else no parsed site
DeviceImport->>NetBoxDB: exact Site.name lookup
alt exact match
NetBoxDB-->>DeviceImport: Site (match_type='exact', confidence=1.0)
else no exact match
DeviceImport->>LocationMapping: resolve_location_mapping('site', value)
LocationMapping->>NetBoxDB: return mapped Site or None
NetBoxDB-->>DeviceImport: Site (match_type='mapping') or None
end
end
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 38d57b1e-ce12-40a0-85d9-06335b565d9c
📒 Files selected for processing (25)
contrib/README.mdcontrib/location_mappings.yamlnetbox_librenms_plugin/api/serializers.pynetbox_librenms_plugin/api/urls.pynetbox_librenms_plugin/api/views.pynetbox_librenms_plugin/filters.pynetbox_librenms_plugin/forms.pynetbox_librenms_plugin/import_utils/device_operations.pynetbox_librenms_plugin/librenms_api.pynetbox_librenms_plugin/migrations/0011_locationmapping.pynetbox_librenms_plugin/migrations/0012_librenmssettings_location_parse_is_regex_and_more.pynetbox_librenms_plugin/models.pynetbox_librenms_plugin/navigation.pynetbox_librenms_plugin/tables/mappings.pynetbox_librenms_plugin/templates/netbox_librenms_plugin/htmx/device_validation_details.htmlnetbox_librenms_plugin/templates/netbox_librenms_plugin/locationmapping.htmlnetbox_librenms_plugin/templates/netbox_librenms_plugin/locationmapping_list.htmlnetbox_librenms_plugin/templates/netbox_librenms_plugin/settings.htmlnetbox_librenms_plugin/tests/test_location_mapping.pynetbox_librenms_plugin/tests/test_location_parse.pynetbox_librenms_plugin/tests/test_utils.pynetbox_librenms_plugin/urls.pynetbox_librenms_plugin/utils.pynetbox_librenms_plugin/views/__init__.pynetbox_librenms_plugin/views/mapping_views.py
- Cap location string length (512 chars) before regex matching to mitigate
polynomial-regex DoS on user-provided parse patterns (CodeQL).
- Read LibreNMSSettings via order_by('pk').first() instead of hard-coded pk=1
so a settings row with any PK is honoured.
- Strip the netbox_object value in LocationMappingImportForm.clean() before
lookup and persist the normalised value.
Add a case-insensitive partial UniqueConstraint on (field_type, Lower(librenms_value)) for region/site/tenant mappings so concurrent writes cannot create ambiguous duplicates that make resolution non-deterministic. location/rack remain excluded (scoped to a parent site). The constraint's backing partial index also serves the unscoped resolution lookups.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds LibreNMS “location” support to the device import workflow by (1) introducing a LocationMapping model (CRUD UI + API + import/export seed YAML) for explicit aliasing of LibreNMS location tokens to NetBox org objects, and (2) adding settings-driven parsing of the LibreNMS location string into structured tokens with best-effort fallback to preserve existing behavior. Also normalizes LibreNMS 26.5.0+ location relationship objects back to a string value for compatibility.
Changes:
- Add
LocationMapping(model, migrations, tables, filters, forms, views, navigation, API endpoints, templates, contrib seed YAML). - Add location parse settings (
location_parse_pattern,location_parse_is_regex) + parsing utilities and integrate into device import site/location resolution. - Normalize LibreNMS device
locationpayload shape changes (dict → name string) in the API client and downstream parsing.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| netbox_librenms_plugin/views/mapping_views.py | Adds CRUD + bulk import/export/changelog views for LocationMapping. |
| netbox_librenms_plugin/views/init.py | Re-exports newly added LocationMapping views. |
| netbox_librenms_plugin/utils.py | Adds location normalization, parsing, settings lookup, and mapping resolution utilities; updates site matching fallback. |
| netbox_librenms_plugin/urls.py | Adds URL routes for LocationMapping UI endpoints. |
| netbox_librenms_plugin/tests/test_utils.py | Updates site-matching tests to cover mapping fallback behavior. |
| netbox_librenms_plugin/tests/test_location_parse.py | Adds unit tests for parsing logic and ImportSettingsForm validation. |
| netbox_librenms_plugin/tests/test_location_mapping.py | Adds unit tests for LocationMapping model behavior and resolver logic. |
| netbox_librenms_plugin/templates/netbox_librenms_plugin/settings.html | Adds settings UI for location parsing + JS change-detection for save button enablement. |
| netbox_librenms_plugin/templates/netbox_librenms_plugin/locationmapping.html | Adds LocationMapping object detail template. |
| netbox_librenms_plugin/templates/netbox_librenms_plugin/locationmapping_list.html | Adds LocationMapping list template text + bulk YAML export button. |
| netbox_librenms_plugin/templates/netbox_librenms_plugin/htmx/device_validation_details.html | Makes matched site in validation details link to the NetBox site object. |
| netbox_librenms_plugin/tables/mappings.py | Adds LocationMappingTable for UI list/bulk operations. |
| netbox_librenms_plugin/navigation.py | Adds “Location Mappings” to the plugin navigation menu. |
| netbox_librenms_plugin/models.py | Adds LocationMapping model and location-parse settings fields on LibreNMSSettings. |
| netbox_librenms_plugin/migrations/0011_locationmapping.py | Creates initial LocationMapping database table. |
| netbox_librenms_plugin/migrations/0012_librenmssettings_location_parse_is_regex_and_more.py | Adds the two new location parse fields to LibreNMSSettings. |
| netbox_librenms_plugin/migrations/0013_locationmapping_uniq_locationmapping_unscoped_ci.py | Adds conditional case-insensitive uniqueness constraint for unscoped mapping types. |
| netbox_librenms_plugin/librenms_api.py | Normalizes LibreNMS 26.5.0+ device location relationship objects to a string. |
| netbox_librenms_plugin/import_utils/device_operations.py | Integrates parsing + mapping resolution into import validation and device creation (site/location). |
| netbox_librenms_plugin/forms.py | Adds settings form validation for parse pattern + LocationMapping create/import/filter forms. |
| netbox_librenms_plugin/filters.py | Adds LocationMappingFilterSet for UI/API filtering. |
| netbox_librenms_plugin/api/views.py | Adds REST API viewset for LocationMapping. |
| netbox_librenms_plugin/api/urls.py | Registers location-mappings API routes. |
| netbox_librenms_plugin/api/serializers.py | Adds LocationMappingSerializer. |
| contrib/README.md | Documents new location_mappings.yaml seed file. |
| contrib/location_mappings.yaml | Provides example seed mappings for region/site/location/rack/tenant. |
- Use models.F("field_type") in the UniqueConstraint to match the generated
migration and Django's expression-constraint API.
- Add select_related("content_type") to the LocationMapping list and YAML
export querysets to avoid per-row content type queries, matching the other
mapping list views.
select_related('content_type') only joins the ContentType row, not the GFK
target object. Add prefetch_related('netbox_object') on the list and YAML
export querysets so rendering netbox_object (table column + to_yaml) issues
one query per content type instead of one per row.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds LibreNMS location handling to the device import flow in two complementary parts:
LibreNMSSettingsfields (location_parse_pattern,location_parse_is_regex) that parse the LibreNMS location string into tokens (e.g. site/location/rack) for import. Parsing is best-effort: when no pattern is configured, or a pattern doesn't resolve a token, the import falls back to whole-string matching so baseline behaviour is preserved.It also fixes a LibreNMS 26.5.0 compatibility issue where the device
locationfield is now a relationship object rather than a flat string.Motivation / Problem
list_devicesreturnedlocationas a dict ({id, location, lat, lng, ...}), which broke exact site matching and surfaced as "No matching site" on the import validation modal.Scope of Change
How Was This Tested?
test_location_mapping.pyandtest_location_parse.pycover the model, parser, settings-aware wrapper (including non-destructive fallback and dict-location normalization), and form validation;test_utils.pyupdated for thefind_matching_siteLocationMapping fallback. Full plugin suite passes (3100 tests).Manual Test Steps
Risk Assessment
Backwards Compatibility
Other Notes
0011_locationmapping,0012_librenmssettings_location_parse_is_regex_and_more, and0013_locationmapping_uniq_locationmapping_unscoped_ci(case-insensitive partial unique constraint for unscoped region/site/tenant mappings, added in response to review).