SOLR-8474: Add Selenium-based JUnit tests for the Admin UI #4738
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| title: > | ||
| Added Selenium-based JUnit test coverage for the Admin UI, driving a headless Chrome against a test cluster. | ||
| type: other | ||
| authors: | ||
| - name: Jan Høydahl | ||
| links: | ||
| - name: SOLR-8474 | ||
| url: https://issues.apache.org/jira/browse/SOLR-8474 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,177 @@ | ||
| <!-- | ||
| Licensed to the Apache Software Foundation (ASF) under one or more | ||
| contributor license agreements. See the NOTICE file distributed with | ||
| this work for additional information regarding copyright ownership. | ||
| The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| (the "License"); you may not use this file except in compliance with | ||
| the License. You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| --> | ||
|
|
||
| # Admin UI (AngularJS) Browser Test Plan | ||
|
|
||
| This document tracks browser-based test coverage of the old AngularJS Admin UI | ||
| (`solr/webapp/web/`), driven by Selenium WebDriver with headless Chrome. | ||
|
|
||
| ## How the tests work | ||
|
|
||
| - Tests live in `solr/webapp/src/test/org/apache/solr/webapp/` and extend | ||
| `AdminUiTestBase`, which starts a 2-node `MiniSolrCloudCluster` whose Jetty | ||
| nodes also serve the Admin UI (opt-in `JettyConfig.Builder#enableAdminUi`), | ||
| then starts a headless Chrome via Selenium WebDriver. | ||
| - A locally installed Chrome/Chromium is required; without one, tests skip via | ||
| `Assume`. Override discovery with `-Dtests.ui.chrome.binary=/path/to/chrome`. | ||
| The matching chromedriver is provisioned (and cached) by Selenium Manager. | ||
| - Display assertions compare UI text against live JSON from the same node's | ||
| admin APIs — never hardcoded values. | ||
| - Tests are grouped per screen/feature, so each screen's display and write | ||
| tests live in the same class. | ||
| - Most tests run against a 2-node cloud cluster; `AdminUiStandaloneTestBase` | ||
| additionally supports standalone (user-managed, no ZooKeeper) nodes, whose | ||
| UI differs (no Cloud/Collections/Schema Designer menus; the per-core menu | ||
| offers query/replication etc. directly). | ||
| - On failure, a screenshot, the page source and the browser console log are | ||
| saved into the test temp dir. | ||
| - Run with: `./gradlew :solr:webapp:test` | ||
|
|
||
| ## Coverage by screen | ||
|
Contributor
There was a problem hiding this comment. I don't. thinks this is useful... this just looks like a lot of content that isn't helpful to me, and will be out of date. Now, if we think AI needs this to understand the tests, but I don't think that either... Also, i wondered if this was a checked in "lets work our way through all the screens", but since everythign has a checked checkbox, i dont' think it's that either. |
||
|
|
||
| ### Smoke navigation — `AdminUiSmokeTest` | ||
| - [x] Every node-level route (`/`, `~logging`, `~logging/level`, | ||
| `~cloud?view=nodes|tree|zkstatus|graph`, `~cores`, `~collections`, | ||
| `~schema-designer`, `~security`, `~java-properties`, `~threads`, `login`), | ||
| every per-collection route and the per-core routes render their main | ||
| content element without severe browser console errors. | ||
|
|
||
| ### Dashboard — `AdminUiDashboardTest` | ||
| - [x] Versions, JVM info and memory bars vs `/admin/info/system`; security | ||
| warning when security is disabled. | ||
|
|
||
| ### Node-level screens — `AdminUiNodeScreensTest` | ||
| - [x] Java Properties: `java.version` value matches `/admin/info/properties` | ||
| - [x] Thread Dump: thread list non-empty, Jetty worker thread shown | ||
| - [x] Cloud > Nodes: one row per live node, ports match the cluster | ||
| - [x] Cloud > Tree: `live_nodes` and `collections` znodes shown | ||
| - [x] Cloud > Graph: d3 SVG renders circles for collection/shard/replicas | ||
| - [x] Cloud > ZK Status: status green, ensemble size shown | ||
| - [x] Security screen: "not enabled" warning without auth | ||
| - [x] Login screen: authentication info page without auth | ||
|
|
||
| ### Collections screen — `AdminUiCollectionsScreenTest` | ||
| - [x] Collection detail display (shards) | ||
| - [x] Create + delete collection via the dialogs (verified via API) | ||
| - [x] Create + delete alias via the dialogs (verified via LISTALIASES) | ||
| - [x] Add + delete replica via the shard detail (verified via cluster state) | ||
| - [x] Reload collection (verified via core start time reset) | ||
|
|
||
| ### Query screen — `AdminUiQueryScreenTest` | ||
| - [x] `*:*` and `id:` queries via the form, `numFound` in the response | ||
| - [x] `rows` and `fl` parameters affect the returned documents | ||
| - [x] Paramsets dropdown applies a paramset created via the API | ||
| - [x] defType dismax/edismax toggles reveal their parameter fields; edismax | ||
| query with `qf` returns the expected results | ||
| - [x] Raw query parameters (e.g. an extra `fq`) are applied | ||
|
|
||
| ### Documents screen — `AdminUiDocumentsScreenTest` | ||
| - [x] Form renders with JSON/XML/CSV document types | ||
| - [x] Index a JSON document via the form; becomes searchable | ||
|
|
||
| ### Schema screen — `AdminUiSchemaScreenTest` | ||
| - [x] Field list browsable, editable-schema action buttons shown | ||
| - [x] Field detail shows flags matching `/schema/fields`; term info loads | ||
| - [x] Add + delete a field via the dialogs (verified via `/schema/fields`) | ||
|
|
||
| ### Paramsets screen — `AdminUiParamsetsScreenTest` | ||
| - [x] Form renders | ||
| - [x] Create + delete a paramset via the form (verified via `/config/params`) | ||
|
|
||
| ### Logging screens — `AdminUiLoggingScreenTest` | ||
| - [x] Logger level tree renders | ||
| - [x] Set + unset a logger level via the tree (verified via API) | ||
| - [x] Events viewer shows a WARN event logged in the server JVM (skips itself | ||
| when the node's log watcher is blind due to shared-JVM log4j state, see | ||
| Known limitations) | ||
|
|
||
| ### Core Admin screen — `AdminUiCoreAdminScreenTest` (cloud) and | ||
| ### `AdminUiCoreAdminStandaloneTest` (standalone) | ||
| - [x] Hosted core listed, matching `/admin/cores` | ||
| - [x] Reload core via the button (success indicator) | ||
| - [x] Standalone-mode menu differences (no cloud menus; core menu offers | ||
| query/replication) | ||
| - [x] Add core via the dialog (pre-created instance dir) | ||
| - [x] Rename core via the dialog | ||
| - [x] Swap cores via the dialog (verified by the indexes exchanging) | ||
| - [x] Unload core, accepting the native confirm dialog | ||
|
|
||
| ### Per-collection display screens — `AdminUiCollectionScreensTest` | ||
| - [x] Analysis: `text_general` tokenizes and lowercases entered text | ||
| - [x] Files: tree lists `solrconfig.xml`, file content renders | ||
| - [x] Segments: at least one segment rendered after commit | ||
| - [x] Plugins/Stats: searcher stats present (needs `metricsEnabled=true`) | ||
| - [x] Collection overview: shard info; Core overview: numDocs | ||
|
|
||
| ### Stream screen — `AdminUiStreamScreenTest` | ||
| - [x] A `search(...)` streaming expression executes and renders all docs | ||
|
|
||
| ### SQL screen — `AdminUiSqlScreenTest` | ||
| - [x] A SQL query executes through the form and the result grid lists the | ||
| documents (the sql module is a test-only dependency of `solr:webapp`) | ||
|
|
||
| ### Replication screen — `AdminUiReplicationScreenTest` (cloud) and | ||
| ### `AdminUiReplicationStandaloneTest` (standalone leader/follower) | ||
| - [x] Renders index version info in cloud mode | ||
| - [x] Follower screen shows the leader's info | ||
| - [x] Disable polling, index on the leader, replicate-now transfers the | ||
| index, re-enable polling — verified via the replication API | ||
|
|
||
| ### Security with BasicAuth — `AdminUiSecurityAuthTest` | ||
| - [x] Unauthenticated visit redirects to login; login form authenticates | ||
| - [x] Security screen shows authn/authz plugins, users, roles, permissions | ||
| - [x] Add a user through the dialog (verified via `/admin/authentication`) | ||
| - [x] Add a role for the user through the dialog (verified via API) | ||
| - [x] Grant a predefined permission to the role (verified via API) | ||
|
|
||
| ### Schema Designer — `AdminUiSchemaDesignerTest` (`@AwaitsFix`) | ||
| - [x] Create a new schema, paste a sample doc, analyze; derived field shown — | ||
| but the designer backend is too flaky under automation | ||
| (see [SOLR-18347](https://issues.apache.org/jira/browse/SOLR-18347)), | ||
| so the test awaits a fix before running by default | ||
|
|
||
| ## Deliberately skipped (effort vs value) | ||
|
|
||
| - **JWT/OAuth login flows**: require an external identity provider or heavy | ||
| mocking; BasicAuth covers the UI's login/session mechanics. | ||
| - **Keystroke-level entry in the security dialogs**: native clicks/keystrokes | ||
| into the absolutely-positioned dialogs proved unreliable in headless Chrome; | ||
| the dialogs are driven via the Angular controller scope instead. Keyboard | ||
| entry is covered by the login form and the other screens' forms. | ||
|
|
||
| ## Known limitations | ||
|
Contributor
There was a problem hiding this comment. some o fthese items might be resolved in the "migrate to v2" work, jthere were some weaknesses in events identified... |
||
|
|
||
| - The generated js-client bundle (`libs/solr/index.js`) only exists inside the | ||
| built WAR, not in the source tree tests serve from. `AdminUiTestBase` serves | ||
| a minimal stub defining the `solrApi` global (only `reloadCollection` is used | ||
| by the AngularJS UI); a future improvement could serve the real bundle when | ||
| it has been built. | ||
| - Every test cluster in the JVM registers a log-watcher appender under the same | ||
| name in the shared log4j config, so a later cluster's watcher can be blind; | ||
| the events-viewer test detects this via the API and skips itself. | ||
| - The shared menu code logs a benign | ||
| `TypeError: Cannot read properties of null (reading 'name')` from | ||
| `$scope.showCore` while the per-collection menu resolves (filtered in the | ||
| console-error assertion; candidate for a JIRA). | ||
| - The core overview ping widget answers 503 when the configset has no | ||
|
Contributor
There was a problem hiding this comment. Should this item have a seperate JIRA to get fixed?
Contributor
There was a problem hiding this comment. (thanks for logging the other JS weaknesses as a JIRA) |
||
| healthcheck file (allowed in the affected tests). | ||
| - The Schema Designer's backend transiently fails its own prep/analyze calls | ||
| with "version mismatch, retry" and recovers via its retry dialog; its API | ||
| errors are excluded from the console-error assertion. | ||
| - ASF Jenkins has no Chrome, so these tests skip there; they run on developer | ||
| machines and could run in a GitHub Actions workflow (Chrome preinstalled on | ||
| `ubuntu-latest`) as a follow-up. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| 8fcc3779ff85fae5164cd1b977798ca1af388e06 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| 82212e5b3633e7a65fca8bec525762cfb6c9bae4 |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this docs, while nice to see in hte PR< I don't think need to stay long term?
I wish you coul dlabel content "Here for hte life of the PR, but once merged to main it goes away"!