Repository navigation
ci(bench): ERD, charts/groupings and multi-tab SQL scenarios in the app benchmark (#1173) - #1255
Merged
Merged
Conversation
…pp benchmark (#1173) Scenarios in benchmark/app_perf_bench.dart, on the Demo Playground: - charts_5000, groupings_5000: Charts (line, pie, bar) and Groupings over the 5 000 rows of the heavy query. - multi_tab_run: a run in one of five tabs holding results, with the number of query panes it rebuilt. - erd_open_200, erd_hover, erd_drag, erd_auto_layout: a 200-table schema joined by foreign keys, created in the playground through the editor. A Benchmark job runs them under xvfb, and scripts/ci/app_bench_report.py reports build p50/p90/max, raster and stutters per scenario against its limit, plus the pane count. Unit tests for the report run in CI.
github-actions
Bot
added
performance
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1173
Stacked on #1254 (#1132): both change the benchmark workflow and the CI step that tests its scripts. Merge #1254 first; this diff then shrinks to its own commit.
Scenarios (
benchmark/app_perf_bench.dart, Demo Playground, no database server)charts_5000,groupings_5000: Charts (Line, Pie, Bar) and Groupings over the 5 000 rows of the heavy query.multi_tab_run: five tabs hold results, and a query runs in one of them. The bench prints how many query panes the run rebuilt (paneBuildCount).erd_open_200,erd_hover,erd_drag,erd_auto_layout: a 200-table schema joined by foreign keys, created in the playground through the SQL editor. The scenarios open the Diagram, hover across it for 3 s, drag a card for 2 s, and run Auto layout.CI
benchmark.ymlgets an App scenarios performance job. It runs only these scenarios (ONLY=...) under xvfb and writes the report to the job summary. Informational andcontinue-on-error, like the grid bench. It also runs on changes tolib/features/erd/**andlib/features/results/**.scripts/ci/app_bench_report.pyreports build p50/p90/max, raster p50 and stutters per scenario. Build p90 is checked against a per-scenario limit: 16.7 ms for charts, groupings, the tab run and the drag; 8.3 ms for hover; 50 ms for opening and laying out the diagram. The report also checks that the tab run rebuilds at most one pane. A missing scenario counts as over.scripts/ci/test_app_bench_report.py(8 cases) runs in the CI analyze job. It passes locally.The issue mentions 50 and 200 tables; this measures 200, the harder case.