feat: time-series download analytics backed by TimescaleDB - #2027
feat: time-series download analytics backed by TimescaleDB#2027gnugomez wants to merge 21 commits into
Conversation
fcae59e to
d66b01f
Compare
d66b01f to
35e5126
Compare
There was a problem hiding this comment.
Pull request overview
Adds opt-in TimescaleDB-backed download analytics, CDN log ingestion, a public time-series API, and weekly download statistics in the web UI.
Changes:
- Introduces analytics storage, ingestion, API, metrics, and tests.
- Adds the weekly-downloads extension card and frontend service types.
- Expands frontend extension points and shared UI infrastructure.
Reviewed changes
Copilot reviewed 111 out of 114 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
webui/vite.config.mts |
Inlines MUI test dependencies |
webui/test/unit/support/test-providers.tsx |
Extends provider test helpers |
webui/test/unit/support/menu-queries.ts |
Adds menu query helper |
webui/test/unit/pages/user/avatar.spec.tsx |
Tests desktop menu contributions |
webui/test/unit/pages/admin-dashboard/admin-dashboard.spec.tsx |
Tests contributed admin pages |
webui/test/unit/main.spec.tsx |
Tests user-loading context |
webui/test/unit/default/menu-content.spec.tsx |
Tests mobile menu contributions |
webui/test/unit/components/weekly-downloads.spec.tsx |
Tests weekly downloads card |
webui/src/pages/user/avatar.tsx |
Supports contributed menu entries |
webui/src/pages/search/search-header.tsx |
Reuses compact control styling |
webui/src/pages/extension-detail/weekly-downloads.tsx |
Implements analytics card |
webui/src/pages/extension-detail/use-extension-download-series.ts |
Fetches annual download series |
webui/src/pages/extension-detail/extension-detail.tsx |
Updates chrome context reference |
webui/src/pages/extension-detail/extension-detail-overview.tsx |
Mounts weekly downloads |
webui/src/pages/admin-dashboard/nav-types.ts |
Defines contributed admin pages |
webui/src/pages/admin-dashboard/admin-dashboard.tsx |
Integrates contributed pages |
webui/src/page-settings.ts |
Adds consumer extension points |
webui/src/main.tsx |
Exposes user-loading state |
webui/src/layout/scroll-restoration.tsx |
Renames scroll helper |
webui/src/layout/app-navbar.tsx |
Supports extended navbar blur |
webui/src/layout/app-layout.tsx |
Uses navbar chrome provider |
webui/src/index.ts |
Expands public frontend exports |
webui/src/extension-registry-types.ts |
Adds analytics response types |
webui/src/extension-registry-service.ts |
Adds analytics API client |
webui/src/default/theme.tsx |
Updates popup styling |
webui/src/default/menu-content.tsx |
Supports mobile menu contributions |
webui/src/context/navbar-chrome-context.tsx |
Consolidates navbar chrome state |
webui/src/context/extension-tint-context.tsx |
Removes superseded context |
webui/src/context.ts |
Adds userLoading |
webui/src/components/pill.tsx |
Adds reusable pill primitive |
webui/src/components/page-primitives.tsx |
Adds shared style primitives |
webui/src/components/extension-searchfield.tsx |
Reuses slash primitive |
webui/src/components/category-pill.tsx |
Reuses generic pill |
webui/src/app-providers.tsx |
Updates provider documentation |
webui/CHANGELOG.md |
Documents frontend changes |
server/src/test/resources/org/eclipse/openvsx/storage/log/fastly.log |
Removes obsolete fixture |
server/src/test/resources/org/eclipse/openvsx/analytics/ingestion/aws/fastly.log |
Adds Fastly ingestion fixture |
server/src/test/resources/org/eclipse/openvsx/analytics/ingestion/aws/cloudfront.log |
Extends CloudFront fixture |
server/src/test/resources/application.yml |
Limits test connection pools |
server/src/test/java/org/eclipse/openvsx/storage/StorageUtilServiceUploadFileTest.java |
Updates ingestion dependencies |
server/src/test/java/org/eclipse/openvsx/storage/StorageUtilServiceTest.java |
Tests ingestion-aware counting |
server/src/test/java/org/eclipse/openvsx/storage/log/FastlyLogFileParserTest.java |
Removes obsolete parser tests |
server/src/test/java/org/eclipse/openvsx/storage/log/CloudFrontLogFileParserTest.java |
Removes obsolete parser tests |
server/src/test/java/org/eclipse/openvsx/repositories/RepositoryServiceSmokeTest.java |
Covers renamed queries |
server/src/test/java/org/eclipse/openvsx/RegistryAPITest.java |
Updates test wiring |
server/src/test/java/org/eclipse/openvsx/eclipse/EclipseServiceTest.java |
Updates test wiring |
server/src/test/java/org/eclipse/openvsx/analytics/ingestion/RawDownloadRecordTest.java |
Tests raw records |
server/src/test/java/org/eclipse/openvsx/analytics/ingestion/jobs/IngestionJobsTest.java |
Tests job scheduling |
server/src/test/java/org/eclipse/openvsx/analytics/ingestion/DownloadIngestionMetricsTest.java |
Tests ingestion metrics |
server/src/test/java/org/eclipse/openvsx/analytics/ingestion/CountryCodesTest.java |
Tests country normalization |
server/src/test/java/org/eclipse/openvsx/analytics/ingestion/aws/FastlyLogFileParserTest.java |
Tests Fastly parsing |
server/src/test/java/org/eclipse/openvsx/analytics/ingestion/aws/CloudFrontLogFileParserTest.java |
Tests CloudFront parsing |
server/src/test/java/org/eclipse/openvsx/analytics/ingestion/aws/AccessLogRecordTest.java |
Tests download extraction |
server/src/test/java/org/eclipse/openvsx/analytics/DownloadEventTest.java |
Tests event validation |
server/src/test/java/org/eclipse/openvsx/analytics/DownloadAnalyticsServiceTest.java |
Tests series aggregation |
server/src/test/java/org/eclipse/openvsx/analytics/DownloadAnalyticsEndpointTest.java |
Tests end-to-end analytics |
server/src/test/java/org/eclipse/openvsx/analytics/DownloadAnalyticsDisabledTest.java |
Tests disabled configuration |
server/src/test/java/org/eclipse/openvsx/analytics/DownloadAnalyticsAPITest.java |
Tests public endpoint |
server/src/test/java/org/eclipse/openvsx/admin/AdminAPITest.java |
Updates test wiring |
server/src/test/java/org/eclipse/openvsx/adapter/VSCodeAPITest.java |
Updates test wiring |
server/src/test/java/org/eclipse/openvsx/AbstractTimeseriesContainerTest.java |
Adds Timescale test base |
server/src/test/java/org/eclipse/openvsx/AbstractPostgresContainerTest.java |
Makes image configurable |
server/src/main/resources/db/migration-timeseries/V1__Download_Analytics.sql.conf |
Configures Flyway transaction mode |
server/src/main/resources/db/migration-timeseries/V1__Download_Analytics.sql |
Creates analytics schema |
server/src/main/java/org/eclipse/openvsx/storage/StorageUtilService.java |
Captures request-path events |
server/src/main/java/org/eclipse/openvsx/storage/log/FastlyLogFileParser.java |
Removes legacy parser |
server/src/main/java/org/eclipse/openvsx/storage/log/DownloadCountService.java |
Removes legacy scheduler |
server/src/main/java/org/eclipse/openvsx/storage/log/DownloadCountProcessor.java |
Removes legacy processor |
server/src/main/java/org/eclipse/openvsx/storage/log/CloudFrontLogFileParser.java |
Removes legacy parser |
server/src/main/java/org/eclipse/openvsx/repositories/RepositoryService.java |
Renames ingestion queries |
server/src/main/java/org/eclipse/openvsx/repositories/DownloadIngestionRepository.java |
Provides ingestion-state queries |
server/src/main/java/org/eclipse/openvsx/mirror/aop/DownloadCountServiceAspect.java |
Removes obsolete mirror aspect |
server/src/main/java/org/eclipse/openvsx/LocalRegistryService.java |
Reports analytics availability |
server/src/main/java/org/eclipse/openvsx/json/RegistryVersionJson.java |
Adds analytics feature flag |
server/src/main/java/org/eclipse/openvsx/entities/DownloadIngestion.java |
Renames ingestion entity |
server/src/main/java/org/eclipse/openvsx/analytics/timescale/TimeseriesDatabaseConfiguration.java |
Configures TimescaleDB |
server/src/main/java/org/eclipse/openvsx/analytics/timescale/TimescaleDownloadAnalyticsRepository.java |
Implements Timescale repository |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/RawDownloadRecord.java |
Defines raw record |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/jobs/LogIngestionJob.java |
Schedules ingestion jobs |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/jobs/IngestionJobRequest.java |
Defines job request |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/DownloadRecordSource.java |
Defines source contract |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/DownloadIngestionRunner.java |
Orchestrates ingestion |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/DownloadIngestionMetrics.java |
Adds operational metrics |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/CountryCodes.java |
Normalizes countries |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/aws/LogFileParser.java |
Defines parser contract |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/aws/FastlyLogFileParser.java |
Parses Fastly logs |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/aws/CloudFrontLogFileParser.java |
Parses CloudFront logs |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/aws/AwsDownloadRecordSource.java |
Reads S3 access logs |
server/src/main/java/org/eclipse/openvsx/analytics/ingestion/aws/AccessLogRecord.java |
Converts access records |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadSeriesRow.java |
Defines repository row |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadSeriesRequest.java |
Defines series query |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadSeriesPoint.java |
Defines series point |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadSeriesJson.java |
Defines API payload |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadSeriesInterval.java |
Defines bucket intervals |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadSeriesGroupBy.java |
Defines grouping dimensions |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadEvent.java |
Defines analytics event |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadAnalyticsService.java |
Builds and caches series |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadAnalyticsRepository.java |
Defines storage interface |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadAnalyticsConfiguration.java |
Wires analytics services |
server/src/main/java/org/eclipse/openvsx/analytics/DownloadAnalyticsAPI.java |
Exposes public endpoint |
server/src/dev/resources/application.yml |
Enables development analytics |
server/scripts/generate-properties.sh |
Generates Timescale properties |
server/build.gradle |
Supports test image overrides |
docker-compose.yml |
Adds TimescaleDB service |
doc/development.md |
Documents development databases |
deploy/openshift/application.yml |
Documents OpenShift settings |
deploy/kubernetes/configmap.yaml |
Documents Kubernetes settings |
deploy/docker/configuration/application.yml |
Documents Docker settings |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Jordi Gómez Hidalgo <31970428+gnugomez@users.noreply.github.com>
Signed-off-by: Thomas Neidhart <thomas.neidhart@eclipse-foundation.org>
|
This PR changes too many things at the same time. It is not clear to me what happens when Need to split this PR into smaller pieces to make the change manageable without introducing a regression for the existing functionality or when |
Related to #2025
Adds opt-in download analytics to the registry: per-extension download counts over time, ingested from CDN access logs into a TimescaleDB hypertable, exposed through a public API and surfaced as a weekly-downloads card on the extension detail page.
Disabled by default (ovsx.analytics.enabled, default false).
Screen.Recording.2026-08-24.at.11.08.49.mov