Skip to content

Conversation

@ericsciple
Copy link
Collaborator

@ericsciple ericsciple commented Dec 5, 2025

Overview

This PR optimizes the webhook payload JSON files used for GitHub Actions autocompletion and hover documentation.

Total JSON payload (gzipped): 42 KB → 36 KB (15% reduction)

File Before After
webhooks.json 12 KB 8 KB
webhooks.objects.json 11 KB 7 KB
webhooks.strings.json 2 KB
Other files (unchanged) 19 KB 19 KB
Total 42 KB 36 KB

Changes

New Optimizations

  1. Compact array format — Params converted from verbose objects to type-dispatched arrays:

    // Before
    {"name": "issue", "description": "The issue itself.", "childParamsGroups": [...]}
    
    // After  
    ["issue", "The issue itself.", [...]]
  2. String interning — Duplicate property names stored in webhooks.strings.json and referenced by non-negative index

  3. Negative indices for objects — Object references now use negative numbers (-(index + 1)) to distinguish from string indices, allowing all duplicate names to be interned

File Renames

  • objects.jsonwebhooks.objects.json
  • Added webhooks.strings.json for interned property names

Other Changes

  • Moved event filters from TypeScript to event-filters.json for maintainability
  • Added "//" comment key to webhooks.json pointing to documentation
  • Added CI validation job (validate-webhooks) to verify optimization correctness

Size Reduction

File Before After
webhooks.min.json 265 KB 82 KB
webhooks.objects.min.json 199 KB 64 KB
webhooks.strings.min.json 6 KB
Total 464 KB 152 KB

Testing

  • All existing tests pass
  • Added 3 validation tests that compare optimized output against full source (webhooks.full.json) to ensure no data loss
  • Lint and format checks pass

Documentation

Updated docs/json-data-files.md with:

  • Compact format specification
  • String interning explanation
  • Negative index scheme for object references
  • Size reduction breakdown

@ericsciple ericsciple force-pushed the users/ericsciple/25-12-webhooks branch from bfb514f to 4dacd2c Compare December 6, 2025 05:53
@ericsciple ericsciple changed the title Add JSON optimization plan for webhooks/objects data files Optimize webhooks/objects JSON with string interning Dec 6, 2025
@ericsciple ericsciple changed the title Optimize webhooks/objects JSON with string interning Optimize webhooks JSON with compact structure and string interning Dec 6, 2025
@ericsciple ericsciple force-pushed the users/ericsciple/25-12-webhooks branch from 4dacd2c to 27aec5d Compare December 6, 2025 06:23
run: cd languageservice && npm test -- --testPathPattern=eventPayloads
- name: Verify validation tests ran
run: |
if [ ! -f languageservice/src/context-providers/events/webhooks.full.validation-complete ]; then
Copy link
Collaborator Author

@ericsciple ericsciple Dec 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the test is normally skipped when the full webhooks file doesn't exist, I made the test write a marker file. With the marker file, we can be sure the test actually ran.

The full webhooks file (unoptimized) is written above by the script npm run update-webhooks

exit 1
fi
validate-webhooks:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this job to validate the optimized webhooks JSON is equivalent to the full webhooks JSON file.

Refer changes to the file eventPayloads.test.ts in this PR

@ericsciple ericsciple force-pushed the users/ericsciple/25-12-webhooks branch from 27aec5d to 8fbd093 Compare December 6, 2025 06:32
@@ -1,310 +0,0 @@
import {promises as fs} from "fs";
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I renamed this file to update-webhooks.ts but there were also significant updates anyway

@@ -1,7 +1,8 @@
import {data, DescriptionDictionary} from "@actions/expressions";

Copy link
Collaborator Author

@ericsciple ericsciple Dec 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eventsPayloads.ts reads the compact webhook JSON files which contains the keys and descriptions for each GitHub event payload. This information is used for github.event.* syntax validation, autocompletion, and hover descriptions.

A unit test that runs during CI validates the compact webhook JSON files are parsed into a form that exactly matches the full unoptimized webhooks JSON.

The script update-webhooks.ts downloads the full webhook JSON from the GitHub REST API description repository:

import schemaImport from "rest-api-description/descriptions/api.github.com/dereferenced/api.github.com.deref.json";

@ericsciple ericsciple force-pushed the users/ericsciple/25-12-webhooks branch 3 times, most recently from 10dd69f to 2b6d46c Compare December 6, 2025 06:59
@ericsciple ericsciple marked this pull request as ready for review December 6, 2025 07:04
@ericsciple ericsciple requested a review from a team as a code owner December 6, 2025 07:04
Copilot AI review requested due to automatic review settings December 6, 2025 07:04
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes webhook payload JSON files used for GitHub Actions autocompletion and hover documentation by introducing a compact array format, object deduplication, and string interning. The optimizations achieve a 55% reduction in minified file size (464 KB → 209 KB) and 65% reduction when gzipped (63 KB → 22 KB).

Key Changes:

  • Introduced compact array format for params using type-based dispatch instead of verbose objects
  • Implemented string interning to deduplicate 414 property names into a shared strings.json file
  • Extracted event filtering configuration from TypeScript to event-filters.json for use in both generation and test code

Reviewed changes

Copilot reviewed 11 out of 14 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
languageservice/src/context-providers/events/strings.json New string table containing 414 interned property names referenced by index
languageservice/src/context-providers/events/eventPayloads.ts Updated to load separate string table and decode compact format with string interning support
languageservice/src/context-providers/events/eventPayloads.test.ts Added comprehensive validation tests comparing optimized output against full source
languageservice/src/context-providers/events/event-filters.json New JSON file defining dropped/kept events, replacing hardcoded TypeScript constants
languageservice/script/webhooks/update-webhooks.ts New generation script implementing compact format conversion, deduplication, and string interning
languageservice/script/webhooks/event-filters.ts TypeScript constants for event filtering (note: appears unused, actual imports use JSON file)
languageservice/script/webhooks/deduplicate.ts Updated to handle both compact and legacy param formats with helper functions
languageservice/package.json Updated scripts to include new strings.json in minification and point to new update-webhooks script
docs/json-data-files.md Comprehensive documentation of compact format, string interning, and optimization details
.gitignore Updated patterns for new full.json and validation-complete marker files
.github/workflows/buildtest.yml Added separate validate-webhooks job to verify optimization correctness
languageservice/script/webhooks/index.ts Removed (replaced by update-webhooks.ts)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ericsciple ericsciple force-pushed the users/ericsciple/25-12-webhooks branch 4 times, most recently from d953314 to c034e38 Compare December 6, 2025 19:41
- Convert params to compact array format (type-based dispatch)
- Intern duplicate property names into string table
- Use negative indices for object references to distinguish from string indices
- Rename objects.json → webhooks.objects.json, add webhooks.strings.json
- Move event filters to JSON for maintainability
- Add CI validation job to verify optimization correctness

Reduces combined minified size by ~67% (453 KB → 148 KB), ~27% gzipped (23 KB → 17 KB).
@ericsciple ericsciple force-pushed the users/ericsciple/25-12-webhooks branch from c034e38 to c04c1b2 Compare December 6, 2025 20:05
@ericsciple ericsciple marked this pull request as draft December 8, 2025 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants