Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,60 @@ versioning follows [SemVer](https://semver.org/).

## [Unreleased]

### Fixed

- **A comment may hold a quote, and a string a comment marker, in the same
grammar.** Rules ran one after another, and no order renders both: strings
first read `// don't stop, won't stop` as a comment holding the string
`'t stop, won'`; comments first read `"https://jsray.org"` as a string
holding a comment. Every grammar had picked one. Measured on beta.4, 27
grammars cut a line comment holding two quotes — JavaScript and TypeScript,
Python, PHP, shell, Ruby, SQL, YAML and the eleven C-family grammars among
them — and JavaScript, the C family and PHP read `"/* x */"` inside a string
as a comment. SQL fared worst: its strings may span lines, so `-- don't`
opened a literal that ran on to the next apostrophe anywhere below it.

A rule may now carry `group`, and adjacent rules sharing one compete by
position: whichever begins first owns the text to its own end, and listed
order only breaks a tie at the same index. Comments, strings, heredocs, regex
literals, JSON keys, C preprocessor lines and JavaScript parameter lists are
spans wherever they exist. The field is optional — `Grammar` is still
`GrammarRule[]`, and a rule without it behaves exactly as before.

Three forms added in beta.4 were part of the problem. Ruby's `%w[…]`, Perl's
`q{…}` and Elixir's `~r/…/` sat ahead of the comment rule, so
`# prefer %w[a b]` lost the rest of its comment; and a heredoc opening written
inside a comment, such as `# cat <<EOF`, turned the lines below it into a
string.

Found along the way, and fixed by the same change: a JavaScript regex holding
a quote — `s.split(/"/)` — opened a string that took the rest of the line; a
Java annotation named in a comment, `// see @Override`, cut the comment in
two; and JSONC read the `//` in `"https://…"`, which editor settings and
tsconfig files are full of, as the start of a comment.

Checked beyond the tests by rendering every JavaScript, PHP, shell, YAML and
CSS file in the four JSRay repositories, and each code block in their docs,
with beta.4 and with this build, and reading the differences by class. None
was a regression; beta.4 had three block comments swallowing code where this
build has none.

- **A template placeholder may hold a template of its own.**
`` `${ok ? `a ${b}` : 'c'}` `` and `` `${items.map((x) => `<li>${x}</li>`)}` ``
ended the outer template at the inner one's closing backtick. That was wrong
before and survivable; once spans compete, the outer template's real closing
backtick opened a new template running to the next backtick in the file, and
everything between rendered inverted. jsray-terminal's own tests, which carry
a Python script in a template, were where it showed. A placeholder now admits
one nested template and one level of braces, and each alternative inside it
still begins with its own character, so the pattern has one parse.

### Added

- **Private class members** — `#count`, `this.#count`, `#count in obj` — are
coloured as properties. They were plain text, and the type rule split `#Foo`
at the word boundary, colouring `Foo` and leaving the `#` bare.

## [0.0.2-beta.4] — 2026-09-09

Five languages gain the literals they never had, by way of the one thing the
Expand Down
7 changes: 6 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,12 @@ G.mylang = [
```

Key points:
- **Rule order determines priority.** Comments / strings always go first.
- **Rule order determines priority — except among spans.** Comments, strings,
and anything else that opens a span (a regex literal, a heredoc) carry
`group: 'span'` and compete by position: whichever begins first owns the text
up to its own end, and listed order only breaks a tie at the same index.
Ordering them against each other cannot work — strings first breaks
`// don't … won't`, comments first breaks `"https://…"`.
- **Declaration rules go before `keyword`** (otherwise `function`/`def`/`class` are consumed by the keyword rule first and the declaration name is never captured).
- Use `lookbehind: true` with patterns like `(\bfunction\s+)` to mark a prefix; the prefix is consumed but not colored.
- Use `inside: [...]` to re-apply a sub-grammar to captured text (parameter lists and template-string interpolations rely on this).
Expand Down
483 changes: 320 additions & 163 deletions dist/jsray.js

Large diffs are not rendered by default.

8 changes: 5 additions & 3 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,11 @@ code string ──tokenize(code, rules)──▶ token stream ──renderer─
`tokenize`, `render`, `applyTheme`, `detectLanguage`, `normalizeLanguage`,
`languages`. UMD-ish export: CommonJS `module.exports` + `global.JSRay`.

**Grammar rule ordering matters.** Strings must be matched before comments
(a `#` or `//` inside a string must not start a comment), declaration rules
before keyword rules (otherwise `function`/`def` consume the name).
**Grammar rule ordering matters — except among spans.** Comments, strings and
other span openers share `group: 'span'` and compete by position, so a `#` or
`//` inside a string stays text and a quote inside a comment stays comment; no
fixed order gets both right. Everywhere else order is priority: declaration
rules before keyword rules (otherwise `function`/`def` consume the name).
See CONTRIBUTING.md for the full checklist.

---
Expand Down
2 changes: 1 addition & 1 deletion docs/development.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ JSRay 生态的工程参考:全部官方仓库的架构、契约、约定与工
5. **主题运行时** —— `applyTheme(themeBlock, root)` 写入 `--jr-*` CSS 变量。默认目标是携带 `data-theme` 的元素(通常是 `<body>`):主题样式表通过 `[data-theme]` 选择器把同名变量定义在那里,写到祖先节点的内联变量会被遮蔽。
6. **公开 API** —— `highlight`、`highlightElement`、`highlightAll`、`tokenize`、`render`、`applyTheme`、`detectLanguage`、`normalizeLanguage`、`languages`。UMD 式导出:CommonJS `module.exports` + `global.JSRay`。

**语法规则顺序至关重要。** 字符串必须先于注释匹配(字符串里的 `#` 或 `//` 不能触发注释),声明规则先于关键字规则(否则 `function`/`def` 会吃掉声明名)。完整清单见 CONTRIBUTING.md。
**语法规则顺序至关重要 —— 跨度类规则除外。** 注释、字符串以及其它会开启一段跨度的规则共用 `group: 'span'`,按起始位置竞争:字符串里的 `#` 或 `//` 保持为文本,注释里的引号保持为注释 —— 任何固定顺序都无法两边都对。其余规则仍按顺序定优先级:声明规则先于关键字规则(否则 `function`/`def` 会吃掉声明名)。完整清单见 CONTRIBUTING.md。

---

Expand Down
4 changes: 2 additions & 2 deletions docs/languages.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ Recognizes:
- Parameter lists → `tk-var-param` (both `function name(a, b: T = 0)` and `(a, b) => ...`)
- Builtin variables: `console`, `window`, `document`, `globalThis`, `Math`, `JSON`, ...
- Builtin functions (as `.fn(` or `fn(`): `log`, `fetch`, `parseInt`, ...
- Template strings `` `...${id}...` `` with inline `${}` interpolation highlighted
- Template strings `` `...${id}...` `` with inline `${}` interpolation highlighted, including a template nested inside a placeholder (`` `${ok ? `a ${b}` : 'c'}` ``) and one level of braces (`${fn({ a })}`)
- Regex literals `/pattern/flags`, context-sensitive (only after `=` `(` `,` `return`, etc.)
- Numeric literals including separators (`1_000_000`), binary/octal/hex, and the BigInt suffix (`10n`)
- `ALL_CAPS` constants, `.property` access, `@decorator`
- `ALL_CAPS` constants, `.property` access and private members (`#count`), `@decorator`

```ts
async function fetchUser(id: number): Promise<User> {
Expand Down
4 changes: 2 additions & 2 deletions docs/languages.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@
- 形参列表 → `tk-var-param`(含 `function name(a, b: T = 0)` 和 `(a, b) => ...`)
- 内置变量:`console`, `window`, `document`, `globalThis`, `Math`, `JSON`, ...
- 内置函数(出现在 `.fn(` 或 `fn(` 位置):`log`, `fetch`, `parseInt`, ...
- 模板字符串 `` `...${id}...` ``,含 `${}` 内联高亮
- 模板字符串 `` `...${id}...` ``,含 `${}` 内联高亮,包括占位符里再套一层模板串(`` `${ok ? `a ${b}` : 'c'}` ``)和一层花括号(`${fn({ a })}`)
- 数字字面量含分隔符(`1_000_000`)、二/八/十六进制,以及 BigInt 后缀(`10n`)
- 正则字面量 `/pattern/flags`,上下文敏感(前面是 `=` `(` `,` `return` 等才识别)
- `ALL_CAPS` 常量、`.property` 访问、`@decorator`
- `ALL_CAPS` 常量、`.property` 访问与私有成员(`#count`)、`@decorator`

```ts
async function fetchUser(id: number): Promise<User> {
Expand Down
2 changes: 1 addition & 1 deletion integrity.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"algorithm": "sha256",
"note": "Base64 SHA-256 digests of the released Core assets. Integrations copy the relevant digest at sync time and verify their bundled snapshot against it.",
"files": {
"dist/jsray.js": "sha256-ZU8tfz1KrOvaewDqhJC9D5rzbItkXfreHJF+zkQ7EUU=",
"dist/jsray.js": "sha256-KOSefuYNXQrhcQMaV4c25TpTRhgUMtNgX5WvtT3NGGc=",
"dist/jsray.css": "sha256-6Fuva+aZwCcmltNi2oN+1yWIJoKE+1rLpUxZvD9iutc=",
"dist/themes/aurora.css": "sha256-S8X+R8XZNC8WRdHOen839zMkPxVFol6PPTrpZibbeJA=",
"dist/themes/default.css": "sha256-vENOigRjwn1hW6wPjdM4tbZ58Ja44FMsa+akavPiMSI=",
Expand Down
Loading
Loading