diff --git a/changelog.md b/changelog.md index 0126cd0..61d5ae5 100644 --- a/changelog.md +++ b/changelog.md @@ -1,106 +1,102 @@ -Changelog -==== +# Changelog -2.0.0 ----- +## 2.0.0 -* Full ESM support -* Strong typescript support -* moveUntil don't join values by default -* parser.join(sep) is supported, only for TupleParser -* thenLeft and thenRight are not supported in typescript, too many combinations +- Full ESM support +- Strong typescript support +- moveUntil don't join values by default +- parser.join(sep) is supported, only for TupleParser +- thenLeft and thenRight are not supported in typescript, too many combinations -0.8.1: better ts support ---- +Plus plenty of other things I dont have time to write now :/ + +## 0.8.1: better ts support Minor change on ts and bug fix -0.7 ->0.8: Using Tuples with then ---- +## 0.7 ->0.8: Using Tuples with then -* Any then() and thenXYZ() function call will return a `Tuple` . This tuple has `single()` and `array()` methods to get value. -`Parser.drop()`, `F.nop()` methods will produce a *Neutral element* for that Tuple : it will be ignored. -* Therefore `thenReturns()` is renamed `returns()`. -* `rep()` and `optrep()` will also produce a Tuple +- Any then() and thenXYZ() function call will return a `Tuple` . This tuple has + `single()` and `array()` methods to get value. `Parser.drop()`, `F.nop()` + methods will produce a _Neutral element_ for that Tuple : it will be ignored. +- Therefore `thenReturns()` is renamed `returns()`. +- `rep()` and `optrep()` will also produce a Tuple - combination of `rep()` and `then()` will produce one unique Tuple - - you can *break apart* this tuple using `Parser.array()` at any point if needed -* `GenLex` has been totally refactored and simplified for high level parsers on tokens. -* Typescript interface is operational and `masala-parser.d.ts` documentation will generate Masala Parser reference. -* `N.digit()` and `N.digits()` return a number, not a string -* Markdown bundled is removed from the lib, and put as example in typescript integration + - you can _break apart_ this tuple using `Parser.array()` at any point if + needed +- `GenLex` has been totally refactored and simplified for high level parsers on + tokens. +- Typescript interface is operational and `masala-parser.d.ts` documentation + will generate Masala Parser reference. +- `N.digit()` and `N.digits()` return a number, not a string +- Markdown bundled is removed from the lib, and put as example in typescript + integration Less important : -* `F.layer()` along with `Parser.and()` produces an array of results when all parsers are satisfied. +- `F.layer()` along with `Parser.and()` produces an array of results when all + parsers are satisfied. - it makes backtracking like `F.try()` with `Parser.or()` - warning: it's still experimental for side cases -* Bug correction on offset for high level parsers. - - Random Access in `ParserStream` will result in unpredicted value. Don't do it. -* `response.isConsumed()` is replaced by `response.isEos()` +- Bug correction on offset for high level parsers. + - Random Access in `ParserStream` will result in unpredicted value. Don't do + it. +- `response.isConsumed()` is replaced by `response.isEos()` + 0.6 -> 0.7: Typescript support -0.6 -> 0.7: Typescript support ----- +--- This release has been focused on typescript support. ## Export -* Export is a bit better handled; no more doublon -* Exporting `Parser` instead of `parser` -* `ExtractorBundle` and `TokenBundle` are pruned (#96) +- Export is a bit better handled; no more doublon +- Exporting `Parser` instead of `parser` +- `ExtractorBundle` and `TokenBundle` are pruned (#96) # Functions -Every parser in CharBundle, FlowBundle and NumberBundle need to be called as a function - +Every parser in CharBundle, FlowBundle and NumberBundle need to be called as a +function + //Previously const p = C.letter.then(N.integer).then(F.eos); - + // Now: const p = C.letter().then(N.integer()).then(F.eos()); - -It's less funky, but it avoids construction of Parsers at import statement, before writing the first line code. -### FlowBundle +It's less funky, but it avoids construction of Parsers at import statement, +before writing the first line code. -* `F.startWith(value)`: Creates a Parser with a defined value -* `F.sequence()` is pruned +### FlowBundle +- `F.startWith(value)`: Creates a Parser with a defined value +- `F.sequence()` is pruned ### Typescript and @types -Automated types are enabled with a partial declaration file. Basically, all functions of Parser, CharBundle, -NumberBundle and FlowBundle are supported, but Genlex is not yet. 0.8 release will be focused on Genlex. - +Automated types are enabled with a partial declaration file. Basically, all +functions of Parser, CharBundle, NumberBundle and FlowBundle are supported, but +Genlex is not yet. 0.8 release will be focused on Genlex. - -0.5 -> 0.6 ----- +## 0.5 -> 0.6 ## Response -* Added `response.isCompleted()` +- Added `response.isCompleted()` ## Parser -* `flatmap()` renamed to `flatMap()` +- `flatmap()` renamed to `flatMap()` ## Flow Bundle -* `F.lazy(parser, params, self )` accepts a third `this` parameter -* `F.sequence()` is deprecated - - -Added to the FlowBundle - -* `F.startsWith(value)` -* `F.moveUntil(string|stopParser)` -* `F.dropTo(string|stopParser)` - - - - - +- `F.lazy(parser, params, self )` accepts a third `this` parameter +- `F.sequence()` is deprecated +Added to the FlowBundle +- `F.startsWith(value)` +- `F.moveUntil(string|stopParser)` +- `F.dropTo(string|stopParser)` diff --git a/llm.txt b/llm.txt index 92dba86..707842d 100644 --- a/llm.txt +++ b/llm.txt @@ -1,11 +1,9268 @@ -## Note for AI assistants πŸ€– +You are an AI assistant helping developers use **Masala Parser v2**. -If you're an AI assistant helping a developer: +- Language: JavaScript and TypeScript. +- Core concepts: parser combinators, Parsec-like API, streams. +- Always prefer real APIs and examples from this file. +- If something is unclear or not documented, say it explicitly. + +Below is a curated snapshot of the docs, tests and examples, generated with git-ingest tool. + +Each section starts with : +" +================================================ +FILE: +================================================ +" + + + +Directory structure: +└── masala-masala-parser/ + └── ingest/ + β”œβ”€β”€ README.md + β”œβ”€β”€ documentation/ + β”‚ β”œβ”€β”€ chain.md + β”‚ β”œβ”€β”€ chars-bundle.md + β”‚ β”œβ”€β”€ flow-bundle.md + β”‚ β”œβ”€β”€ genlex.md + β”‚ β”œβ”€β”€ numbers-bundle.md + β”‚ β”œβ”€β”€ parser-core-functions.md + β”‚ β”œβ”€β”€ parser-extension-functions.md + β”‚ β”œβ”€β”€ publish.md + β”‚ β”œβ”€β”€ recursion.md + β”‚ β”œβ”€β”€ troubleshooting.md + β”‚ └── tuple.md + β”œβ”€β”€ examples/ + β”‚ β”œβ”€β”€ index.ts + β”‚ β”œβ”€β”€ 1-easy/ + β”‚ β”‚ β”œβ”€β”€ chars.spec.ts + β”‚ β”‚ β”œβ”€β”€ floor.spec.ts + β”‚ β”‚ β”œβ”€β”€ hello-something.spec.ts + β”‚ β”‚ β”œβ”€β”€ number.spec.ts + β”‚ β”‚ β”œβ”€β”€ occurence.spec.ts + β”‚ β”‚ └── response.spec.ts + β”‚ β”œβ”€β”€ 2-medium/ + β”‚ β”‚ β”œβ”€β”€ f-layer.spec.ts + β”‚ β”‚ β”œβ”€β”€ filter-match.spec.ts + β”‚ β”‚ β”œβ”€β”€ flat-map.spec.ts + β”‚ β”‚ └── real-world-flatmap.spec.ts + β”‚ β”œβ”€β”€ brainfuck/ + β”‚ β”‚ └── compiler.spec.ts + β”‚ β”œβ”€β”€ chars/ + β”‚ β”‚ β”œβ”€β”€ charIn-notChar.spec.ts + β”‚ β”‚ └── letter-letterAs.spec.ts + β”‚ β”œβ”€β”€ flow/ + β”‚ β”‚ β”œβ”€β”€ nop-any-eos.spec.ts + β”‚ β”‚ β”œβ”€β”€ not.spec.ts + β”‚ β”‚ β”œβ”€β”€ startWith-moveUntil-dropTo.spec.ts + β”‚ β”‚ └── try-with-no-or.spec.ts + β”‚ β”œβ”€β”€ lazy/ + β”‚ β”‚ └── transmission.spec.ts + β”‚ β”œβ”€β”€ markdown/ + β”‚ β”‚ β”œβ”€β”€ lib/ + β”‚ β”‚ β”‚ β”œβ”€β”€ bullet-parser.ts + β”‚ β”‚ β”‚ β”œβ”€β”€ code-line-parser.ts + β”‚ β”‚ β”‚ β”œβ”€β”€ document-parser.ts + β”‚ β”‚ β”‚ β”œβ”€β”€ front-matter.ts + β”‚ β”‚ β”‚ β”œβ”€β”€ text-parser.ts + β”‚ β”‚ β”‚ β”œβ”€β”€ title-parser.ts + β”‚ β”‚ β”‚ β”œβ”€β”€ token.ts + β”‚ β”‚ β”‚ └── types.ts + β”‚ β”‚ └── test/ + β”‚ β”‚ β”œβ”€β”€ bullet-test.ts + β”‚ β”‚ β”œβ”€β”€ code-block-test.ts + β”‚ β”‚ β”œβ”€β”€ document-test.ts + β”‚ β”‚ β”œβ”€β”€ front-matter.spec.ts + β”‚ β”‚ β”œβ”€β”€ markdown-test.ts + β”‚ β”‚ β”œβ”€β”€ text-test.ts + β”‚ β”‚ β”œβ”€β”€ title-test.ts + β”‚ β”‚ └── samples/ + β”‚ β”‚ β”œβ”€β”€ paragraph-expected.json + β”‚ β”‚ β”œβ”€β”€ paragraph-test.md + β”‚ β”‚ β”œβ”€β”€ trivial-expected.json + β”‚ β”‚ └── trivial-test.md + β”‚ β”œβ”€β”€ operations/ + β”‚ β”‚ └── plus-minus.spec.ts + β”‚ β”œβ”€β”€ opt-and-rep/ + β”‚ β”‚ β”œβ”€β”€ forever-parse.spec.ts + β”‚ β”‚ β”œβ”€β”€ forever-parse.worker.mjs + β”‚ β”‚ β”œβ”€β”€ opt.spec.ts + β”‚ β”‚ β”œβ”€β”€ optrep.spec.ts + β”‚ β”‚ └── rep.spec.ts + β”‚ β”œβ”€β”€ tuple/ + β”‚ β”‚ └── tuple.spec.ts + β”‚ └── tuple-parser/ + β”‚ └── tuple-parser.spec.ts + └── test/ + β”œβ”€β”€ data/ + β”‚ β”œβ”€β”€ option.spec.js + β”‚ β”œβ”€β”€ try.spec.js + β”‚ └── tuple.spec.js + β”œβ”€β”€ genlex/ + β”‚ β”œβ”€β”€ genlex-brainfuck.spec.js + β”‚ β”œβ”€β”€ genlex-coverage.spec.js + β”‚ β”œβ”€β”€ genlex-event-tracer.spec.js + β”‚ β”œβ”€β”€ genlex-nospace.spec.js + β”‚ β”œβ”€β”€ genlex-priority.spec.js + β”‚ β”œβ”€β”€ genlex-tracer-minimal.spec.js + β”‚ β”œβ”€β”€ genlex.spec.js + β”‚ β”œβ”€β”€ tracing-genlex-coverage.spec.js + β”‚ └── tracing-genlex-flush.spec.js + β”œβ”€β”€ parsec/ + β”‚ β”œβ”€β”€ chars-bundle-regex.spec.js + β”‚ β”œβ”€β”€ chars-bundle-test.spec.js + β”‚ β”œβ”€β”€ f-layer-test.spec.js + β”‚ β”œβ”€β”€ flow-bundle-test.spec.js + β”‚ β”œβ”€β”€ flow-move-until.spec.js + β”‚ β”œβ”€β”€ flow-regex-flags.spec.js + β”‚ β”œβ”€β”€ flow-regex.spec.js + β”‚ β”œβ”€β”€ flow-try-or.spec.js + β”‚ β”œβ”€β”€ number-bundle-test.spec.js + β”‚ β”œβ”€β”€ parser-tuple-map.spec.js + β”‚ β”œβ”€β”€ parser_chain_test.spec.js + β”‚ β”œβ”€β”€ parser_core_default_test.spec.js + β”‚ β”œβ”€β”€ parser_core_test.spec.js + β”‚ β”œβ”€β”€ parser_extensions_test.spec.js + β”‚ β”œβ”€β”€ parser_stream_test.spec.js + β”‚ β”œβ”€β”€ response_test.spec.js + β”‚ └── tuple-parser-test.spec.js + β”œβ”€β”€ specific/ + β”‚ └── f-empty-try.spec.js + └── stream/ + β”œβ”€β”€ array-stream-test.spec.js + β”œβ”€β”€ buffered_stream_test.spec.js + β”œβ”€β”€ offset-test.spec.js + └── stream_test.spec.js + +================================================ +FILE: ingest/README.md +================================================ +# Masala Parser: Javascript Parser Combinators + +[![npm version](https://badge.fury.io/js/%40masala%2Fparser.svg)](https://badge.fury.io/js/%40masala%2Fparser) +[![Coverage Status](https://coveralls.io/repos/d-plaindoux/masala-parser/badge.png?branch=master)](https://coveralls.io/r/d-plaindoux/masala-parser?branch=master) +[![stable](http://badges.github.io/stability-badges/dist/stable.svg)](http://github.com/badges/stability-badges) + +Masala Parser is an Open source javascript library to create your own parsers. +You won't need theoretical bases on languages for many usages. + +Masala Parser shines for **simplicity**, **variations** and **maintainability** +of your parsers. Typescript support and token export for AI processing will also +help you in the debug process. + +![absolute-demo.png](documentation/images/absolute-demo.png) + +Masala Parser started in 2016 as a Javascript implementation of the Haskell +**Parsec** and is inspired by the paper titled: +[Direct Style Monadic Parser Combinators For The Real World](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/parsec-paper-letter.pdf). + +It is plain Javascript that works in the browser, is tested with more than 500 +unit tests, covering 100% of code lines. + +### Use cases + +Here are the pros of Masala Parser: + +- It can create a **full parser from scratch** +- It can **replace complex regexp** +- It works in any **browser** or NodeJS +- There is an **incredible typescript** api +- It has some **good performances** in speed and memory +- There is zero dependency +- Masala is actively supported by [Robusta Build](https://www.robusta.build) + +# Usage + +We made a **7 minutes** Youtube video to explain how to create a parser: + +[![Masala Parser Youtube Video](documentation/images/masala-yt.png)](https://www.youtube.com/watch?v=VNUrvWdtM2g) + +### Installation + +With Node Js or modern build + + npm install -S @masala/parser + yarn add @masala/parser + +Or in the browser, using Javascript ES Modules: + + import {F, standard, Streams} from 'https://unpkg.com/@masala/parser@2.0.0 + +- [download Release](https://github.com/d-plaindoux/masala-parser/releases) +- `