Configuration format

[touchmarine]

Touch uses a configuration file to configure the parser and formatter. It tells Touch which elements to recognize and defines the templates used for rendering. Currently, the only supported configuration format is JSON (and the templates are defined using Go templates). This issue is about whether JSON should stay the sole configuration format or would it be beneficial to add other formats, presumably with more features.

The goal of a configuration format is that it's easy to read, write, and understand. The barrier to entry should be low, even for newcomers. That means it mustn't be complex and should be already well-known. Likewise, it should be available on as many platforms as possible.

JSON was chosen because it's simple and widespread—it has a simple feature set, explicit syntax (no YAML-like quirks), is well-known (at least amongst developers), and is widely available (e.g. in Go's standard library and in the browser). However, JSON is a data interchange format and is missing a couple of features that would be nice to have, such as multiline literal strings and comments.

Writing and reading templates in plain JSON is difficult. You need to use escape sequences for newlines, tabs, and backslashes. To try and solve this in the simplest way possible, I created a tool called extjson (to tool extjson) which acts as a preprocessor. extjson is used to convert "extended" JSON to plain JSON before using it. Extended JSON is like JSON but also supports TOML-like multiline literal strings. (Actually, extended JSON doesn't even parse JSON, it only searches for multiline literal strings and converts them to JSON strings.)

For example, this extended JSON:

"Templates": {
	"html": '''
<blockquote {{- template "HTMLAttributes" .}}>
	{{template "children" .}}
</blockquote>
'''
}

converts to this JSON:

"Templates": {
	"html": "<blockquote {{- template \"HTMLAttributes\" .}}>\n\t{{template \"children\" .}}\n</blockquote>\n"
}

By making extjson a tool and not directly implementing it, Touch keeps its options open and is by no means bound to it or extended JSON. Plain JSON, on the other hand, will almost certainly be always supported by Touch. As such, any other configuration format would be added in addition to it.

Another feature I think would be beneficial are comments. They would allow for documenting the config and thus making it easier to navigate and understand, especially for newcomers. Currently, comments can only be added in templates using the Go template comments. But maybe that's enough?

Here is a list of common configuration and data interchange formats which offer additional features:

• YAML—superset of JSON, widespread; complex and has a lot of quirks
• TOML—similar syntax to INI, less known tables notation instead of objects
• HCL—JSON-like syntax, accepts also JSON; does too much/is too much of a language (has expressions, templates, etc.)
• CUE—data validation language that can be used for configuration; does too much

All of these formats and more seemingly fit the bill but I am wondering whether they're worth the added complexity. They are either too complex and do too much or aren't very well-known or widespread. But at the same time, only JSON is probably not enough as I already needed to introduce extended JSON which is totally unknown, confusing to newcomers, and not available on any platform.

I'm interested in hearing your thoughts. At the same time, I will post any updates on the issue here as I go along and use JSON and extended JSON for longer periods of time and encounter new pain points.

[touchmarine]

If I understand you correctly, you are suggesting using HCL as the format for the syntax tree (e.g. DOM in HTML) like described in the atlas schemaspec link. That's pretty neat, but I don't think it's necessary.

I think it might be helpful if I explain how Touch's parser works. Basically, it works like most markdown parsers but it's generalized. While most parsers know how to parse each specific element, the Touch's parser knows only how to parse types of elements. (See TOUR for more.)

The config that you feed into the parser defines the set of elements. The elements are recognized based on their element type (each type has its own rules for how the element can be written) and the delimiter/marker used (elements of different types are differented by their delimiters). The parser then generates a node tree (like DOM) which carries the name of the parsed element and other useful information like its type and children.

Then, to render a format (e.g. export to HTML), the generated node tree is walked and for each encountered element a template is executed. The template that will be executed is defined in the config inside elements.

In conclusion, Touch already has a way to accept parameterized elements and output different formats. I hope I understood you correctly and thanks for the help! And HCL having a LSP is definitely a plus.