This basically reverts the earlier change from parbreaks to par nodes because:
- It is simpler and less nested
- It works way better with functions that layout their body inline like `font`, which where buggy before, previously
The original reasons for changing to par nodes were:
- the envisioned design of the layouter at that time (based on dynamic nodes etc.), which is not relevant anymore
- possibly existing benefits with regards to incremental compilation, which are unsure and outweighed by the immediate benefits of the parbreak-representation
- Refactors the tokenizer to be lazy: It does not emit pre-parsed function tokens, but instead allows it's mode to be changed. The modes are tracked on a stack to allow nested compute/typesetting (pop/push).
- Introduces delimited groups into the parser, which make it easy to parse delimited expressions without handling the delimiters in the parsing code for the group's content. A group is started with `start_group`. When reaching the group's end (matching delimiter) the eat and peek methods will simply return `None` instead of the delimiter, stopping the content parser and bubbling up the call stack until `end_group` is called to clear up the situation.
- In addition to syntax trees there are now `Value`s, which syntax trees can be evaluated into (e.g. the tree is `5+5` and the value is `10`)
- Parsing is completely pure, function calls are not parsed into nodes, but into simple call expressions, which are resolved later
- Functions aren't dynamic nodes anymore, but simply functions which receive their arguments as a table and the layouting context
- Functions may return any `Value`
- Layouting is powered by functions which return the new `Commands` value, which informs the layouting engine what to do
- When a function returns a non-`Commands` value, the layouter simply dumps the value into the document in monospace
- Use fontdock for indexing fonts and querying
- Typst binary now automatically indexes and uses system fonts in addition to a fixed font folder!
- Removes subsetting support for now (was half-finished anyways, plan is to use harfbuzz for subsetting in the future)
- Adds font width configuration support
- Problems -> Diagnostics
- Position -> Pos
- offset_spans -> Offset trait
- Size -> Length (and some more size types renamed)
- Paper into its own module
- scope::Parser -> parsing::CallParser
- Create `Decorations` alias
- Remove lots of double newlines
- Switch from f32 to f64
- Forced line breaks with backslash followed by whitespace
- (Multline) raw text in backticks
- Set font class fallbacks with [font.family] (e.g. [font.family: monospace=("CMU Typewriter Text")])
- More sophisticated procedure to find end of function, which accounts for comments, strings, raw text and nested functions (this is a mix of a feature and a bug fix)
- Dynamic models instead of SyntaxTrees
- No more ParseResult/LayoutResult
- Errors and Decorations which are propagated to parent contexts
- Models are finally clonable
Laurenz
Martin Haug