Compared with Incremark

Incremark was our first streaming Markdown tool. QuireKit Markdown continues the same goal: show formatted content while an AI response is still arriving.

The redesign addresses an easy-to-miss problem: text received later can change how an earlier sentence should appear. The new engine reduces repeated parsing and rendering while allowing later definitions to update earlier text correctly.

Watch a tree change

“Middle insert” changes the same source three times: bold text in the second paragraph, inline code in the first, then a two-item list between paragraphs. Each edit builds on the previous result.

Play to follow each fragment into the source and along the highlighted path to an amber affected branch, followed by blue updated nodes. Green branches retain their native objects. Pause, step, go back, or select an insertion and phase. The view has no drag, pan or zoom.

The third edit is at a paragraph boundary. Although the final paragraph's literal text is unchanged, its native branch is rebuilt: the affected scope depends on syntax and context, not just changed characters. “Tail append” and “Late reference” remain available; in the latter, bar becomes the earlier link. Trees and output come from the real engine. The motion is teaching choreography, not an internal execution log.

This tree animation also uses our Flowchart library: nodes, edges, moving fragments and tree replacements share one public player.

Compare the earlier flow

Switch between “Earlier Incremark” and “Current algorithm” to follow a new chunk through the system. Our existing Flow Editor renderer provides zoom and a larger view.

Why redesign it

Earlier Incremark divided a growing response into finished blocks and the part still being written. When more text arrived, it tried to preserve earlier parse results and reparse only the tail. That reduced repeated work.

But finishing a paragraph does not always settle its meaning. An AI can write the text first and provide a link destination at the end:

更多内容见[文档][api]。

[api]: https://example.com/docs

Until the last line arrives, [文档][api] has no destination. Once it arrives, “文档” should become a link. Footnotes work similarly: a reference can appear long before its explanation.

The earlier version handled these cases by recording references and filling in their destinations later. The difficulty is that a later definition can also change which brackets form a link. Adding a destination to an existing node is then insufficient. Earlier syntax needs another look, and the page needs the corresponding update.

Two concrete problems in the earlier implementation

Defining the same link twice

Suppose the paragraph and each destination arrive in order:

[文档][api]

[api]: /first

[api]: /second

Markdown uses the first definition of a label, so “文档” should link to /first. The earlier implementation overwrote the first value when merging definitions, changing the link to /second. The current engine preserves the first definition.

Linking the wrong word

Consider another short example. The first line arrives, followed by the definition for baz:

[foo][bar][baz]

[baz]: /target

The intended reading is literal [foo], followed by the link [bar][baz]. The clickable word is bar.

The earlier version had already recognized [foo][bar] as one reference and left [baz] as another. When the destination arrived, baz became clickable. Filling in the address could not correct the earlier grouping. The current engine checks grouping against the definitions that actually arrived, so bar becomes the link.

We ran both earlier engines on these examples again:

How the new algorithm handles a stream

1. Keep the source that arrived

Every append updates the same Markdown source. Stopping preserves the received text without adding fences, brackets or HTML closing tags for display. Switching messages starts a new parsing session.

2. Let Lezer identify reusable parse results

Lezer records headings, paragraphs, lists and links in a syntax tree. When text arrives, it uses syntax context to decide which earlier results remain usable and which need parsing again. We use that mechanism for streamed appends.

For example, starting a paragraph at the end can leave unaffected earlier paragraphs reusable. Changes to lists or fences are handled according to their syntax, rather than treating a blank line alone as proof that earlier content is finished.

3. Let later definitions correct earlier references

The new engine collects the current link definitions, then checks which paragraphs' reference decisions were affected. [foo][bar][baz] needs a new grouping decision; repeated definitions keep the first value.

It also records labels a paragraph queried, including those not yet defined. When those records provide enough evidence that a new definition did not affect the paragraph, its result remains reusable.

This whiteboard shows the dependencies: when api arrives, paragraph B needs another check. An introduction and a paragraph that only queried faq can remain reusable when their other inputs match.

4. Check displayed content before reusing it

Unchanged syntax can still have a different link destination or footnote number. The rendering cache checks those inputs too, along with components and HTML configuration. Matching entries retain their output; affected blocks, or blocks without enough evidence, are generated again.

As the AI continues writing, reusable earlier content stays in place. When a later definition changes its meaning, earlier links and footnotes can still update.

Who owns these steps

The core owns parsing, reference decisions and cache validation. Vue and React own their components and commit lifecycle; the application owns network delivery and optional diagram renderers.

All three diagrams retain their native source. You can also integrate these flowchart, mind-map and whiteboard renderers in your own documentation.

Watch a stream

Start the example to watch text arrive before its link and footnote definitions. The same footnote is referenced twice, and its body also contains a link. Stop to retain the text received so far.

You can also paste either short example into the Playground to inspect the new engine's output.

What about speed

Earlier Marked is fast. Switching to Lezer does not make every Markdown input faster to parse. The current optimizations reduce two kinds of repeated work: parsing unaffected text again and rebuilding reusable display output.

An earlier original-demo parse measurement put Lezer/MSFE ahead of the old Micromark and behind the old Marked. Those measurements predate the current rendering cache and cannot rank the current version.

A large list, table or long paragraph can still be processed as one block. Streaming performance depends on parsing, rendering and browser updates together. See performance for the measurement method.

Using it and migrating

QuireKit Markdown supports Vue and React with shared parsing rules. Basic usage still takes Markdown text. You can replace links, code blocks and other elements with your components, add custom syntax, and explicitly enable HTML.

The new package does not retain earlier Incremark APIs, so existing plugins need adaptation. Start with Vue or React, then follow custom syntax and nodes for extensions. This remains an unreleased preview. Earlier Incremark and the current core, Vue and React packages all use MIT.

Further implementation reading

These details are useful when extending the parser or studying the cache:

• Syntax reuse: CodeMirror maps text transactions and TreeFragment. Our Lezer layer exposes top-level block reuse while preserving native grammar and reuse decisions. Reference handling belongs to the MSFE parser wrapper.
• Reference decisions: Complete paragraphs record labels actually queried by grammar and whether they exist. Those decisions are reused only when query results match; ranges without complete records are checked conservatively. Rendering separately checks destination and title changes.
• Rendering cache: Entries check native Tree identity, source range, reference targets, footnote navigation and rendering configuration. Custom nodes can declare additional dependencies with cacheKey(document); without it, output is generated again. See performance for conditions and cache limits.
• Frameworks and resources: Vue and React have their own components. React accepts candidate caches after commit and starts resources after client commit. Cancellation and release handle switching, unmount and late results; see React integration.

The two examples were checked on 2026-10-11 against @incremark/core 1.0.2, commit 98f6b95. Earlier results come from each engine's AST and definitions; current results come from React server rendering, checked against an independent full GFM parse and CommonMark reference rules. These examples identify specific problems in that version, without extending the findings to later Incremark versions.

Source: earlier parser session, reference recognition, definition merging.