Neorg link system

Continue from yesterday.

For local links ({this} or {* this}), nothing is problem... or really?

* Link resolving

convert link target query like : this : * foo to rich object

...we can't generalize link resolving because it may require different response by context.

e.g. neorg or neorg-language-server will require lsp_types::Location object. But norgolith requires anchor tag.

So two are completely different thing..?

Let's see what norgolith will do internally to convert every app-links to anchors:

1. from workspace, path, scope, first get root_uri for workspace (it might be under https://!)

2. resolve path to absolute path starting from workspace uri.

   • if workspace is not provided (for current workspace), strip workspace abspath from abspath. (here we need to resolve the link like neorg or neorg-langauge-server)

   • if workspace is provided, join path directly to workspace uri.

// used everywhere
AnchorReference(Markup) -> LocalTarget(..) | AppTarget(..)

// used for language-server
AppTarget(Option<Workspace>, AppPath, Scope) -> Location<AbsPath, NodeHash>

// used to export link (only used on norgolith)
Location<AbsPath, NodeHash> -> {unknown}

hash node based on its type, range, and position from entire AST.

NOTE: When hashing section, include heading range instead of full section range. Because section content can change quite often

HTML exporter should be included in stdlib. (to have standard, unified way to export markup) When exporting a linkable, check if it's app-link.

1. If linkable is an anchor reference, resolve it to get Target which is an enum that can either be LocalTarget(..) or AppTarget(..)

   use (norg/resolve-anchor ctx [pos markup]) to resolve an anchor refernce. ctx should hold the entire AST to be used in this scenario. This api will internally use (neorg/resolve-anchor path [pos markup]) when neorg/resolve-anchor is avaliable.

2. Convert linkable to export target (e.g. anchor tag string for HTML)

   • If the link target is LocalTarget, convert locally from stdlib (norg/export/linkable export-target target markup?).

   • If the link target is AppTarget, use (neorg/export/linkable export-target target markup?) instead.

Wait, how should I resolve the anchor? There are multiple approaches to design this:

• Always use last definition from entire document (enabling anchor definitions placed at bottom of document)

  • But what is a last definition? What if it is at bottom by position, but inside a heading?

  • Also, we loose the ability to change the anchor definition in-document.

• Use programming language style approach (scoping, shadowing).

  • But this prevents placing anchor definition after the reference.

  • Also, do we really need scoping?

Document example to give more clear view:

[anchor] A

[anchor]{definition 1}

[anchor] B

* heading
  [anchor]{definition 2}

  [anchor] C (scoped)

  [anchor]{definition 3}
*
[anchor] D

[anchor]{definition 4}

* heading
  [anchor]{definition 5}

Thoughts:

• I'm fine with not being able to put anchor definitions at bottom. It feels weird but it's fine. It's just a different approach.

• I don't think we need to scope links by default. I think people won't care about scopes a lot when writing a document. Also, paragraphs/markups might confuse them about "which element creates a scope and which does not?" Can we make it opt-out? Maybe in form of [anchor]{definition}(scoped).

Conclusion:

Use most recent definition (ignoring section scopes. find definition by physical position)

@document.meta
title: 2025-05-06
created: 2025-05-06T04:50:04+00:00
updated: 2025-05-06T04:50:04+00:00
@end
* (hide) References
  #id ref-list
  - [neorg]{:/doro/neorg}
  - [typst]{:/doro/typst}
  - [commonmark]{:/doro/commonmark}
*

[neorg]

• #id ref-list is used to locate the unordered list containing anchor definition

• last *\n is used to set section level to 0.

WAIT NO

I completely missed the footnotes/definitions. They SHOULD come after the link.

Actually nevermind. They are not a double reference like anchors. They use node id and query to target the footnote definition from document. Which is done in AST root level anyways.

paragraph\fn(1)

@def-fn 1
footnote definition
@end

 |
 V

paragraph{# fn-1}[1](superscript)

#(id fn-1)
@group
footnote definition
@end

/posts page from norg document

Now with concrete base of workspace-aware link system, making .ul-docs infirm tag is way easy. I just need to add (neorg/create-app-target base path) api to generate clean app-links from absolute path to each documents.

What's next:

• implement (norg/resolve-anchor ctx markup)

• implement (neorg/doc/read-meta path) to get metadata and check for fields like title, updated or draft.

  change ctx.meta type to struct (not table), iterate through document until ctx.meta is not nil

• after all those, implement dynamic janet query caching system

  • organize neorg APIs to be pure function without side effects

  • setup NorgBerg server to memoize dynamic queries

idea: dynamic janet query

Using raw SQL query to query from DB doesn't feel extensive. So how about just using janet script? We can cache the output by saving both script and result in DB.

But we should be careful and try to avoid side-effects. e.g. querying document that generates metadata dynamically.

To prevent this, those janet scripts used as a query should be pure function. Function that doesn't have side-effects. Function that always return same output from same input.

We can capture the workspace state (like how git does), pass to function and memoize the query output.

This sounds fun.