More flexible splitting of the bibliography

[alyst]

Fixes #15.

The limitation of the current Alexandria approach is that bibliography lists are limited to a single prefix:
the splitting of the bibliography discussed in the example is by the reference type, not by the prefix.

This PR allows more flexibility:

• different prefixes could be used interchangeably throughout the document (also alexandria-prefix() allows setting the default prefix).
• load-bibliography() is replaced by the collect-citations(id, prefix-filter: ...) which allows creating lists of citations and references with mixed prefixes.
• render-bibliography() is simplified: instead of requiring the user to manually compose the filtered list of references,
  it accepts the ID of the bibliography list created by collect-citations(),
  and allows further filtering by prefixes/details with the user-specific filter: ref => bool function.

The regression is that full is not supported (since the list is assembled from multiple prefixes).

The manual and the examples are updated.

Unfortunately, it turned out to be much more invasive that I have anticipated, but I hope it can be useful.

Also, it's my first experience of writing something in Typst (and in Rust) :)

[ensko]

Hi @alyst, thank you for your contribution. Unfortunately it is really hard to review this PR as it makes many unrelated changes as well. From what I could quickly see, it

• makes general changes to the manual (not related to new features)
• bumps the version in cargo.toml (despite the plugin not being published on crates.io, i.e. the version doesn't really mean anything)
• does unrelated refactors, e.g. renames data structures (Entry to Reference )

Also there's stuff about citegroups in there that has nothing to do with the features listed in your PR description.

Some changes are imo outright detrimental:

• bump the version in typst.toml outside of making a release
• merge internal.typ into hayagriva.typ. Since hayagriva is exported from lib.tyb, this makes the functions visible to users of the package, which was the express reason for having a separate module
• change the semantics of render-bibliography.title to something that's not compatible with std.bibliography.title
• some variables in Typst code use snake_case instead of kebab-case convention

The regression of full bibliographies no longer working is of course also very serious and by itself blocks merging this.

I appreciate you helping out with Alexandria, but it would really be important to bring this PR in a better shape. I would maybe leave this PR as-is, and create new, more focused PRs for individual features you'd like to see. Even if the feature is somewhat "invasive" to implement, there's quite a bit of unrelated changes that could be removed from this PR.

Two more remarks that are not directly about your PR, but may influence your future contribution:

• The fact that Alexandria uses prefixes at all is actually due to limitations; I think when I designed it, I was under the impression that I need prefixes to only process the citations/references I'm supposed to. If there's a way around that/I was outright mistaken, getting rid of prefixes would be preferable.
• There was some recent discussion about implementing multiple bibs directly in Typst. I don't know if that work has started/has priority right now, but it may be that Alexandria is not needed for much longer!

[alyst]

@SillyFreak Thanks for your initial review! I agree with most of your comments -- clearly I have marked it as a draft since I myself consider the PR needs improvements, and your feedback was required.
So I will try to work on it, if you still think you would like to see this implemented.
To do this, can you please clarify some of your comments:

bump the version in typst.toml outside of making a release

That's just to disambiguate the new development version from the released one.
Different platforms have different policies of when/how the version bump happens.
Please let me know what you will prefer / what is recommended the way for typst.

merge internal.typ into hayagriva.typ

I will try to pay more attention to what is internal/public.
I moved it because these functions clearly belonged to hayagriva processing and constituted just a few lines of code.

change the semantics of render-bibliography.title to something that's not compatible with std.bibliography.title

I can completely revert it, but it is also quite logical that for multiple bibliographies one would like to have more control over how the title is rendered, e.g. whether these titles should always be at the first level.
Of course, that could be tweaked with the # set rules, as done in the example, but maybe something could be done in render-bibliography to make it more straightforward.

some variables in Typst code use snake_case instead of kebab-case convention

I tried to keep all public identifiers kebab case and use snake case for the internal ones, but I can make internal ones kebab-case too.

The regression of full bibliographies no longer working is of course also very serious and by itself blocks merging this.

I agree. I can add full to the alexandria() call, where the sources are defined (it is possible to have multiple alexandria calls).

The fact that Alexandria uses prefixes at all is actually due to limitations; I think when I designed it, I was under the impression that I need prefixes to only process the citations/references I'm supposed to. If there's a way around that/I was outright mistaken, getting rid of prefixes would be preferable.

I think prefixes have the same goal as biblatex categories (\DeclareBibliographyCategory), but are actually easier to use, because you don't have a separate instruction to map your reference to a category (\addtocategory), see biblatex manual.
Whereas prefixes could be considered a workaround for section-specific bibliographies, I think they are a natural thing for generic citations.
They could be called tags/categories, and the syntax may be @key[tag=first], but they still should be a part of the multiple bibliographies mechanism.
At the moment hayagriva does not have support for categories and for "runtime" tweaking of the reference lists.
Also, render-bibliography() + collect-citations() try to match \printbibliography (the latter has more complex numbering logic, which I think is only possible to implement in typst/bibliography.rs)

There was some recent discussion about implementing multiple bibs directly in Typst. I don't know if that work has started/has priority right now, but it may be that Alexandria is not needed for much longer!

It would be great if Typst naturally supports it and have biblatex feature parity. When I did my research few months ago, I have not seen these discussions.
Particularly, some things like detection of citation groups and proper numbering logic are hard to implement and make fast in the Typst package, at least with the current Typst capabilities.
If that is something they are actively working on and plan to release soon, there's no need to work on this PR -- I just got it working for my document. :)
Otherwise I have seen people requesting the features that this PR implements.

makes general changes to the manual (not related to new features)

I can move them to a separate PR: when I was first reading it, it was not clear for me that alexandria() requires the use of prefixes in citations, and I was confused a bit with how the example explanation and the example contents are split into sections.

does unrelated refactors, e.g. renames data structures (Entry to Reference)

Again, I can move it into a separate PR, but that classes' and fields' renames were meant to more closely follow the hayagriva names (and the field names in alexandria itself).

[ensko]

Please let me know what you will prefer / what is recommended the way for typst.

Yes, I've seen platforms where the conventions are different. I don't think Typst as a whole has clear conventions, but my own are: only bump the version on release, since that is when you can be sure which version number should be bumped.

I can completely revert it, but it is also quite logical that for multiple bibliographies one would like to have more control over how the title is rendered, e.g. whether these titles should always be at the first level.

I'd prefer to revert since I want to perfectly mirror the built-in APIs wherever possible, and bibliography() always creates a level 1 heading. The workarounds are using title: none and set heading(offset: ...), which users of the built-in bibliography are already (potentially) familiar with.

I tried to keep all public identifiers kebab case and use snake case for the internal ones, but I can make internal ones kebab-case too.

May convention is to use kebab-case for all Typst identifiers. If there are exported but logically internal identifiers, I'd prefix them with an underscore (e.g. _csl-to-string), but I also try to avoid that when possible.
Related, my state identifiers are e.g. __alexandria-config: since they're not only exported but global, they get an extra leading underscore and the package name to make accidental collisions even more unlikely, but otherwise they're kebab-case.

(I probably should put some of that in a contributions document, so that people can refer to it from the start)

At the moment hayagriva does not have support for categories and for "runtime" tweaking of the reference lists.

I think the Hayagriva side is not that bad actually: you have a driver that you add citation requests to (e.g. all references for a full bib here), and if you filter that first, you have categories. Custom numbering and sorting is a bit harder this way, though.

When I did my research few months ago, I have not seen these discussions.

Yes, this is very recent, last month. See here if you're interested: https://discord.com/channels/1054443721975922748/1246016569323159584/1383386231370420276

I can move them to a separate PR: when I was first reading it, it was not clear for me that alexandria() requires the use of prefixes in citations, and I was confused a bit with how the example explanation and the example contents are split into sections.

Yes please. Doc improvements are always welcome, and I unfortunately can't look at docs of my own package with an unbiased eye. It's just that in this PR, it distracts from the rest of the changes.

but that classes' and fields' renames were meant to more closely follow the hayagriva names (and the field names in alexandria itself)

Agreed; that struct probably predates me building a more complete mental model of the Hayagriva side. Resource sounds like a sensible name.

---

Re the immediate future of this PR:

If that is something they are actively working on and plan to release soon, there's no need to work on this PR

I don't know the exact timeline. If my time this summer permits I would like to help in the implementation myself, but I also can't make any guarantees.

I think it would make sense to try and describe the desired API in detail first, so that even if Typst still takes a while, Alexandria has a better chance to go in the direction Typst will eventually take. I like what I see in the manual though; I think this could be broadly the way to go.

[ensko]

fyi, I have written down some design notes here and also shared them with the Typst team to see if that is a direction they consider promising: https://typst.app/project/rOOEGr1WPuYxg25tAkEz09

This design does not use prefixes, but I think the bibliography-fence() is in effect very similar to your alexandria-prefix(): it makes sure that all citations in a region of a document refer to a certain bibliography.

I'd be interested in your thoughts, including whether you see anything that is impossible or less convenient with that design.