☕️ Chicago's second favorite bean
This library provides a configurable way to (1) collate data from multiple sources into a single denormalized dataframe, (2) tokenize the records into sequences of discrete integers, (3) winnow the records in preparation for downstream inference tasks, and (4) visualize both individual records and the tokenization process. It benefits from previous experience preparing data for training foundation models on tokenized electronic health records. 1 2 3 4 5
Install the latest release from PyPI:
pip install cocoa-tokenizerThis installs the cocoa command. To see it run end to end on public data,
follow the
MIMIC-IV demo,
which installs cocoa with its demo extra (on Python 3.11 to 3.14):
pip install 'cocoa-tokenizer[demo]'The extra brings clifpy, the CLIF
consortium's Python package, which ships the demo as CLIF tables, and
cocoa get-demo writes them out.
Each stage of the pipeline is a cocoa command, driven by a YAML config, and
cocoa pipeline runs the first three in turn:
collate → tokenize → winnow
↘ visualize
The documentation covers:
cocoa collate: pulls raw tables into a single denormalized, MEDS-like table of events, and splits the subjects into train, tuning, and held-out sets.cocoa tokenize: turns each subject's events into a sequence of integer tokens, with a vocabulary and quantile bins learned on the training split.cocoa winnow: cuts timelines into a past and a future at a threshold and flags the outcome tokens on each side, for evaluation.cocoa visualize: draws a subject's tokenized timeline, or the learned vocabulary, as an interactive page.cocoa combine-datasets: merges processed directories into one.- Configuration: the YAML config each stage reads, and how to change its keys on the command line.
- The CLI reference, with every command's options, and the recipes, worked examples from a run on public data to tokenizer transfer.
Contributions are welcome, from bug reports and typo fixes to new features. To get started, see the contributing guide.
For training models and running inference on this data, check out cotorra.
Footnotes
-
M. Burkhart, B. Ramadan, Z. Liao, K. Chhikara, J. Rojas, W. Parker, & B. Beaulieu-Jones, Foundation models for electronic health records: representation dynamics and transferability, arXiv:2504.10422 ↩
-
M. Burkhart, B. Ramadan, L. Solo, W. Parker, & B. Beaulieu-Jones, Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models, Pacific Symposium on Biocomputing 31 (2026), 173–188 ↩
-
L. Solo, M. McDermott, W. Parker, B. Ramadan, M. Burkhart, & B. Beaulieu-Jones, Efficient generative prediction for EHR foundation models: the SCOPE and REACH estimators, arXiv:2602.03730 ↩
-
I. Lee, L. Solo, M. Burkhart, B. Ramadan, W. Parker, & B. Beaulieu-Jones, Representation before training: a fixed-budget benchmark for generative medical event models, arXiv:2604.16775, to appear in ML4H 2026 ↩
-
M. Burkhart, L. Solo, I. Lee, S. Charles, Z. Liao, K. Chhikara, D. Therese, W.-T. Liao, C. Gao, W. Parker, & B. Beaulieu-Jones, Federated generative event models for tokenized electronic health records, arXiv:2608.02939 ↩
