Skip to content

About

A configurable collator and tokenizer

Topics

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

 
 

Repository files navigation

cocoa bean

Cocoa: a configurable collator

PyPI Version DOI SWH tests codecov

☕️ Chicago's second favorite bean

About

This library provides a configurable way to (1) collate data from multiple sources into a single denormalized dataframe, (2) tokenize the records into sequences of discrete integers, (3) winnow the records in preparation for downstream inference tasks, and (4) visualize both individual records and the tokenization process. It benefits from previous experience preparing data for training foundation models on tokenized electronic health records. 1 2 3 4 5

Installation

Install the latest release from PyPI:

pip install cocoa-tokenizer

This installs the cocoa command. To see it run end to end on public data, follow the MIMIC-IV demo, which installs cocoa with its demo extra (on Python 3.11 to 3.14):

pip install 'cocoa-tokenizer[demo]'

The extra brings clifpy, the CLIF consortium's Python package, which ships the demo as CLIF tables, and cocoa get-demo writes them out.

Usage

Each stage of the pipeline is a cocoa command, driven by a YAML config, and cocoa pipeline runs the first three in turn:

collate  →  tokenize  →  winnow
                      ↘  visualize

The documentation covers:

  • cocoa collate: pulls raw tables into a single denormalized, MEDS-like table of events, and splits the subjects into train, tuning, and held-out sets.
  • cocoa tokenize: turns each subject's events into a sequence of integer tokens, with a vocabulary and quantile bins learned on the training split.
  • cocoa winnow: cuts timelines into a past and a future at a threshold and flags the outcome tokens on each side, for evaluation.
  • cocoa visualize: draws a subject's tokenized timeline, or the learned vocabulary, as an interactive page.
  • cocoa combine-datasets: merges processed directories into one.
  • Configuration: the YAML config each stage reads, and how to change its keys on the command line.
  • The CLI reference, with every command's options, and the recipes, worked examples from a run on public data to tokenizer transfer.

Contributing

Contributions are welcome, from bug reports and typo fixes to new features. To get started, see the contributing guide.

Next steps

For training models and running inference on this data, check out cotorra.

Footnotes

  1. M. Burkhart, B. Ramadan, Z. Liao, K. Chhikara, J. Rojas, W. Parker, & B. Beaulieu-Jones, Foundation models for electronic health records: representation dynamics and transferability, arXiv:2504.10422 ↩

  2. M. Burkhart, B. Ramadan, L. Solo, W. Parker, & B. Beaulieu-Jones, Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models, Pacific Symposium on Biocomputing 31 (2026), 173–188 ↩

  3. L. Solo, M. McDermott, W. Parker, B. Ramadan, M. Burkhart, & B. Beaulieu-Jones, Efficient generative prediction for EHR foundation models: the SCOPE and REACH estimators, arXiv:2602.03730 ↩

  4. I. Lee, L. Solo, M. Burkhart, B. Ramadan, W. Parker, & B. Beaulieu-Jones, Representation before training: a fixed-budget benchmark for generative medical event models, arXiv:2604.16775, to appear in ML4H 2026 ↩

  5. M. Burkhart, L. Solo, I. Lee, S. Charles, Z. Liao, K. Chhikara, D. Therese, W.-T. Liao, C. Gao, W. Parker, & B. Beaulieu-Jones, Federated generative event models for tokenized electronic health records, arXiv:2608.02939 ↩

About

A configurable collator and tokenizer

Topics

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Releases

Used by

Contributors

Languages