-
-
Notifications
You must be signed in to change notification settings - Fork 51
docs: Add Migration docs #622
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 9 commits
ba0cc04
72a43cf
91d1225
654a491
f8827a3
0f1ed4c
1cbd900
8694a5c
0955248
faabe19
ee01684
01041d9
39c15fe
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,5 @@ | ||
| pub mod utils; | ||
|
|
||
| use std::fs; | ||
| use std::path::PathBuf; | ||
| use std::sync::{Mutex, MutexGuard, OnceLock}; | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| //! Test utilities | ||
|
|
||
| /// Formats a code snippet for an error message using the style of rustc | ||
| /// diagnostics | ||
| /// | ||
| /// # Example | ||
| /// | ||
| /// ```text | ||
| /// --> migrations.md:201:1 | ||
| /// | | ||
| /// 201 | Rollback dry run | ||
| /// 202 | | ||
| /// 203 | Target: | ||
| /// 204 | app: customers | ||
| /// 205 | migration: m_0001_initial | ||
| /// | | ||
| /// ``` | ||
| pub fn format_code_snippet( | ||
| literal: &str, | ||
| file_name: &str, | ||
| start_line: usize, | ||
| start_col: usize, | ||
| max_lines: usize, | ||
| ) -> String { | ||
| let lines: Vec<&str> = literal.lines().take(max_lines).collect(); | ||
|
|
||
| // Width of the largest line number, for gutter alignment. | ||
| let last_line_num = start_line + lines.len().saturating_sub(1); | ||
| let gutter_width = last_line_num.to_string().len(); | ||
|
|
||
| let mut out = String::new(); | ||
| out.push_str(&format!( | ||
| "{:width$}--> {}:{}:{}\n", | ||
| "", | ||
| file_name, | ||
| start_line, | ||
| start_col, | ||
| width = gutter_width + 1 | ||
| )); | ||
| out.push_str(&format!("{:width$} |\n", "", width = gutter_width)); | ||
|
|
||
| for (i, line) in lines.iter().enumerate() { | ||
| let line_num = start_line + i; | ||
| out.push_str(&format!( | ||
| "{line_num:width$} | {line}\n", | ||
| width = gutter_width | ||
| )); | ||
| } | ||
|
|
||
| out.push_str(&format!("{:width$} |", "", width = gutter_width)); | ||
|
|
||
| out | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,217 @@ | ||
| --- | ||
| title: Migrations | ||
| --- | ||
|
|
||
| Migrations are how Cot's ORM keeps your database schema in sync with your Model structs. Think of them as version control for your schema: every time you add a model, add a field, or remove something, you generate a migration file that records exactly what changed. Cot then applies those files, in order, to bring any database (a fresh one, or the one in production) up to date. | ||
|
|
||
| Migration files are plain Rust files. There's no separate DSL to learn and nothing gets serialized to JSON or YAML behind your back. If you can read Rust, you can read a migration. | ||
|
|
||
| ## Creating migrations | ||
|
|
||
| Once you've added or changed a struct annotated with [`#[model]`](attr@cot::db::model), run: | ||
|
|
||
| ```bash | ||
| cot migration make | ||
| ``` | ||
|
|
||
| Cot compares your current models against the models recorded in the most recent migrations, works out what changed, and writes a new migration file for you. If nothing changed, it will just tell you so and exit without creating anything. | ||
|
|
||
| A few things worth knowing about how this works: | ||
|
|
||
| * The **app name** for a migration is your crate name by default. You can override it with `--app-name` if you'd like, though in most single-crate projects you'll never need to. | ||
| * Migrations are generated per crate. If you have a Cargo workspace with multiple crates, each one gets its own `migrations` directory and its own independent history. There's currently no way for a migration in one crate to be generated against models living in another crate in the same workspace, that's on the roadmap but not there yet. | ||
|
ElijahAhianyo marked this conversation as resolved.
Outdated
|
||
| * By default, migrations are written to the `src/migrations/` directory inside your crate. You can point this elsewhere with `--output-dir` if needed. | ||
|
|
||
| If you want an empty migration to hand-write yourself (useful for data migrations, or anything that isn't a simple schema change), use: | ||
|
|
||
| ```bash | ||
| cot migration new <name> | ||
| ``` | ||
|
|
||
| This scaffolds a migration file with `forwards` and `backwards` stubs ready to fill in, rather than trying to auto-detect any model changes. | ||
|
|
||
| ## Running migrations | ||
|
|
||
| Migrations are applied automatically whenever your project boots up, so a normal `cargo run` is all it takes. On startup, Cot creates a small internal table called `cot__migrations` (if it doesn't already exist) to keep track of which migrations have already been applied, then walks through your migrations in dependency order and applies anything new. Migrations that are already recorded as applied are skipped, so it's always safe to boot the project again after pulling in new migrations. | ||
|
|
||
| ## Listing migrations | ||
|
|
||
| To see what migrations exist across your project, run: | ||
|
|
||
| ```bash | ||
| cot migration list | ||
| ``` | ||
|
|
||
| This prints every migration Cot can find, one per line, alongside the app (crate) it belongs to. In a workspace, this walks all member crates. | ||
|
|
||
| ## Anatomy of a migration file | ||
|
|
||
| Here's a trimmed down version of what `cot migration make` generates for a simple `Product` model: | ||
|
|
||
| ```rust,ignore | ||
| //! Create a new table for the Product model. | ||
| //! | ||
| //! Generated by cot CLI 0.6.0 on 2026-05-26 16:15:49+00:00 | ||
|
|
||
| #[derive(Debug, Copy, Clone)] | ||
| pub(super) struct Migration; | ||
|
|
||
| impl ::cot::db::migrations::Migration for Migration { | ||
| const APP_NAME: &'static str = "ecommerce"; | ||
| const MIGRATION_NAME: &'static str = "m_0001_initial"; | ||
| const DEPENDENCIES: &'static [::cot::db::migrations::MigrationDependency] = &[]; | ||
| const OPERATIONS: &'static [::cot::db::migrations::Operation] = &[ | ||
| ::cot::db::migrations::Operation::create_model() | ||
| .table_name(::cot::db::Identifier::new("ecommerce__product")) | ||
| .fields(&[ | ||
| ::cot::db::migrations::Field::new( | ||
| ::cot::db::Identifier::new("id"), | ||
| <cot::db::Auto<i64> as ::cot::db::DatabaseField>::TYPE, | ||
| ) | ||
| .auto() | ||
| .primary_key(), | ||
| ::cot::db::migrations::Field::new( | ||
| ::cot::db::Identifier::new("name"), | ||
| <cot::db::LimitedString<255> as ::cot::db::DatabaseField>::TYPE, | ||
| ), | ||
| ]) | ||
| .build(), | ||
| ]; | ||
| } | ||
|
|
||
| #[derive(::core::fmt::Debug)] | ||
| #[::cot::db::model(model_type = "migration")] | ||
| struct _Product { | ||
| #[model(primary_key)] | ||
| id: cot::db::Auto<i64>, | ||
| name: cot::db::LimitedString<255>, | ||
| } | ||
| ``` | ||
|
|
||
| Every migration is just a unit struct that implements the [`Migration`](trait@cot::db::migrations::Migration) trait, made up of four consts: | ||
|
|
||
| * **`APP_NAME`**: the app (crate) this migration belongs to. This, together with `MIGRATION_NAME`, is what Cot uses to identify a migration uniquely. | ||
| * **`MIGRATION_NAME`**: the migration's own name, matching the file name minus the extension (for example `m_0001_initial`). By convention, auto-generated names after the first one look like `m_0002_auto_20260527_004236`, a sequence number followed by a timestamp. Migrations created with `cot migration new my_name` instead get `m_0002_my_name`. | ||
| * **`DEPENDENCIES`**: a list of other migrations (including those in other apps) that must run before this one. See [Dependencies](#dependencies) below. | ||
| * **`OPERATIONS`**: the actual list of schema changes this migration performs, applied in order. See [Operations](#operations) below. | ||
|
|
||
| You'll also notice a second struct below the `Migration` impl, prefixed with an underscore (`_Product` above). This is a snapshot of the model as it looked at the time this migration was generated, marked with `model_type = "migration"`. Cot uses these snapshots to work out what changed the next time you run `cot migration make`, by diffing your current models against the latest snapshot on record. You shouldn't need to touch these by hand, they exist purely so future migrations can be generated correctly. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is not the only reason these structures exists. Note that with each migration being run, a given model can have completely different structure. For the custom migrations, this matters: if you use the wrong version of the model, you might end up doing operations that will straight up fail, or, what's even worse, will silently corrupt your data. In custom migrations, you should only use the snapshot (
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I've created #623 to improve this.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This part is 100% correct, but I'd emphasize here also that that's the Rust representation of what the database table will be after the migration. |
||
|
|
||
| ### The `migrations.rs` module | ||
|
|
||
| Alongside your migration files, Cot maintains a `migrations.rs` module that lists all of them: | ||
|
|
||
| ```rust,ignore | ||
| //! | ||
| //! List of migrations for the current app. | ||
| //! | ||
| //! Generated by cot CLI 0.6.0 on 2026-05-27 00:42:36+00:00 | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Unrelated nit, but I think we should add a note here in the generator that users should not modify this file as there is no point in that. |
||
|
|
||
| pub mod m_0001_initial; | ||
| pub mod m_0002_auto_20260527_004236; | ||
|
|
||
| /// The list of migrations for current app. | ||
| pub const MIGRATIONS: &[&::cot::db::migrations::SyncDynMigration] = &[ | ||
| &m_0001_initial::Migration, | ||
| &m_0002_auto_20260527_004236::Migration, | ||
| ]; | ||
| ``` | ||
|
|
||
| This file is regenerated in full every time you run `cot migration make` or `cot migration new`, so treat it as generated code. It's what your [`App::migrations`](trait@cot::project::App#method.migrations) implementation typically returns. | ||
|
|
||
| ## Operations | ||
|
|
||
| An [`Operation`](struct@cot::db::migrations::Operation) describes one schema change, and knows how to apply itself both [`forwards`](struct@cot::db::migrations::Operation#method.forwards) (when migrating) and [`backwards`](struct@cot::db::migrations::Operation#method.backwards) (when rolling back). You'll rarely construct these by hand since `cot migration make` does it for you, but it's useful to know the shape, especially if you're writing a custom migration. | ||
|
|
||
| For example, adding a field to an existing table: | ||
|
|
||
| ```rust | ||
| # use cot::db::migrations::{Operation, Field}; | ||
| # use cot::db::{DatabaseField, Identifier}; | ||
| const OPERATION: Operation = Operation::add_field() | ||
| .table_name(Identifier::new("customers__product")) | ||
| .field(Field::new(Identifier::new("sku"), <String as DatabaseField>::TYPE).unique()) | ||
| .build(); | ||
| ``` | ||
|
|
||
| Besides [`add_field`](struct@cot::db::migrations::Operation#method.add_field), Cot ships builders for creating a model, removing a field, and removing a model. See the [`Operation`](struct@cot::db::migrations::Operation) for the full list of supported operations. | ||
|
|
||
| ### Custom operations | ||
|
|
||
| Sometimes a schema change isn't expressible with the built-in operations above, or you need to run a data migration alongside a schema one. For that, there's [`Operation::custom`](struct@cot::db::migrations::Operation#method.custom): | ||
|
|
||
| ```rust | ||
| use cot::db::Result; | ||
| use cot::db::migrations::{MigrationContext, Operation, migration_op}; | ||
|
|
||
| #[migration_op] | ||
| async fn forwards(ctx: MigrationContext<'_>) -> Result<()> { | ||
| // run whatever queries you need against ctx.db | ||
| Ok(()) | ||
| } | ||
|
|
||
| #[migration_op] | ||
| async fn backwards(ctx: MigrationContext<'_>) -> Result<()> { | ||
| // undo it | ||
| Ok(()) | ||
| } | ||
|
|
||
| const OPERATION: Operation = Operation::custom(forwards).backwards(backwards).build(); | ||
| ``` | ||
|
|
||
| The [`#[migration_op]`](attr@cot::db::migrations::migration_op) macro just takes care of the boilerplate needed to match the function signature Cot expects. `backwards` is optional. If you skip it and someone later tries to roll this migration back, Cot will simply return an error saying the backwards migration isn't implemented, rather than silently doing nothing. | ||
|
|
||
| ## Dependencies | ||
|
|
||
| Migrations within the same app are always applied in the order their names sort in (which is why the auto-generated names are zero-padded and sequential). Across apps, or whenever ordering needs to be more explicit than that, you can add entries to `DEPENDENCIES`: | ||
|
|
||
| * **[`MigrationDependency::migration`](struct@cot::db::migrations::MigrationDependency#method.migration)**, for depending on a specific migration in a specific app being applied first. | ||
| * **[`MigrationDependency::model`](struct@cot::db::migrations::MigrationDependency#method.model)**, for depending on whichever migration created a given model, without needing to know or hardcode its exact name. | ||
|
|
||
| You'll see `cot migration make` add these automatically whenever a new foreign key points at a model that isn't being created in the very same migration. That's the most common reason you'd end up depending on another app: a `ForeignKey<Customer>` field pointing at a model whose migration lives elsewhere. | ||
|
|
||
| If Cot ever detects a cycle in your migration dependencies, it will refuse to run and tell you so, both when generating new migrations and when booting the project. | ||
|
|
||
| ## Rolling back migrations | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should wait with merging this until #583 is merged then |
||
|
|
||
| Rolling back migrations runs through your compiled project binary in the target dir, not the `cot` CLI: | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We'll need to update this to use the cot CLI once #587 gets merged |
||
|
|
||
| ```bash | ||
| ./path/to/binary migration rollback <MIGRATION_NAME> [--app <APP>] [--dry-run] | ||
| ``` | ||
|
|
||
| `<MIGRATION_NAME>` can be given a few different ways: | ||
|
|
||
| * The full migration name, e.g. `m_0002_add_sku` | ||
| * Just the number, e.g. `0002` | ||
| * The special value `zero`, meaning "roll back everything for this app" | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't like this
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The special case of |
||
|
|
||
| `--app` defaults to your crate name, the same as `APP_NAME` in your migrations. Rolling back to a specific migration is **exclusive**: that migration itself stays applied, and everything after it gets undone. | ||
|
|
||
| Rollbacks aren't limited to a single app either. If another app has a migration that depends on one of the migrations you're rolling back (say, through a foreign key), that dependent migration gets rolled back too, so you're never left with a dangling reference. Migrations that were never applied in the first place are simply skipped. | ||
|
|
||
| Because rolling back can have a wider blast radius than expected once cross-app dependencies are involved, it's worth checking first with the `--dry-run` flag which shows a preview of what migrations will be undone: | ||
|
|
||
| ```bash | ||
| ./target/debug/binary migration rollback <MIGRATION_NAME> --dry-run | ||
| ``` | ||
|
|
||
| This prints the exact rollback plan without touching your database at all, something like: | ||
|
|
||
| ```bash | ||
| Rollback dry run | ||
|
|
||
| Target: | ||
| app: customers | ||
| migration: m_0001_initial | ||
| mode: zero (all migrations in app rolled back) | ||
|
|
||
| Rollback plan: | ||
| 1. customers::m_0002_auto_20260527_004236 | ||
| 2. customers::m_0001_initial | ||
|
|
||
| Summary: | ||
| to roll back: 2 | ||
| skipped: 0 | ||
| database changes: none | ||
| ``` | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we define a custom error struct (and return
Result<(), ThisNewErrorStruct>here) instead of using bespoke logic in-place here? This would allow us to define the fancy printing in just one place in the code and then reuse if needed.