Migrations: how does a schema change reach every database?

A migration is Dart code in the server package that moves the schema one step — carrying the rows already there across it, when it must. The project's migrations live in lib/src/migrations/, one file each, registered in lib/src/migrations/migrations.dart. create writes package imports in that index, using the name from the nearest pubspec.yaml and the directory's path under lib/; directories outside lib/ use relative imports. The server applies them when it starts; bin/migrate.dart applies, rolls back, inspects and drafts them.

Migrations are code rather than a diff computed at deploy time because only a person knows whether a dropped column was really renamed, or what a new NOT NULL column should hold in existing rows. The tools write the draft and refuse to guess those answers.

Expand, then contract across releases

A release here is a deploy to production, including the next production deploy in a project without release branches. The previous release's code must run on the new schema: rolling back to its image and running old and new code side by side both depend on that.

Expand in the release that changes the code: add tables, nullable columns or columns with a default, indexes, or relax constraints that released readers and writers can still use. Relaxing NOT NULL to nullable is an expansion only while new code, including backfills, writes no NULL that released code reads as non-null. Dropping a default that released inserts rely on is a contraction.

Contract one release later, once no running code relies on the old shape: drop, rename, change a type, make existing rows NOT NULL, narrow a constraint, or remove a default that released inserts rely on. A rename of a released column or table takes four steps:

1. Release N: add the new column or table.
2. Release N: write both (or backfill).
3. Release N: switch reads to the new one.
4. Release N+1: drop the old one.

A type change uses the same sequence through a new column. A direct renameColumn or ALTER TABLE … RENAME TO is only for a table or column no released code reads, typically one added in the same unreleased branch. A drop and an add in one table are usually a rename; that does not make a direct rename safe for released readers.

DwDatabaseMigration

A migration adding an invoice table, as create would draft it:

import 'package:dartway_core_server/dartway_core_server.dart';

final class M20261001120000Invoices extends DwDatabaseMigration {
  const M20261001120000Invoices();

  @override
  String get id => '20261001_120000_invoices';

  @override
  String get checksum => '…'; // sealed by `create`, refreshed by `rehash`

  @override
  Future<void> up(DwMigrationContext m) async {
    await m.createTable(
      DwTableSchema(
        'invoice',
        columns: [
          DwColumnSchema.primaryKey(),
          DwColumnSchema(
            'account_id',
            'bigint',
            references: DwForeignKey('dw_account', onDelete: DwOnDelete.cascade),
          ),
          DwColumnSchema('amount', 'bigint'),
          DwColumnSchema('paid_at', 'timestamp with time zone', nullable: true),
        ],
        indexes: [
          DwIndexSchema('invoice_account_id_idx', ['account_id']),
        ],
      ),
    );
  }

  @override
  Future<void> down(DwMigrationContext m) async {
    await m.dropTable('invoice');
  }
}

Real ones: example/dartway_example_server/lib/src/migrations/ (the chat migration there was reviewed by hand — a column renamed rather than dropped and re-added). A direct rename is only appropriate when no released code reads that column; for a released column, use the two-release sequence.

Member	Meaning
id	YYYYMMDD_HHMMSS_name, unique in its namespace; the file is m<id>.dart
checksum	a hash of the file's source, written by create and refreshed by rehash. The ledger stores it; an applied migration whose checksum changed refuses the next run. It is a declared literal because a compiled server has no sources to hash. Whitespace does not count; the commas dart format adds and removes when it wraps do, so create writes // dart format off as the file's first line and the formatter leaves a migration alone
supersededChecksums	checksums of earlier texts the ledger accepts in place of checksum. Empty by default, and for one case only: a migration that could not apply on some databases is corrected, and the earlier text, wherever it did apply, left exactly what the correction leaves — those databases keep their row, the rest run the correction. A change to what an applied migration does is a new migration
dependsOn	DwMigrationRef(namespace, id)s that must be applied first; may name another namespace. Empty by default
transactional	true by default. false only for statements Postgres refuses inside a transaction (CREATE INDEX CONCURRENTLY); such a migration is recorded dirty before it starts, so a crash halfway blocks the next run until someone looks
up(m)	the change
down(m)	irreversible unless overridden (m.irreversible()); m.noop() is the explicit answer for a migration with nothing to undo

A migration never imports row classes. They change, and a migration must still run in six months against the schema of its own day. It describes tables with schema literals — DwTableSchema, DwColumnSchema, DwIndexSchema, DwForeignKey — and works on data with SQL.

DwMigrationContext offers createTable, dropTable, addColumn(table, column, {backfill}), dropColumn, renameColumn, alterColumnNullability(table, column, {nullable, backfill}), alterColumnDefault, alterColumnType(table, column, sqlType, {using}), addForeignKey, dropForeignKey, addUnique, dropUnique, createIndex, dropIndex, and sql / query for everything else. A backfill is an SQL expression computed for each existing row: the column is added nullable, filled and then made NOT NULL, in the migration's transaction. Adding a NOT NULL column with only backfill: is an expansion only if the previous code does not insert into that table; otherwise the column needs a default so its inserts still work.

DwCalendarDay maps to PostgreSQL date. Adopting it for a new field is optional and needs no framework migration. If a project chooses to convert an existing timestamptz field, the project owns that schema change and follows the two-release sequence through a new column. It must name the civil zone in its backfill conversion (or USING expression for a direct type change when no released code uses the old shape), for example (starts_at AT TIME ZONE 'America/Los_Angeles')::date; the selected zone decides which day each instant becomes. The framework does not rewrite timestamp columns or require existing projects to adopt the date type.

The ledger and the rules

Applied migrations are rows of dw_migrations: namespace, id, checksum, batch, order of application, state (applied or dirty) and time. The runner takes a session advisory lock first, so two processes starting at once apply each migration once; the second waits and finds it done.

Before applying anything, the runner compares the ledger with the code and refuses — DwMigrationRefused, nothing applied, every problem listed — when:

• a migration is applied but no longer registered and its id precedes a registered id in its namespace (missing: a gap in the chain);
• an applied migration's checksum differs from the code's and is not one of its supersededChecksums (changed: its source was edited);
• a migration is dirty (a non-transactional one started and never finished);
• a dependsOn names an unregistered migration, the dependencies form a cycle, or one id is registered twice.

An unknown applied migration whose id sorts after every registered id in its namespace is ahead: applied by a newer release than this code. A registered namespace with no migrations treats every applied row as newer. This rule covers dw, modules and app; unregistered namespaces are ignored. A previous release boots on the newer schema, logging one warning per namespace, and the declared-table and column check still runs. DwMigrationRun.ahead reports these refs without including them in the migrations run; status lists them as ahead with no refusal. Gaps, edited checksums and dirty rows (including newer ones) still refuse. Any ahead row blocks rollback by batch or id: use the newer release's code to roll it back. Additive changes can support an image rollback; a removed or renamed table or column still needs a release that has stopped using it before the schema contracts.

Otherwise the pending migrations run as one batch, ordered by dependsOn first, then by namespace — the framework's dw, then the modules' in the order they are given, then the project's — then by id. Each transactional migration runs in its own transaction together with its ledger row. A migration that throws stops the run with DwMigrationFailed; the ones before it stay applied. Every refusal and failure exits non-zero, in the CLI and in the server.

So a project migration follows every framework and module migration, whatever their ids: a project created from an older template has an initial migration older than framework migrations it relies on, and it still runs after them. Within the project, declare dependsOn when a migration must follow one with a later id.

A migration changes the schema

A migration runs once per database. That is right for the schema and wrong for content: rows put there reach only the databases that had not applied it yet, and the day they change, an applied migration cannot be edited — its checksum stops every server that ran it — while a second one that rewrites them has to guess which rows somebody has since corrected by hand.

So a migration writes rows for one reason only: to carry the rows already there across its own schema change — a renamed enum value, a column split in two, rows a new constraint forbids. It says so by passing that statement to m.backfill(sql):

@override
Future<void> up(DwMigrationContext m) async {
  await m.addColumn('project_issue', DwColumnSchema('stage', 'text'), backfill: "'development'");
  await m.backfill("UPDATE project_issue SET stage = 'review' WHERE stage = 'validation'");
}

This addition with only backfill: assumes the previous code does not insert into project_issue; otherwise stage needs a default, as described above.

backfill runs like sql; what it adds is the statement's purpose, and dart run dartway_cli:dartway check holds it: an INSERT, UPDATE or DELETE anywhere else in a project migration fails the build (migrationChangesData). The check reads the migration's string literals — adjacent literals as one string — and passes function, procedure and trigger bodies, which are definitions rather than statements; a statement held in a variable is judged where it is written, so pass it to backfill directly.

Everything else has a place of its own (app server): rows the code declares are a DwSeedRows step that every start converges on, and a value per app with defaults is a settings object (database).

Write data in SQL, not through row classes. A migration is read against the schema of its own day, forever; written through today's row classes it silently changes meaning the next time a field is renamed, while its checksum says nothing happened. That is why DwMigrationContext offers sql, query and backfill and no typed tables.

Migrations written before a project adopted the rule stay as they are — an applied migration is never edited. The project names its latest migration once, in deploy/config.yaml, and only the migrations after it are judged:

migrations:
  dataChecksAfter: 20260918_062959_member_tombstone

A project that never set it — every new one — has all its migrations judged. A value that is not a migration id, or names none in lib/src/migrations/, is a finding of its own.

Namespaces

• dw — the framework's own tables: accounts, identities, session keys, code tickets, command outcomes, jobs, stored files. Listed as DwAppServer.frameworkMigrations. Append-only: a change is a new migration. One migration was corrected in place, because it could not apply where dw_stored_file held rows: 20260914_180000_dw_stored_file_bucket stops on files uploaded before a file recorded its bucket and names the two statements that record it (ALTER TABLE dw_stored_file ADD COLUMN bucket text; UPDATE dw_stored_file SET bucket = '…', the bucket DW_STORAGE_BUCKET named then); a database that applied its first text is accepted by supersededChecksums. 20261009_000002_dw_stored_file_bucket_from_config dropped the column again — a file's bucket is the one the configuration names for its visibility (D-138) — and stops while one visibility's rows name more than one bucket, naming the visibility, the buckets and the statement to run once those objects are copied into one bucket under the same key (UPDATE dw_stored_file SET bucket = '…' WHERE visibility = '…').
• app — the project's.

Framework migrations ship inside a package version and follow the same expand/contract rule: a rollback runs the previous framework version on the new dw_* schema.

Project tables may reference framework tables — a profile references dw_account, an attachment dw_stored_file — but a project never writes migrations for, or queries, the dw_* tables (auth and identity).

bin/migrate.dart

The skeleton's (template/dartway_starter_server/bin/migrate.dart):

Future<void> main(List<String> args) async {
  exitCode = await DwMigrationCli(
    schema: dartwayStarterSchema,
    migrations: appMigrations,
    directory: 'lib/src/migrations',
    modules: {'dw': DwAppServer.frameworkMigrations},
  ).run(args);
}

Without database: the CLI reads DW_DATABASE_* only for the commands that use a database, so rehash runs with none set. Pass database: to point it somewhere else.

schema is the generated DwDatabaseSchema of the row classes, modules the migrations of other namespaces run before the project's replay (their tables are not part of schema), and namespace defaults to app. There is no dartway migrate: the command line is the project's own program, so it compiles against the project's row classes and migrations.

dart run bin/migrate.dart <command> against the database in DW_DATABASE_*:

Command	What it does	Exit
apply	applies pending migrations as one batch	0; 2 refused; 1 failed
rollback	rolls back the last batch	as apply
rollback --batch N	rolls back batch N
rollback --id X	rolls back one migration; X is an id of the project or namespace/id
status	lists every migration: applied, pending, ahead, dirty, changed, missing	2 when any is dirty, changed or missing
create <name>	writes a draft for the difference between the migrations and the row classes; name is snake_case	0
check	verifies files, schema parity and up/down/up	3 when it finds a difference
rehash [id …]	re-seals the checksums of edited migrations (all, or the ids given)	0

Wrong arguments exit 64. A rollback refuses when an applied migration outside the rollback depends on one inside it. When every migration rolled back is transactional, the whole rollback is one transaction: an irreversible migration in the middle leaves the database as it was.

Drafts: create <name>

1. Creates a throwaway database next to the one in DW_DATABASE_* (the user needs the right to create databases) and replays every migration into it.
2. Creates the schema the row classes declare in a separate Postgres schema of the same database and reads both back, so types and defaults compare as Postgres spells them rather than as they were typed.
3. Diffs them and writes lib/src/migrations/m<id>.dart — formatted, sealed with its checksum, with up and the inverse down — and rewrites migrations.dart to register every file in id order.
4. Drops the throwaway database.

Run dart run dartway_cli:dartway generate first: create reads the generated schema, not the row class sources.

No schema difference writes an empty migration, for data work. A change the diff cannot decide is written as a call to decisionRequired('…'), with a comment naming the options:

• dropping a table or a column: only in a release after the code stopped reading the old shape; in the same release, expand instead. The options include renameColumn only for a column no released code reads;
• adding a NOT NULL column without a default: existing rows need a backfill, but if the previous code inserts into that table the column needs a default;
• changing a column's type: only in a release after the code stopped reading the old shape; in the same release, expand through a new column. A direct conversion needs a using expression;
• making a column NOT NULL: only in a release after the code stopped reading the old shape; in the same release, expand instead. Existing nulls need a backfill.

decisionRequired is a function that does not exist, so the draft does not compile until every decision is made. A runtime throw would surface only when the migration runs; a compile error surfaces in the editor, in dart analyze and in every build, and cannot be deployed by accident.

Once written, the draft is an ordinary migration and the author's. Editing it before it is applied anywhere is expected; then run rehash <id>, or check reports the file as changed since sealing. Editing it after it is applied somewhere is what the checksum refuses — reformatting included: an applied migration that check reports as changed is restored as it was sealed, not rehashed. Migrations created before // dart format off was written have no such line, and adding it is an edit too; those are kept out of dart format by hand.

A pending migration edited and not yet rehashed is also refused where it would be applied, as long as its source is on disk: migrate apply, and a server run from its sources with DwAppServer(migrationsDirectory: 'lib/src/migrations'). Applying it would run the new text and record the old checksum, and the rehash after that would make every later start refuse the migration as edited after it was applied. A compiled server has no sources beside it and does not check.

check

check is what CI runs, and what dart run dartway_cli:dartway check runs as migrationsDrift:

1. Files — every migration's declared checksum matches its source; every file is registered in migrations.dart, and every registered migration has a file.
2. Parity — the migrations, replayed into a throwaway database, produce exactly the schema the row classes declare. Differences are listed as the changes still missing. What migrations created that row classes cannot declare — a partial index, a check constraint — is left out of the comparison and printed as a note, so nothing proposes dropping it.
3. Up/down/up — every project migration is rolled back newest first, then applied again one by one; each step up must reproduce the schema its rollback started from. It stops, with a note, at an irreversible migration.

It needs DW_DATABASE_* pointing at a Postgres where throwaway databases can be created — the development one will do.

dart run dartway_cli:dartway check runs dart run bin/migrate.dart check in the server package when DW_DATABASE_HOST is set. Findings are migrationsDrift errors: a schema the migrations do not produce is a server that refuses to start in the next environment. Without a database it prints that the check did not run — never that the migrations are fine. See the conventions checker.

What the server does on start

DwAppServer.start() runs the same runner over {'dw': frameworkMigrations, 'app': migrations} after opening the database. A refusal or a failure throws, and the process exits non-zero: a server never serves over a gap, edit or dirty migration. Ahead rows produce warnings and boot proceeds. Then, when schema is given, it checks that every declared table and column exists (app server).

So a deploy needs no separate migration step. bin/migrate.dart is for development (create, check, rollback) and for inspecting a database (status).

Related

• Database — row classes and what dart run dartway_cli:dartway generate writes.
• Migration notes — the edits a project owes when the framework changes; not database migrations.