How is a DartWay project deployed?

With four subcommands and one file, onto one Linux server running Docker. There are no deploy scripts in a project and no reason to write any: provisioning, rendering, the secret store, restarts and verification belong to the framework.

cd <project>_flutter
# deploy/config.yaml already describes "local"; add a deployment beside it

dart run dartway_cli:dartway deploy check --env staging --local            # the working copy only
dart run dartway_cli:dartway deploy setup --env staging                    # provision the server, render the stack
dartway secret set SMS_API_TOKEN --env staging        # what cannot be generated
dart run dartway_cli:dartway deploy check --env staging                    # DNS, the server, the deployed hosts
dart run dartway_cli:dartway deploy run   --env staging                    # update, build, start, verify

Every subcommand takes --env <environment> and refuses to guess without it, naming the environments deploy/config.yaml declares. Every one that connects takes --as <login> (default: ssh_user from the config) and --identity <key file> (default: your SSH agent). The code is packages/dartway_cli/lib/src/commands/deploy_*.dart, secret_commands.dart and packages/dartway_cli/lib/src/deploy/.

Kamal and GHCR

dart run dartway_cli:dartway deploy kamal generates the Studio deployment contract from deploy/config.yaml and the Flutter package's resolved CLI lock. Configure staging (destination stage) and/or production (destination prod); local is never a destination. Resolve the packages first.

cd <project>_flutter
dart run dartway_cli:dartway deploy kamal
dart run dartway_cli:dartway deploy kamal --check

Commit deploy/kamal/, deploy/studio.yml and .github/workflows/images.yml. Generated files carry their source instructions. --check writes nothing and names every stale file. Server route changes do not affect the generated web configuration. The API service is the root service on its own host only. On the app host, the web container's nginx proxies only /dw/ to DW_API_ORIGIN; it serves every other app path itself. Project routes are available on the API host. Proxy buffering is off for API and storage.

Kamal generation refuses a deployed site: because this topology does not support it yet; use deploy run for those projects. Kamal 2.12 uses kamal-proxy v0.10.2, with 30-second deployment and drain timeouts and five retained containers. The native proxy is configured with net.ipv4.tcp_congestion_control=bbr; provision BBR on the Linux host before booting it.

The generated Images workflow runs after the project's CI workflow completes successfully. Image publication follows the branch: values of the configured deployments: a CI run on one of those branches publishes when it was started by a push or by hand (workflow_dispatch, write access only); enable one of the two there. The template enables master and release by default. workflow_run loads workflows from the default branch, so the initial installation must be merged there before this trigger can run. PR CI successes build without pushing or writing shared caches. Fork PRs skip this privileged workflow. The template ships no Images workflow or generated deployment files; configure targets, run deploy kamal, then commit its output. Projects without a generated Studio manifest skip builds.

It respects DW_CI_RUNS_ON, builds linux/amd64, uses a GHA layer cache and pushes <commit sha> tags to GHCR with GITHUB_TOKEN on those two branches. Existing manifests are skipped; registry failures fail the job. PR runs build without pushing. The OCI source label links the packages to their repository.

Bundled storage is a service built from rustfs/rustfs:1.0.0 with a TLS host; external storage has no service and receives its environment through secrets. Bundled Postgres is an API accessory on postgres:17-alpine, with no host port. Project names start with a lower case letter or digit and contain only lower case letters, digits, _ and -. Invalid repository-derived names are refused with their value and the project: key to set explicitly. Kamal and Images use the same project identity without lowercasing it. Configs list secret names only, including required project secrets; Studio supplies their values, SSH credentials and KAMAL_REGISTRY_PASSWORD. ssh_user comes from the target.

Native checks and ingress policy

deploy check selects its engine from deploy/studio.yml: a native manifest selects Kamal; projects without one retain Compose. --engine kamal or --engine compose selects it explicitly. deploy run remains the Compose operator and always checks Compose. Generating a manifest does not deploy it.

dart run dartway_cli:dartway deploy check --env staging --local
dart run dartway_cli:dartway deploy check --env staging --version <full-commit-sha>

Without --version, native operator checks use the checkout's HEAD. A branch name or abbreviated SHA is refused. Native checks verify generated files, the running service/role/destination and selected image SHA, health, actual TLS routes and active targets, BBR in the proxy namespace, per-file read-only secret mounts, and the native Postgres/storage mounts. A healthy replacement container with the old route still active fails. An overlapping host claim, rollout or read target fails. Docker inspection projects only identities, health status, mount metadata and network addresses; it never reads container environments or health logs. These checks do not prove stored data: prove a backup restore before changing engines.

Declare the actual kamal network subnets under trusted_proxy_networks for each destination, then wire DwServerEnvironment.trustedProxyNetworks, httpBehindProxy and deploymentVersion into DwServerSettings as described in server configuration. Native checks require this explicit trust and compare it with the actual Docker network. The Docker network must exist before that check can pass. The generated API proxy sets forward_headers: true to preserve the chain from the app host; kamal-proxy still appends its actual TCP peer. Web ingress keeps Kamal's TLS default, replacing incoming forwarding headers. Do not set client_ip_header to an untrusted public header.

Generation refuses deploy/nginx.d/*.conf: native Kamal does not run the Compose front nginx and cannot silently preserve its policies. Move per-call peer quotas to DwServerSettings.callRateLimits; review all other snippets before removing them. Compose remains available for a deployment that needs those policies.

File secrets

Declare file names under the project's or destination's requires.files. The API destination mounts each file from /home/<deploy_user>/.config/<project>/<name> to /run/secrets/<name>, read-only. Web and storage receive no such mounts; the secret store's secrets.env is never mounted. Files are not copied into images or converted to environment variables.

Deliver them before the first boot, and again when rotating a credential:

dartway secret put-file /private/path/fcm.json --name fcm.json --env staging

This uses the existing server-side file store. An upload stays in a private temporary directory and atomically replaces the stored file only when complete. A running container keeps its previous mounted file; its replacement receives the updated file. Image rollback uses the currently delivered secret, not a historical credential snapshot. Secret rotation and image rollback are separate operations.

The generator writes an executable deploy/kamal/hooks/api/pre-app-boot and sets the API's hooks_path. Commit the hook with its executable bit. Kamal runs it before each boot group, including kamal app boot and rollback. It uses the installed Kamal gem and the API's resolved SSH settings, including keys, agent and proxy. It checks each selected host without reading secret contents: a real directory with owner-only permissions (700; an inherited setgid bit is allowed), nonempty regular files with mode 400 or 600, no symlinks, and matching file/directory ownership. A failure stops the boot. Do not use --skip_hooks for deployments requiring files. deploy kamal --check also reports a missing executable bit on Unix.

Delivery to a new host remains a prerequisite: generating config does not upload files, and the generated Images workflow handles no secret contents. Studio's deploy runner must preserve the generated hooks and deliver files before invoking Kamal. No Studio file-upload interface is supplied by this CLI.

The Studio check image reads a checkout mounted read-only at /project, with --project /project --env stage --version <full-commit-sha> (or prod). The generated manifest supplies {version} as an argument placeholder: the runner must substitute the selected deployment SHA, not the framework image tag. This image is an outside-only checker and requires no SSH, Docker socket or Git. Both API /health and app /dw/health must return the selected SHA in ETag; a healthy old deployment fails. It verifies TLS by making HTTPS requests, health on both hosts, the live socket hello, web caching and bundled storage visibility and CORS preflights on both buckets, and exits nonzero on a failure. With bundled storage, the API's DW_STORAGE_PROVISION=true makes DwFileStorageSetup.provision create both buckets, set their policies, configure GET/PUT CORS from the entry point's explicit allowedOrigins: env.server.allowedOrigins, and write the visibility probe into each before serving. No origins leaves CORS untouched. A storage answering 501 NotImplemented warns with its endpoint and bucket and continues; the outside CORS probe still reports the actual state. Other CORS errors stop provisioning. This works with DW_STORAGE_VERIFY_BUCKETS=false; Kamal has no storage-init service.

The check image tag follows the resolved dartway_cli: Git uses the lock's resolved ref SHA, path uses the checkout's HEAD, and pub.dev uses the lock's exact version. A path dependency on a vendored dartway_cli inside the project's own git work tree is refused; pin it by git ref so the tag identifies a framework commit. A Git/path SHA tag becomes available after that framework commit reaches master and its image build passes; uncommitted checkout changes do not get an image of their own. Framework CI builds SHA tags on master. A push to stable copies that image to the published CLI version, skipping prereleases and existing version tags with a notice. A version tag belongs to the commit at which that CLI version was published. When pub.dev has no such version, promotion ends successfully with a notice and pushes nothing. After a later publish, run Check image version on stable by hand. The PR runs this promotion in dry-run mode. Hosted projects get this capability with the first CLI release after #530.

Studio owns server preparation and deploy execution. A live project's move and its existing Postgres/RustFS volumes need a separate project migration. The Compose deploy run workflow below remains available for projects awaiting that move. After the first framework image push, the owner must make the dartway-check GHCR package public in its package settings.

One file describes an environment

deploy/config.yaml holds one environment per top-level key, plus three keys that belong to the project rather than to one of its machines: requires (below); local, which is what a developer's own machine starts a server with and is not a deployment at all — see Secrets; and migrations, whose dataChecksAfter names the last migration dart run dartway_cli:dartway check leaves out of migrationChangesData (Migrations). The server has no configuration file of its own — it is configured by its environment, which the deploy derives from this file (database coordinates, storage, port) and from the secret store on the server (passwords, keys). Nothing writes config.yaml; the deploy only reads it.

Key	Required	Meaning
host	yes	The server's address
ssh_user	yes	Login for provisioning — root, or a user with passwordless sudo
deploy_user	no	The unprivileged user that owns the checkout and runs the stack; defaults to dw_admin
os	yes	The server's operating system (ubuntu)
repo, branch	yes	The repository the server checks out, and the branch it deploys
project	no	The deployment's name on the server — checkout, secret store, Compose project, data volumes; absent means the last segment of repo. Lower case letters, digits, _ and -, as Compose accepts
ssl_email	yes	Where Let's Encrypt writes about expiring certificates
api_domain	yes	The server, for mobile apps and webhooks
app_domain	yes	The Flutter web app
site	no	domain and source: a directory of the repository served as static files, or none for a site that lives elsewhere
database	no	bundled or external; absent means bundled — a Postgres container in the stack
storage	no	bundled or external; absent means no file storage
storage_domain	with bundled	The public host of the bundled storage; refused without storage: bundled
min_free_disk	no	Minimum free space on Docker's data root before building; defaults to 10GB. Positive sizes in MB or GB (1024-based), for example 512MB or 10GB
build_cache_keep	no	Build cache retained after a successful deploy; defaults to 10GB. Positive sizes in MB or GB, for example 512MB
registry_mirror	no	Pull the official Docker Hub images of the stack (postgres, nginx) through a mirror such as mirror.gcr.io; the bundled storage and certbot come from where they live
firewall_ports	no	TCP ports to open beyond SSH, 80 and 443
requires.secrets	no	Secrets nobody can generate, as environment variable names
requires.files	no	Secret documents, by file name, mounted into the server

requires is also read at the top level of the file, where it states what the project needs wherever it runs; an environment's own requires adds to it. Declaring it once is the point: the same list repeated per environment drifts in the one direction nobody notices, and the environment that was forgotten is the one whose deploy stops.

The file is validated as it is read, every problem at once, and an unknown key is one of them: a key the deploy does not read is a setting somebody believes is in force. Every domain must be a host name, and each role needs a host of its own — Nginx routes by server_name, so two roles on one host means one of them is never reached.

Names are derived, not configured. The project name is project when it is set, and otherwise the last segment of repo without .git; the checkout is /home/<deploy_user>/<project>. With database: bundled, the database and its role are named after the server package without _server; with database: external they are a managed provider's own, delivered as DW_DATABASE_NAME/DW_DATABASE_USER — there is nothing to derive them from. The storage buckets are named after the same prefix with dashes.

A cloud image can already carry a system group named like deploy_user — DigitalOcean's Ubuntu 24.04 image ships an empty admin group, for instance (#328). Setup reuses that group when the user does not exist yet, unless the group is one a sudoers rule already grants privileges to (%admin on Ubuntu's stock sudoers; %sudo on both Debian's and Ubuntu's): joining one of those would hand the "unprivileged" deploy user a path to root, so setup refuses instead and asks for a deploy_user that does not collide with it. Setting deploy_user: admin explicitly still hits this refusal on a host whose stock admin group exists — that guard is not what fixes #328. What does is the default: dw_admin is fixed rather than derived from the project, because no stock image or package ships a group under the dw_ prefix, so an operator who does not set deploy_user never meets this collision at all.

Moving the repository

The project name is the deployment's whole identity on the server: the checkout /home/<deploy_user>/<project>, the secret store ~/.config/<project>, the Compose project and with it every data volume (<project>_postgres_data). Derived from repo, it changes when the repository moves to another owner or is renamed — and the same server then holds a stack under the old name that the new one does not see. Pin the old name in each environment of deploy/config.yaml when the repository moves:

stage:
  repo: [email protected]:new-owner/new-name.git
  project: old-name   # the name the stack on the server already has

run, setup and check hold this with the stack-identity check: they list the stacks on the server — data volumes (<name>_postgres_data, <name>_storage_data) and secret stores (~/.config/<name>/secrets.env), by name only — and refuse when there are stacks and none is this project's, naming the ones found and the project: line that would match. A server with no stack at all passes. A second project on the same host is set up once with deploy setup --new-stack; after that its own secret store names it.

Three hosts, one server process

app.example.com   →  the web image;  only /dw/ (the live socket included) → the server
api.example.com   →  everything → the server
example.com       →  the site directory, static            (optional)
files.example.com →  the bundled storage                    (with storage: bundled)

The app calls its own origin. On the app host the front Nginx serves the web image and sends /dw/ (including /dw/live and /dw/health) to the server, so a browser makes same-origin calls: no CORS, no preflight, and the server answers none. Web release builds use Uri.base.origin. The api host proxies everything to the server — calls, the live socket, and the project's own doors (DwHttpRoute) such as a webhook. The server serves no static files. Local development has the same shape through dart run dartway_cli:dartway dev (The CLI).

Storage has a host of its own because a presigned URL signs its host. Browsers upload to the bundled storage directly (see Uploads); a storage hidden behind the app's origin would be a different host from the one the URL was signed for. Inside the stack's network that host is an alias of the proxy, so the server reaches storage by the very URL a browser uses — no second address to configure.

The rendered Nginx:

• port 80 answers only the ACME challenge and redirects everything else to HTTPS;
• one certificate covers every served host, named after api_domain;
• Host is passed through as $http_host, port included — the live socket admits a browser whose Origin is the host the upgrade was sent to;
• /dw/live gets proxy_read_timeout 1h and no buffering; the server pings every 20 seconds by default, so the timeout only ends a dead connection;
• request bodies on the app and api hosts are limited to 16m — above the server's own default of 1 MiB, so the server's refusal (which a client reads) comes first rather than an Nginx HTML page; the storage host has no limit and streams the body, since the presigned URL bounds the size;
• deploy/nginx.d/{http,api,app}/*.conf are included — at the http level, in the app host, and inside the api host's location /.

The stack

setup renders docker-compose.yml into the checkout. The services:

Service	What it is
postgres	With database: bundled: postgres:17-alpine, on the volume postgres_data, with a pg_isready healthcheck. Absent with database: external — no service, no volume, no depends_on on it
server	Built from <project>_server/Dockerfile with the project root as context. Secrets through env_file: .env; PORT=8080 always, and with database: bundled (and, with the bundled storage, DW_STORAGE_*) DW_DATABASE_* in the compose file too — with database: external none of DW_DATABASE_* is rendered at all, all five (HOST, PORT, NAME, USER, PASSWORD) come from .env like any other secret. Exposed to the network only. stop_grace_period: 45s; healthcheck GET /health with a 600-second start period, because migrations run before the port opens
web	Built from <project>_flutter/Dockerfile; deploy run passes DW_BACKEND_URL from app_domain as a build argument. The template's web release calls Uri.base.origin
storage, storage-init	With storage: bundled: RustFS, and a one-shot job (a generic S3 client) that creates and configures both buckets on every deploy
nginx	nginx:1.30.5-alpine (pinned, raised with the framework) on 80 and 443
certbot	Renews the certificate every 12 hours

Data-bearing images are pinned — Postgres to its major, the storage to an exact release — so a deploy on a later day cannot move a data directory under a binary that refuses it. The stateless proxy and certbot follow upstream. Values in the compose file that the deploy derives win over the same names in .env, which is why the secret store refuses those names — and why it accepts DW_DATABASE_* only with database: external, where the compose file no longer sets them.

deploy/compose.override.yml is where a project adds what a standard deployment does not have. It is never copied: every Compose call the CLI issues names it from the checkout (docker compose -f docker-compose.yml -f deploy/compose.override.yml …), so git reset --hard on a deploy refreshes it. For the command a person types by hand on the server, the deploy writes docker-compose.override.yml as a two-line include: of the committed override, marked dartway-bridge — otherwise a bare docker compose up -d applies a strictly smaller stack and exits 0. A file under that name without the marker is moved to docker-compose.override.yml.retired; with no override in the checkout and a foreign file in its place, the deploy refuses, because that file may be the only place its services are declared.

Database: bundled or someone else's

database: bundled (the default) is the postgres service above — a container on the stack's own volume, reached over the private network, no TLS. database: external points at a managed Postgres instead: DigitalOcean Managed Postgres, RDS, Cloud SQL, or any Postgres somebody else runs. Nothing about it is rendered — no service, no volume, no depends_on — and the server reaches it exactly as it would reach any database named by its own environment. Deliver with dartway secret set --env <environment>:

Secret	Required	Meaning
DW_DATABASE_HOST	yes	The provider's host
DW_DATABASE_PORT	yes	Its port — a managed provider rarely uses 5432
DW_DATABASE_NAME	yes	The database name it assigned
DW_DATABASE_USER	yes	The role it assigned
DW_DATABASE_PASSWORD	yes	Its password — never generated here; it is the provider's credential
DW_DATABASE_SSL	no, default true	false only for a database that genuinely speaks no TLS
DW_DATABASE_MAX_CONNECTIONS	no, default 10	The pool ceiling — managed plans often cap total connections in the tens, so raising this without checking the plan's limit starves every other client of the same database
DW_DATABASE_CA_FILE	no	The provider's CA certificate verifies the server's own — verify-full instead of require (dartway/dartway#342). Set to where the file is mounted, e.g. /run/secrets/db-ca.pem; the name after the last / must be declared under requires.files (below)

deploy check gains database-reachable (below): on the deployment host over SSH, because a managed provider's firewall usually trusts that host's address and not the maintainer's — a throwaway, pinned Postgres client (never built or run otherwise with database: external) attempts a real connection with the same sslmode the server itself will use (require, verify-full with a CA file, or disable when DW_DATABASE_SSL is explicitly false), and reports why when it fails (a malformed stored value, DNS, refused, a timeout, authentication, the server not offering TLS, or its certificate not verifying against the configured CA). A bare TCP probe would pass while the password or the certificate is still wrong. DW_DATABASE_PORT, _SSL, _MAX_CONNECTIONS and _CA_FILE are validated exactly as the server parses them — a positive integer, true or false, and (for the CA file) a name declared under requires.files and actually delivered there — before anything connects, so a value that would crash the server at startup is a check failure naming it, not a pass. DW_DATABASE_CA_FILE set together with DW_DATABASE_SSL=false is refused as a contradiction, by the server itself (DwDatabaseConfig.fromEnvironment) and by this check.

Delivering the CA file: declare its name under requires.files (e.g. db-ca.pem), upload it with dart run dartway_cli:dartway deploy secret put-file <path/to/ca.pem> --name db-ca.pem --env <environment> — it is then mounted read-only at /run/secrets/db-ca.pem exactly like any other declared file — and set DW_DATABASE_CA_FILE to that same mounted path. DigitalOcean Managed Postgres offers the cluster's CA certificate as a download next to the connection details in its control panel, or on the command line: doctl databases get-ca <cluster-id> -o json | jq -r .certificate | base64 --decode > db-ca.pem (confirmed against doctl's own --help; there is no doctl databases connection --format CA). RDS publishes a combined regional bundle (rds-ca-*-bundle.pem) on its own documentation pages. Cloud SQL's instance page has a "Server CA certificate" download under its connections tab — unconfirmed here whether its default per-instance certificate carries the connecting address in subjectAltName (see the warning below); check before relying on verify-full against it.

The server certificate must carry a subjectAltName for the address DW_DATABASE_HOST names — dart:io (BoringSSL) has no fallback to the deprecated Subject CN field the way libpq/OpenSSL still does, and separately requires the leaf to declare extendedKeyUsage: serverAuth; a certificate with neither is refused outright, chain and all, with an error that does not name either omission (see the database page). Every provider named above issues certificates with a proper SAN, and this is exactly what a certificate authority is expected to include — but a self-hosted or lesser-known provider's own CA is worth checking (openssl x509 -in cert.pem -noout -text | grep -A1 'Subject Alternative Name') before depending on verify-full against it. database-reachable connects through libpq, which is more permissive here, so a config it passes can still be one the server itself refuses at startup for exactly this reason — the check does not (and, without openssl in the pinned Postgres client image it runs, cannot cheaply) verify the SAN itself.

An existing config with no database key keeps deploying bundled, exactly as before.

DigitalOcean Managed Postgres

• Connect on the direct port, not the connection pooler. DigitalOcean publishes two ports: the direct one (usually 25060) and PgBouncer in transaction mode (usually 25061). Use the direct port — DW_DATABASE_PORT: 25060. The framework needs a session, not a transaction: the server holds a session-scoped LISTEN for live updates (DwPostgresDatabase), takes a session pg_advisory_lock while migrating (DwMigrationRunner) so two servers never migrate at once, and caches prepared statements per connection. Behind a transaction-mode pooler LISTEN silently delivers nothing (no error, no update — a class of bug that surfaces as "the app doesn't refresh" weeks later), the migration lock does not hold across the pool's own transactions, and prepared statements are meaningless once a "connection" is a new backend on every transaction.
• The database name is usually defaultdb (DW_DATABASE_NAME), the default user doadmin or a role created for this project (DW_DATABASE_USER).
• Add the droplet's address as a trusted source in the database's firewall — that is what database-reachable needs already admitted before it can report anything but a timeout.

Sizing connections

One server process holds DW_DATABASE_MAX_CONNECTIONS (default 10) plus one more for the session LISTEN above — so a plan's usable connection count has to clear MAX_CONNECTIONS + 1, not just MAX_CONNECTIONS. DigitalOcean's smallest Managed Postgres plan allows on the order of 22 usable connections in total (the rest are the provider's own); size DW_DATABASE_MAX_CONNECTIONS to what the plan actually grants this one server, leaving room for anything else that connects (psql by hand, a reporting tool). A deploy never runs two servers at once: the old one stops before the migration run and the new server start, so no headroom is needed for an overlap.

Switching an existing environment from bundled to external

Moves no data. The server that starts against the newly declared external database starts on an empty one — its migrations create the schema, not the rows a bundled Postgres was holding. Move the data yourself first (a pg_dump/pg_restore between the two, out of scope here) before pointing deploy/config.yaml at the external one. A deploy's own rollback — starting the previous server image again after a failure — uses whatever .env the store renders at the time, which after the switch is the external database's, not the bundled one's: there is no bundled Postgres left running to fall back to once the config has moved.

Storage: bundled or someone else's

Two buckets, whatever the option (D-038b): a public one whose objects anyone reads and whose listing nobody gets, and a private one that reads nothing unsigned. A rule's visibility picks the bucket.

storage: bundled runs RustFS in the stack. storage-init (a generic, pinned S3 client) creates <prefix>-public and <prefix>-private (keeping existing buckets and their objects), sets the public bucket's policy to anonymous s3:GetObject only — never a canned ACL, which would also grant listing — clears any policy on the private one, sets bucket CORS admitting the app origin on both (what a browser's presigned PUT needs), and writes a probe object into both. The server receives DW_STORAGE_ENDPOINT (the storage host), both bucket names, DW_STORAGE_PUBLIC_BASE_URL=https://<storage_domain>/<prefix>-public, path-style addressing and DW_STORAGE_VERIFY_BUCKETS=false: the server reaches the storage only through the proxy, and the proxy starts after the server, so the bucket check a server normally runs at startup cannot run there. The outside probe runs the same check once the stack is up. DW_STORAGE_ACCESS_KEY and DW_STORAGE_SECRET_KEY are generated into the secret store.

storage: external is an S3-compatible storage somebody else runs. Deliver DW_STORAGE_ENDPOINT, DW_STORAGE_ACCESS_KEY, DW_STORAGE_SECRET_KEY and the buckets your rules need (DW_STORAGE_PUBLIC_BUCKET with DW_STORAGE_PUBLIC_BASE_URL, DW_STORAGE_PRIVATE_BUCKET) with secret set. The bucket check stays on: the server refuses to start on a missing bucket, a private bucket anyone can read, or a public one nobody can.

Moving files to another storage or other buckets — bundled to external, or a bucket renamed — is a copy, not a migration: copy every object of the public bucket into the new public bucket and every object of the private one into the new private bucket, under the same keys, then point the DW_STORAGE_* secrets at the new place and deploy. The database names no bucket: a file's bucket is the one the configuration names for its visibility (D-138), so every existing file is served from the new buckets at once. Neither the bucket check nor the outside probe reads objects, so a key that was not copied surfaces only when someone opens that file.

setup — provision a server and render its stack

setup is the only subcommand that writes infrastructure. In order:

1. Root privileges — root, or passwordless sudo; an unreachable host is not reported as a privilege problem.
2. Base packages and Docker, installed only where Docker is missing.
3. BBR congestion control: loads tcp_bbr now and on boot, verifies that the kernel provides it, and writes the host's fq queue discipline and BBR selection under /etc/sysctl.d. The rendered front proxy also selects BBR in its own network namespace, where the browser-facing TLS sockets actually live; a host-only setting is not sufficient because congestion control is namespaced. The proxy is the only service that accepts long-RTT public TCP, including traffic for storage on its own domain.
4. The deployment user, created if absent and added to the docker group.
5. The stack-identity guard: setup refuses when the server runs other stacks and not this project's — the shape of a repository that moved without project: — before the secret store is created under the new name. --new-stack skips this one check, for a genuinely second project on the same host, and nothing else.
6. The secret store, with the secrets that are only random strings generated in place: with database: bundled, DW_DATABASE_PASSWORD; and with the bundled storage, the storage keys. An external database's DW_DATABASE_* are never generated — they are the provider's own credentials, delivered with secret set. Existing values are never replaced — regenerating the database password would lock the server out of a database initialised with the old one.
7. A repository key, for a git@ repository: generated on the server, so the private half exists nowhere else. When the server cannot yet reach the repository, setup stops and prints the public key, asking for it to be registered as a read-only deploy key — the server only ever fetches, and a writable key turns access to the box into access to the repository.
8. The checkout of branch.
9. .env with the generated secrets — what the compose file itself interpolates.
10. A data volume guard: if the server already has a data volume of this project and an expected one is missing — a config change renamed or replaced what a volume held — setup refuses. Compose would otherwise create that volume empty and serve it beside the real data, silently. run runs the same guard (below) right before it starts anything, because a server is not always setup again after a config change.
11. docker-compose.yml, nginx.conf, the nginx.d directories, the override bridge and the project's Nginx snippets, then docker compose config --quiet over the result.
12. The firewall: ufw (installed when absent), OpenSSH, 80, 443 and firewall_ports.
13. A one-day self-signed certificate, so Nginx can start at all — the real one cannot be issued until Nginx answers the challenge, and the first run issues it.

It ends by naming the required secrets still to deliver. Idempotent throughout: every step finds what it needs or creates it, so re-running setup against a live server is the supported way to pick up a change to the rendered files. --dry-run prints the rendered docker-compose.yml and nginx.conf, the snippets it would upload and the secrets it would generate, and touches nothing.

run — deploy

First run evaluates the working-copy checks of deploy check and refuses on any error, pointing at dart run dartway_cli:dartway deploy check --local for the detail. Before its first remote step it also verifies that the host allows BBR; if not, it stops before replacing anything and asks for dart run dartway_cli:dartway deploy setup. Next it checks this server runs this project's stack (stack-identity), the same guard setup runs. The check runs on every invocation, with --resume, --retry-failed and --skip-git-update too, and comes before the step record, the checkout or any step is touched. A moved repository stops here with exit 1 and the reason stack-identity. The server is left untouched and nothing is recorded. The check is not a step, so a resume still passes over the steps the resumed run finished. An existing server converges with one dart run dartway_cli:dartway deploy setup, followed by dart run dartway_cli:dartway deploy run. Then:

1. updates the checkout to origin/<branch> with git reset --hard — the server mirrors the repository, and a stray edit on the box must not block a deploy (skipped with --skip-git-update). CI can pass --revision <sha> (7–64 hexadecimal characters) to deploy exactly the verified commit. It must belong to origin/<branch>, and the run refuses rather than moving backwards when the server is already at a descendant of that commit;
2. writes the override bridge;
3. renders docker-compose.yml and nginx.conf from deploy/config.yaml and this version of the CLI, and says of each whether it changed. Both are derived files, and a derived file written once goes stale in silence: a CLI that had learnt to pass a new build argument met a compose file rendered before that argument existed, and the deploy died inside docker build blaming the project's Dockerfile — while the file to fix was on the server and in no repository. The write goes through cat >, never a rename, because the proxy has its configuration bind-mounted;
4. renders .env from the secret store, refusing — by key name and line number, never by value — when the store is absent, a line is malformed, a key is declared twice, a key is one the compose file sets, or a required secret is missing or empty;
5. checks the merged Compose configuration;
6. the same data volume guard setup runs (above): refuses when an expected data volume of the rendered stack does not exist while another data volume of this project does — the shape of a config change (a rename, a different storage backend) about to serve fresh data next to the real one. One implementation, run from both commands;
7. makes the TLS certificate cover every served host, before anything is built or replaced, through the proxy the previous deploy left running. That proxy answers the ACME challenge for a host added to the configuration since: its port-80 server comes before the nginx.d/http snippets, so Nginx makes it the default for any name no project port-80 server claims. A snippet that names a host with server_name on port 80 takes the challenge for that host. The order is what the previous deploy rendered, so a configuration whose snippets came first gets it from the deploy after the one that renders it. The hosts are read from the certificate Nginx serves, and one that already names them all is left alone, so a routine deploy asks Let's Encrypt nothing; a host added since (a storage domain, a site) extends the lineage with --expand, under the same name; the self-signed certificate of setup is replaced by an issued one, and a failed first issuance puts the self-signed certificate back. Let's Encrypt fails for reasons of its own, and a failure here stops the deploy with the previous version still serving. With no proxy running — a first deploy, a stand that is down — nothing is serving that a failure could take down, and the certificate is left to step 13;
8. reads free space on Docker's data root (docker info, falling back to /var/lib/docker), refuses below min_free_disk (default 10GB), then builds the images. The space check is read-only and runs again before every build, including a resumed build;
9. with the bundled storage, starts it and runs storage-init, printing what it did; with database: bundled, starts Postgres — with database: external there is nothing of the database's to start, the server reaches it directly;
10. replaces the server, one version at a time. The serving server stops gracefully — calls in flight are answered, live sockets close with "server stopping" — and from then on the proxy answers 502, which the app's client retries for up to 30 seconds (a command keeps its idempotency key, so a retry never runs it twice). The new image applies the migrations in a one-off run that serves nothing and takes no jobs (DW_MIGRATE_ONLY=true), then the new server starts and has to become healthy. The gap is the stop, the migrations and the start: seconds for a routine deploy, as long as a migration takes for a heavy one. No two versions ever run at once: old code never writes into a new schema, never claims a job only the new code declares. When the migrations fail (they roll back) or the new server does not become healthy, the image that was serving is started again and the step fails with the server's own log; after a failure past the migrations the previous code runs on the new schema, and the message says so;
11. replaces the web app, and converges the rest of the stack (up -d --remove-orphans);
12. checks the Nginx upstreams against the applied stack — docker compose config --services on the server — and runs nginx -t inside the running proxy. Nginx resolves an upstream once, when it starts, so a snippet naming a service the stack does not have fails at the next proxy restart; this stops the deploy before that restart;
13. makes the certificate cover every served host once more, through the proxy now running: what a first deploy could not ask for at step 7 is issued here, and a failed first issuance puts the self-signed certificate back. On a routine deploy step 7 has already covered every host, and this asks nothing;
14. restarts Nginx and checks it is still running afterwards: restart exits 0 for a proxy that dies a second later on its configuration;
15. verifies the stack from outside (the probes below);
16. cleans up after success: removes unused server and web images belonging to this Compose project, keeping both rollback images and every image still used by a container, then trims the build cache with the build_cache_keep budget, retaining shared cache keys for kept images. Other stacks' images, Postgres, RustFS, Nginx and certbot are preserved. Cleanup reports the removed image and cache-record counts, bytes freed, and the build cache bytes shared with kept images in one line. A cleanup failure is a warning; the deploy still exits 0. --resume retries only cleanup, without replacing services or repeating outside probes.

What keeps a push routine is not that the rendering is skipped but that it is idempotent and reported: a run that changes nothing says "unchanged" for both files. Everything else infrastructural — users, directories, the secret store, the firewall, the first certificate — is still setup alone. --dry-run prints the plan and the probes, and executes nothing.

A step outlives the connection that started it

Outside probes run on the invoking machine; their outcome is journalled before cleanup. If that invocation stops before recording their success, or probes fail, a resume verifies again. The remote steps run on the server detached from the ssh session: its script is written to ~/.config/<project>/deploy-run/ of the deploy user and started in a session of its own (setsid), streams to files, exit code written last. The call that starts a step waits for it, so a routine deploy still makes one connection per step; when that connection breaks, fresh ones wait for the same step for up to fifteen minutes, and past that run stops waiting and says the step is still running. Nothing the invoking machine does — losing its network, or dying because it is a container of the stack whose server step 10 replaces — stops a step midway.

A step that stops without an exit code reports why:

• A remaining <id>.exit.tmp, or less than 1 GiB free on the step directory's filesystem or Docker's data root, reports diskFull, names the short path and its free and total space, and asks to free old images or build cache and run again with --resume.
• With room left, it keeps the message that the machine restarted or something killed the process, followed by free space on both paths.

Both interrupted states are retried by --resume, with or without --retry-failed. No cleanup runs automatically. A build refused by the space check names Docker's data root and the configured minimum; free space or lower min_free_disk in deploy/config.yaml before trying again.

Disk. A successful deploy cleans its old images automatically, including images left by earlier runs. The server and web rollback images are pinned before building and recorded in the deployment journal, so a resume keeps the same targets. If a container image digest cannot be inspected, the tag is accepted only when Docker proves that image still holds the running container. After an interrupted deployment moved the tag, the next fresh deploy recovers by itself: it tries the <project>-<service>:dw-previous pin that deployment left, under the same proof. When neither holds the running container, it is refused before stopping the service. Images go before cache: the containerd image store shares BuildKit snapshots with exported images. Every prune uses --keep-storage with build_cache_keep; shared cache keys for retained images survive so unchanged builds can reuse their layers. Docker excludes shared bytes from that budget, and so does the check after the prune: cleanup warns only when the cache no kept image shares stays above the limit, and reports the shared bytes as information. The reported byte count is the observed increase in free space on Docker's data filesystem (negative changes are reported as zero; containerd may reclaim additional bytes asynchronously). The cache limit applies to the Docker daemon's builder across the machine, not only this stack.

If an older CLI moved the image tag before it failed and left no usable rollback pin, rerun with dart run dartway_cli:dartway deploy run --env <env> --without-rollback-image to consciously allow this one deployment when the running image cannot be resolved. The flag is not stored in config. The CLI records an empty previous-image entry and warns that a failure leaves the service down until the next deploy. When the image can be resolved, it is still pinned for rollback even with the flag.

That covers deploying from inside the stack being deployed (DartWay Studio deploying itself), but the steps after the interruption still need someone to run them: dart run dartway_cli:dartway deploy run --env <env> --resume. It reads the record of the last deployment on the server and, in that deployment's plan, passes over the steps that finished, waits for a step still running instead of starting it a second time, judges again from its output a finished step whose success is checked beyond its exit code (the upstream check), and runs everything from the first step that actually runs — including the services and probes when verification has not finished. Once outside verification passed, a cleanup retry skips every successful step and runs only cleanup. Except for cleanup and the read-only outside probes, a step that ended badly in the deployment being resumed is not run again: the resume stops with that step's reason and the output the server kept, because a self-deploy resumes after every interruption and a failing step would otherwise be repeated until the attempts ran out — each time stopping the server it had just started again. --retry-failed with --resume runs it once more (against the checkout that deployment updated to); after fixing code, deploy anew rather than resume. With --revision, resuming first verifies that a successfully completed checkout is still at that revision. A mismatch, or a recorded plan that had no checkout step, is refused with revision-mismatch; a checkout that previously failed is retried with the requested revision.

A new run starts a new record, says where the previous deployment stopped if it did not finish, and refuses while a step of it is still running — two deployments never interleave on one server.

Progress for a program

--progress json writes one JSON object per line on stdout and moves the prose to stderr. A program reads the events; the prose may change wording at any time, the events may not.

event	Fields
notice	message — what the server said about the previous deployment
plan	steps (id, title) — --dry-run only
run_started	environment, resume, optional revision, steps (id, title)
revision	commit (full hash), subject — after the checkout update, or before the steps when there is none
step_skipped	index, count, id — done by the deployment being resumed
step_started	index, count, id, title, picked_up — waiting for a step already on the server
step_finished	index, count, id, exit_code; stdout, stderr for a step whose output is its result
cleanup	images_removed, cache_records_removed, freed_bytes, cache_shared_bytes when measured; warning, optional message. A warning leaves run_finished.exit_code at 0
step_failed	index, count, id, reason (exit, verdict, busy, exception); exit_code, stdout, stderr or message; state (diskFull or vanished) for a step stopped without an exit code; resumed: true when the step failed in the deployment being resumed and was not run again
services	services (name, status)
probe	title, passed, warning, detail
run_finished	ok, exit_code; optional failed_step and reason (bbr-unavailable, checks, nothing-to-resume, revision-mismatch, revision-not-found, revision-not-on-branch, stack-identity, superseded, unreachable, verification)

Cleanup warnings still close the step: step_finished includes its actual exit code, even when non-zero; an exception emits step_failed with reason: exception. Both leave the successful deployment at exit 0.

Then it verifies from outside, as a browser and an app would, retrying failed probes up to twelve times five seconds apart:

• GET /health on the api host and GET /dw/health on the app host answer 200 ok;
• GET / on the app host is the Flutter index.html, served with a revalidating cache policy, and the build's entry points are not served for reuse without revalidation;
• main.dart.js (or a Wasm build's main.dart.wasm) answers a request accepting gzip with Content-Encoding: gzip and a JavaScript or Wasm content type; an HTML fallback is not a bundle. index.html and flutter_bootstrap.js do not load CanvasKit from www.gstatic.com (an old project Dockerfile is warned about without failing the deploy). The generated useLocalCanvasKit setting selects local files even though Flutter retains the CDN fallback in its loader;
• /dw/live upgrades through both hosts (from the app origin on the app host) and the server speaks on the socket;
• where declared: the site answers 200 text/html; the storage preflight admits a PUT from the app origin with the headers a presigned upload is bound to; the probe object reads anonymously from the public bucket and not from the private one, and neither bucket lists its keys.

A deploy whose probes fail exits 1 and names what was observed.

check — would a deployment work?

check changes nothing. It prints the environment it resolved, then asserts over the working copy and over the network. Exit 1 on any error; warnings and skipped checks do not fail it. --local skips DNS, the server and the deployed hosts — the form that needs no SSH key and no host yet.

Id	Level	Asks
secret-names	error	requires.secrets names nothing the compose file sets itself
stack-names	error	The project prefix makes a valid database name and bucket names
site-source	error	The site directory holds an index.html committed to Git — the server has only what Git has
docker-context	warning	A .dockerignore exists at the build context root
dockerfiles-present	error	Both <project>_server/Dockerfile and <project>_flutter/Dockerfile exist
docker-context-packages	error	Each image copies every package it depends on, and .dockerignore admits it — a missing one fails as pub get exit code 66, three layers from the cause
dependencies-inside-context	error	No image resolves a package by a path outside the project — the context is the project root, so pub get in the image cannot find it though every checkout resolves. pubspec_overrides.yaml counts unless .dockerignore keeps it out. The usual cause is an unpublished framework taken from a local checkout: depend on it by git with a pinned ref instead
server-signals	error	The server image's ENTRYPOINT (or CMD) is in exec form, so the binary is PID 1 and receives the SIGTERM a deploy sends; under /bin/sh -c it is killed mid-call when the grace period runs out
locked-dependencies	error	Every image's pub get runs --enforce-lockfile, and the package has a committed pubspec.lock — otherwise pub may resolve a different set inside the container than the project was tested with, and a deploy has already failed that way
web-flutter-version	error	The web image builds on the Flutter .fvmrc names — it installs it from .fvmrc, or its Flutter image carries that tag. Flutter pins some packages an app resolves, so a lock written on one Flutter fails --enforce-lockfile on another
web-backend-url	error	If the web Dockerfile declares ARG DW_BACKEND_URL, its build consumes the argument that deploy run passes from app_domain. Skipped when the Dockerfile declares no such argument
web-file-modes	warning	The web image grants everyone read on the files it serves (chmod -R a+rX build/web). COPY keeps modes, and under a deploy user's umask of 077 the files a commit adds are served as 403 while every page answers 200
web-cache-policy	warning	The web image's Nginx configuration revalidates every Flutter entry point
nginx-upstreams	error	Every proxy_pass in the rendered Nginx and in deploy/nginx.d/ names a service of the stack or the override
override-web-build	warning	deploy/compose.override.yml does not rebuild web — that would name the API address a second time, and nothing compares the copies
local-secrets-untracked	error	deploy/secrets.yaml is not tracked by Git
local-secrets-cover-environment	warning	deploy/secrets.yaml holds every required secret for the environment (reported by check only; run does not evaluate it)
dns-public-hosts	error	Every served host resolves to the deployment host — a mismatch otherwise burns the Let's Encrypt rate limit
images-resolve	error	Every pinned base image (nginx, certbot, Postgres with database: bundled, and with storage: bundled the storage and its init image), resolved through registry_mirror exactly as the renderer resolves them into the compose file — against its own registry, a manifest HEAD with the anonymous token the registry's challenge asks for, so a vanished tag is caught here rather than mid-run, at the step that starts it. A transient answer (no route, a timeout, a rate limit, a registry's own 5xx) is a skip, not a failure: this machine's network is not a fact about the image
ssh-reachable	error	Key-based SSH works; when it fails the other server checks are skipped
deploy-user	error	The deployment user exists
docker-available	error	Docker Compose is usable by the deployment user
docker-free-space	warning	Docker's data root has at least min_free_disk free (default 10GB); names the path and available space below the threshold. The build itself refuses below this minimum
stack-identity	error	The server runs this project's stack, or none at all — not only other ones; see Moving the repository
proxy-congestion-control	error	The running front proxy reports bbr from its own network namespace; the host's value cannot stand in for this reading
host-congestion-control	warning	The host reports bbr; re-run deploy setup when it does not, so SSH and registry pulls use the framework setting too
runtime-secrets	error	Every required secret is in the server store with a value, and nothing reserved is
secret-files	error	Every requires.files entry is delivered and mounted into the server container, read from the configuration Compose will actually run
database-reachable	error	With database: external: DW_DATABASE_PORT/_SSL/_MAX_CONNECTIONS/_CA_FILE are validated exactly as the server parses them, then a throwaway, pinned Postgres client on the deployment host connects to the stored coordinates with the same sslmode the server will use (require, verify-full with a CA file, or disable) and runs a query — a real authenticated connection, not a bare TCP probe. A failure names why: a malformed stored value, DNS, refused, a timeout, authentication, the server not offering TLS, or its certificate not verifying against the configured CA. Skipped with database: bundled
secrets-match-local	warning	The server store and deploy/secrets.yaml hold the same key names
outside	error	The same outside probes run ends with, against whatever is deployed now; runs even when SSH fails
web-resources-local	warning	The deployed startup files do not load CanvasKit from www.gstatic.com; an old web Dockerfile is named with the migration that adds --no-web-resources-cdn

Images

dartway create gives a project both Dockerfiles, the web image's nginx.conf and a .dockerignore; deploy check holds them to the rules above.

• The server image builds from the project root (it needs the shared package beside the server), copies the shared and server packages by name, compiles bin/server.dart with dart compile exe, and runs the binary on Alpine with ca-certificates — without them every outbound HTTPS call from the server fails — and ENTRYPOINT ["/app/server"] in exec form.
• The web image builds with Flutter from the project root, without a backend build argument. It runs flutter build web --release --no-web-resources-cdn, uses Uri.base.origin for calls, and serves the build with nginx:1.30.5-alpine and <project>_flutter/nginx.conf. The same image serves stage and prod. deploy run supplies STUDIO_APP_ORIGIN (the app's own address) as a build argument for projects whose Dockerfile forwards it to a Dart define; an app without the binding needs no such argument. A Kamal web image omits the define: the Studio binding takes its own Uri.base.origin in a web release.
• .dockerignore denies everything and admits the packages by role suffix (*_server/, *_flutter/, *_shared/), minus build output and .env — so the working copy's history, build output and local secrets never reach the Docker daemon.

What the web image lets a browser keep fails silently. A Flutter web build hashes nothing: index.html, flutter_bootstrap.js, main.dart.js and everything under assets/ carry the same names in every build. The familiar "cache for a year" rule lands on exactly the files that change on every deploy, and a browser holding one under a long max-age keeps running the previous build with nothing on either side to report it. So nginx.conf serves everything with Cache-Control: no-cache and an ETag (a 304, not a download) and keeps the long-lived, immutable rule for names that carry a content hash. web-cache-policy reads that configuration; the outside probe asks the deployed site. Fixing it does not reach a browser that already holds a copy — tell whoever you can reach to hard-reload.

The rendered front proxy compresses responses from the app and optional static-site hosts with gzip (level 5, for JavaScript, Wasm, CSS, JSON, SVG, text and web manifests) once they reach 1 KiB. It leaves the API host and the app host's /dw/ call locations untouched: compression here owns large static delivery, while the call protocol remains exactly what the server emits. Nginx adds Vary: Accept-Encoding; its compressed weak ETag continues to make If-None-Match revalidation a 304. The cache-control split above is unchanged.

Secrets

local: this machine

local is an environment like any other in these two files, and the only one with no machine to reach: deploy/config.yaml > local holds what the team shares — the coordinates of the development containers — and deploy/secrets.yaml > local holds what is the developer's own. The project's entry points read both through DwLocalEnvironment (dartway_core_server), in that order, and a real environment variable beats them. A deployed server reads neither: .dockerignore admits only the packages into a build context, so deploy/ never enters an image, and the runtime stage holds nothing but the compiled binary.

dartway secret list --env local             # both halves, each key marked with its file
dartway secret set SMS_API_TOKEN --env local

init has nothing to generate there (the coordinates are committed), push and pull have nowhere to go (the file is the store), and put-file has nothing to mount — a document a local server reads is a path on that machine, so a variable points at it. Each says so rather than doing something surprising.

On a server

Output of deploy run (including resumed steps), deploy setup and deploy check is masked on the target before it leaves over SSH. The filter reads secrets.env as the deployment user and replaces occurrences of stored values with ***: the value as stored, and three percent-encoded forms of it — fully encoded (only A-Za-z0-9_.~- kept), the URI component form (also keeping !*()), and the RFC 3986 userinfo form Uri(userInfo:) produces (also keeping !$&'()*+,;=:) — each with its hex digits in either case. Other partial encodings are not recognised. Values shorter than 6 characters remain visible, so common values such as true and 5432 do not obscure unrelated output. Both stdout and stderr are filtered, including the output in JSON progress events. Before the store exists, output passes through unchanged.

Command output is collected until the command finishes. An SSH login other than deploy_user needs passwordless sudo -u <deploy_user> to run the filter. If an existing store cannot be read or the filter cannot run, the command fails without returning unmasked output.

The store is one file on the server, /home/<deploy_user>/.config/<project>/secrets.env, mode 600, outside the checkout so the git reset --hard of a deploy cannot touch it. Lines are KEY='value', single-quoted so nothing in a value is interpreted — which is why a value cannot hold a single quote or a line break. Names are upper-case environment variable names, not starting with COMPOSE_ (Compose would read those as its own settings). Every deploy renders the checkout's .env from the store, on the server, so no value makes a round trip through the maintainer's machine. A running server keeps the environment it started with: a changed secret takes effect on the next run.

Subcommand	Does
secret init	Creates the store and generates the random-string secrets that are missing; never replaces a value. setup does this too
secret set <KEY>	Stores one value read from stdin, so it appears in neither shell history nor a process list. Refuses an empty value and the names the compose file sets
secret list	Names only — which are required, generated, empty or refused — plus the stored files. Values are never read
secret put-file <path>	Uploads a document (a service-account JSON) into the store with mode 600; --name stores it under another name. It reaches the server only when declared under requires.files, mounted read-only at /run/secrets/<name> — re-run setup after declaring one
secret push	Adds to the server store what it lacks from the environment's section of deploy/secrets.yaml; a key whose server value differs is refused by name, never replaced silently
secret pull	Copies keys the server has and the local file lacks; --overwrite KEY[,KEY…] takes the server value for exactly the named differing keys

deploy/secrets.yaml is optional: the maintainer's copy of every environment's secrets, one section per environment and no shared section, git-ignored by the skeleton's deploy/.gitignore. Secrets can live on the servers alone.

push's default is additive, not a replace. It sends a key the server lacks or holds empty, leaves a key whose server value already matches alone, and — for a key whose server value differs — refuses the whole push, sending nothing, naming every such key. That is what keeps it from being the easy way to move an environment to a new host: setup generates a fresh DW_DATABASE_PASSWORD (with database: bundled) and storage keys there, bound to that host's own data volume, and a push that quietly overwrote them with an old host's values (still sitting in deploy/secrets.yaml) would lock the server out of its own database on the next run.

--overwrite KEY[,KEY…] replaces exactly those named keys on purpose — there is no --overwrite-all, since naming the key is the point. A key dartway secret list marks generated is bound to the data already on the server on every path that touches it, not only a replace: naming it in --overwrite is required for --prune to drop it or --allow-emptying to blank it too, on top of whichever of those two flags would otherwise be enough on its own for an ordinary key — it is the kind of value a swap, a drop or a blank all break the server over the same way. push refuses before sending anything when a value cannot be stored at all, and keeps its two older guards against losing information, each behind its own flag: --prune allows dropping keys the server has and the file does not (usually the file is behind, not the server holding junk), and --allow-emptying allows blanking a value the server has — unaffected by --overwrite for an ordinary key. Before sending anything, push prints a per-key plan — add, keep (same), overwrite, differs — refused, drop — the same shape --dry-run prints, which sends nothing.

The comparison behind that plan runs on the server itself, over the local values sent on stdin — only key names and a checksum of the store travel back, never a value, the same rule the store keeps everywhere else. That checksum is checked again right before push writes, so a second push, or a hand edit, landing in between the two is refused rather than silently undone; and a store that exists but cannot be read fails the comparison outright, rather than being read as an absent one that every key would then look new against.

pull prints its plan as key names only: additions, requested overwrites, remaining differing keys, server-empty keys and local-only keys. Without --overwrite, a difference includes a ready-to-run command naming every differing key whose server value is non-empty. --overwrite KEY[,KEY…] replaces only those named values; there is no blanket overwrite because the local file may hold the only copy of a rotation that has not been pushed yet. An empty server value never replaces a non-empty local value: pull refuses and directs the maintainer to push instead. --dry-run prints the same plan without writing. A completed write is read back and verified; if the file cannot be parsed or a changed key does not match, its original bytes are restored.