--- name: ox-manifest description: Author and review ox deploy manifests (ox.toml at the repo root) for the ox1 engine, validate them with `ox check` before pushing, and run ox through its CLI (make the project, set variables, deploy, read logs). Use when generating or editing ox.toml, fixing a validation error, or preparing, deploying or debugging a repo on ox. --- # ox manifest authoring `ox.toml` sits at the repo root and is the whole deploy contract for the ox1 engine: the web process, workers, cron, services, build. No LLM runs at deploy time. ox deploys one repo onto one Ubuntu 26.04 host with systemd, Caddy, PostgreSQL 18, Redis, neo4j, qdrant, and MariaDB (`mysql`) from apt or mise, and mise toolchains. There is no Docker. Releases are built from `git archive` of the commit, and each git submodule is exported at the commit it pins. Use this skill whenever you generate or edit an `ox.toml`, fix a validation error, or prepare, deploy or debug a repo on ox. ## First: the ox CLI Set up the `ox` CLI before anything else. `ox check` validates the manifest offline, and the other commands do everything the console does (make the project, set variables, deploy, read logs and backups), each with `--json`, so you can work through ox without a browser ("Deploying with the ox CLI" below). 1. Run `ox version`. If `ox` is not found, ask the person to install it and wait until they say it is done: `curl -fsSL https://deploywithox.com/install.sh | sh` (macOS or Linux; it installs `ox` in `/usr/local/bin`, or in `~/.local/bin`, which must be on the `PATH`). It is their machine: run the script yourself only when they ask you to. 2. Run `ox projects --json`. Exit code 3 means not signed in: ask the person to run `ox login` in their own terminal. It prints a code and a URL, they approve the code in a browser signed in to ox Cloud, and the CLI keeps its token in `~/.config/ox/credentials` (0600) for 90 days. It waits until the code is approved, so never start it where you cannot read its output while it runs. Never ask for or print the token. 3. If the person would rather not install it, write the manifest from this skill and tell them it was not run through `ox check`; they deploy from the dashboard. ## Workflow 1. **Start from `ox check --json`.** It reads the repo's production recipe for you (the Dockerfile a release workflow builds, release workflows, `Procfile`, `app.json`, compose files) and returns every value with its source, an `evidence` block of the recipe's facts with `file:line`, and `conflicts` where a value differs from the recipe. Fill only what it cannot: secrets, `domains`, the operator's choices, and a reason for each conflict you settle in ox.toml (the recipe's value, or yours with a comment). Then read on for what it does not read (step 2's list). Read its image facts before you write `start`: a `workdir` not marked `container` is the release (`$PWD`), `env` and `export` values rewritten to `$PWD` are paths in the release and ones marked `container` are paths in the image, not on the server, and the entrypoint's `exec` facts are what its script can run, each `conditional` one only on some arguments or settings, so read the script to see which one your process needs; never copy the container's start (its `/run.sh`, `dumb-init` or `setpriv`), and set an export the app needs in your command when a hint names one. 2. **Inspect the repo.** Read the files that drive detection: lockfiles, `package.json`, `pyproject.toml` / `requirements.txt` / `manage.py`, `go.mod`, `Cargo.toml`, `mise.toml` / `.mise.toml` / `.nvmrc` / `.python-version`, `prisma/schema.prisma`. The detection table below says what ox fills from each. Then read the repo's production recipe when it has one (a `Dockerfile` at the root or in a deploy folder like `tools/deploy/docker/`, `docker-compose.yml`, `Procfile`, `app.json`, its self-hosting docs): it is the setup the project tests, and your plan must match it. Install what it installs (a `uv sync` or `poetry install` there without `--no-dev` or `--only main` installs the dev group, so declare that install: Saleor imports pytest at run time), run every process it runs (web, workers, a scheduler) and its migrations, plus a scheduler the app ships that the recipe does not start (Outline's `cron` service, which its default `--services` leave out; a `cron.php`; a token-guarded `/api/cron` route) as a worker or a `[cron]` entry, since nothing else runs it, and give its environment's required values in ox.vars. Its last stage is what runs: copy its build steps after the install (an asset build like Litestar's `uv run app assets build`), its `ENV` lines the app reads into ox.vars (Mealie's `PRODUCTION=true`), and every folder it makes writable for the app (a `mkdir`, `chmod`, `chown` or `VOLUME` of a path inside the app) into `[storage] keep` (Logto's start rewrites `packages/cli/alteration-scripts`). Migrate with the command it runs; when it runs none because the app migrates as it starts, declare no `migrate`, unless the app has that step as a command of its own: then run it as `migrate`, because a first deploy applies every migration and inside the start that counts against the 120 s health wait (authentik's server and worker both run `lifecycle/migrate.py` as they start, more than 360 migrations took over two minutes and the worker's health check gave up; `migrate = "python -m lifecycle.migrate"` runs them first, and the starts find nothing left to do). Its `apt-get install` lines name Debian packages: use the Ubuntu 26.04 name `ox check` suggests. ox check reads none of these, so carry them yourself: a job in another workflow job or a Makefile target the image build depends on (AFFiNE's native server build, PhotoPrism's `make` targets), with every step of that job up to its upload, renames included, so the job that downloads its artifact finds the same files under the same names (AFFiNE's `build-server-native` renames `server-native.node` to `server-native.x64.node`, `.arm64.node` and `.armv7.node`, one per matrix target, and the server bundle resolves all three: build it once and copy it to each name, or the bundle writes no `dist/main.js` and still exits 0), an app's environment schema in code (Outline's), and a CLI's own options (Mautic's install command). 3. **Write the smallest ox.toml.** Declare only what detection cannot know: `domains`, the `[services]` the app needs, `[workers]`, `[cron]`, and any command detection gets wrong. Leave `[tools]` versions, `[build] install`, `[build] commands`, `[app] start`, and `[static]` out when the repo names them. Detection never overrides a declared value. A build runs `[build] install` (the lockfile's install, detected) and then `[build] commands`; declaring `commands` replaces only the detected build steps, never the install, so list a detected step you still need (Django's `collectstatic`) in your `commands`: `ox check` hints each one you left out. Declare `install` only when the packages are not installed from the root lockfile (a monorepo, a venv on a specific Python) or the repo's recipe installs other groups or extras than the detected install (step 2). `install = ""`, `commands = []` and `migrate = ""` say there is none (a static folder served as it is, beside a `package.json` that builds a library): unset keeps what detection finds. A repo with no `ox.toml` at all deploys zero-config, and dependencies even imply `[services]`; once an `ox.toml` exists, `[services]` is exactly what it declares. An `ox.toml` with neither `[app]` nor `[static]` still gets the detected web app when a web framework or a static site proves it. When only a generic entry point does (a `package.json` start script with no web framework dependency, a Go main package, a Cargo binary) and the file declares `[workers]` or `[cron]`, `ox check` stops until it sets `[app] start` or `[app] enabled = false`. A project that serves no web page (a bot, a queue consumer) says `[app] enabled = false`, which also drops a detected static site. 4. **Run `ox check`** from the repo root. It is offline and reads nothing but the checkout. It prints the resolved plan with each value's source (`declared`, `detected:`, `default`) and every problem at once. Read the `build.install` and `build.commands` rows: they are exactly what runs, in order. An app with dependencies and no `build.install` row ships a release with no packages, and it crashes at start. Fix every problem and rerun until it prints `Ready to deploy.` `ox check --json` gives the same data as JSON. These are the same checks preflight runs: a bad manifest or a missing variable stops the deploy before the host is touched. 5. **List the variables the operator must set.** Read `.env.example` (or `.env.sample` / `.env.template`) at the root or one directory deep (not a test suite's or docs site's folder: `test`, `tests`, `e2e`, `playwright`, `cypress`, `spec`, `__tests__`, `testdata`, `docs`). Every key there that ox does not provide and that the operator must answer (an empty example, a secret or login like `SECRET_KEY` or `DB_PASSWORD`, a placeholder like `changeme`, an address of the developer's machine or network like `localhost` or `10.0.0.5`, or a URL with a login in it, user info or a `?token=`) must be set in the dashboard's Variables tab, and the deploy is refused until each one is set. A key with a real example value (`DEBUG=False`, `WORKERS=4`) is not asked: the app's own default stands. Read the app's environment schema in code too (Outline's `server/env.ts`), not only the example file: when the app takes mail as a host and port, ask for host, port, user, password and from address as separate variables, since Outline's `.env.sample` names only a mail service and a plain relay then left email sign-in off. Hand the operator the list with a one-line note per key. Commit the example file with placeholder values only; real secrets never go in the repo. 6. **Push.** A push to the default branch deploys at once (GitHub's webhook), with a poll every minute as the fallback (or the operator presses Deploy). A failed commit is not retried until the next push or a manual Deploy. ## Schema (TOML; every key is optional, unknown keys fail) ```toml domains = ["example.com"] # served by [app], or by [static] when there is no [app] packages = ["ffmpeg"] # apt system libraries only (no toolchains; nothing that changes sign-in, boot or the firewall, like openssh-server or ufw; a server a package ships stays off, declare it in [services] to run it) required_variables = ["JWT_SECRET"] # keys the app cannot start without that no .env.example lists: asked before the first deploy; Generate makes one only for a secret the app makes for itself (JWT_SECRET, SESSION_SECRET, a salt), never for a provider's key or token internal = true # serve nothing public: no preview host, no domain, no PUBLIC_URL; the app runs for its workers on loopback. Refused with `domains` [app] # the one web process; gets $PORT enabled = false # no web process and no detected static site (workers only); takes no other [app] key; default: the detected app start = "serve" # default: detected (§4.2) health = "/healthz" # HTTP 2xx/3xx on this path; default: any answer to GET / but 502-504 port = 9034 # pin; default: allocated and recorded memory = "512M" # this process's cap; default: none, the project shares the server. Set it only to a number the repo gives sandbox = "relaxed" # may write anywhere in the project home, every release included; the only value; default: strict (writes its home and the release it runs; PHP: only its data dir and tool caches) max_upload = "2G" # a PHP app's largest upload (upload_max_filesize, post_max_size 8M above); default: 512M, or a repo php.ini's own limits; PHP apps only stop_before_start = true # an app that cannot run two copies at once (a lock in its data folder, a second fixed port, one instance per database): stop the running copy, then start the new one; costs a short gap per deploy; default: false [static] # files served by Caddy dir = "dist" # relative to the repo root spa = true # a path with no file or page serves 200.html, else index.html; a missing file (.js, .ico, ...) is a 404 api = ["/api", "/admin"] # these prefixes go to [app]; requires [app] paths = { "/static" = "backend/static" } # more built directories, each served at its URL path [build] # runs as the project user with every variable but PORT; OX_RELEASE is the commit, and .git holds it alone (no history, branch, tags or remote; a server's agent that was not updated builds with no .git); no CI is set, as in a Docker build install = "npm ci --include=dev" # runs first; default: the lockfile's install commands = ["make build"] # runs after install; default: detected migrate = "make migrate" # runs after build, before the switch; default: detected [workers] worker = "celery -A app worker" # short form bot = { run = "python bot.py", memory = "512M", sandbox = "relaxed", port = true, health = "/health" } jobs = { run = "celery -A app worker", stop_timeout = "10m" } # a stop waits this long for running tasks (default 90s, up to 1h); the command must be the main process: no pipe or &, and a redirect only after exec stream = { run = "exec node streaming/index.js", port = true, paths = ["/api/v1/streaming"] } # these parts of the site go to this worker, websockets included (default: none) admin = { run = "exec node admin.js", port = true, public = true } # an address of its own, in $ADMIN_PUBLIC_URL; domains = [...] adds names (default: none) [cron] # 5-field cron or @hourly/@daily/@weekly/@monthly, UTC digest = { schedule = "0 7 * * *", run = "python manage.py send_digest" } [services] # built-in types: postgres, redis, neo4j, qdrant, mysql, clickhouse, meilisearch, rabbitmq postgres = { version = "18", extensions = ["vector"] } analytics = { type = "postgres" } # a second database: ANALYTICS_DATABASE_URL redis = {} # private by default qdrant = { port = 9200 } # private only; pin its port [services.search] # a custom service: an entry with run (Meilisearch is also built in: meilisearch = {}) run = "export MEILI_MASTER_KEY=\"$SERVICE_PASSWORD\"; exec meilisearch --db-path \"$SERVICE_DATA_DIR/db\" --http-addr 127.0.0.1:$PORT --env production" tool = "github:meilisearch/meilisearch@1.12.0" # a mise tool; or packages = [...] (apt) health = "/health" # HTTP 2xx/3xx; default: TCP connect memory = "512M" # cap and capacity floor; default 128M provides = { MEILI_URL = "http://${host}:${port}", MEILI_MASTER_KEY = "${password}" } backup = true # default: its data dir is backed up daily [tools] # a mise tool = version (no npm:/cargo:/pipx:-style package backends, no php, no erlang); default: from repo files (§4.2) node = "24" bun = "1.3" [storage] keep = ["media"] # repo paths, linked to the project data dir: writable, survive releases; each is backed up daily as a .tar.gz (Back up now, restore) while the app runs; never services/ or .hf/ (ox's own) [limits] # the project's whole slice (processes + builds); default: none memory = "1G" # set it only to a number the operator gives: a build counts too cpu = 1.5 # cores [models] huggingface = ["org/repo", "org/repo@revision"] ``` Rules behind the fields: - **Names.** Worker and cron names match `^[a-z][a-z0-9-]{0,27}$`, and `app` is reserved. A worker name may not start with `app-`, `cron-` or `svc-` (ox names the web, cron jobs and services that way): put the word at the end instead (`a-app`, not `app-a`). - **Sizes and CPU.** `memory` takes `K`/`M`/`G` suffixes, `cpu` a positive number of cores. A per-process `memory` larger than `[limits] memory` fails validation. - **Ports.** Leave `[app] port` out: ox allocates and records one, and both sides of the zero-downtime switch need their own. A pin (1024-65535, unique on the host) runs one side only and restarts in place, with a moment of downtime per deploy, and a staging copy or preview whose ox.toml pins the same port cannot start beside it. Every process gets a port of its own in `$PORT`: the app, each worker, each cron job, and `ox exec` runs. A worker or cron whose program listens anyway (a worker mode that still serves a status page) binds it on its own, and a program that ignores `PORT` sees no difference. `port = true` on a worker means the project's other processes call it: ox publishes `_URL` and starts it, and waits for its `health`, before the app switches. Name it so that key is free: a worker `media` sets `MEDIA_URL` in every process, over Django's own setting, and `ox check` refuses a key the code reads with a default other than an `http(s)://` or `ws(s)://` URL (call it `media-server`). A worker with `health` is checked on its own `$PORT`, before the app switches too. A server that does not read `PORT` must be told it: granian and uvicorn `--host 127.0.0.1 --port $PORT`, hypercorn `--bind 127.0.0.1:$PORT`, daphne `-b 127.0.0.1 -p $PORT`, waitress-serve `--listen 127.0.0.1:$PORT`, `flask run --port $PORT`; without it they listen on 8000, 8080 or 5000 and `ox check` refuses the start (daphne's `-b` is only its host). gunicorn reads `PORT` itself. Between 9010 and 9999 a process may listen only on its own `$PORT`: a fixed port there fails to bind (permission denied), so never hard-code one. - **A second web process.** When the browser reaches a second process of the app on the app's own address (Mastodon's streaming server at `/api/v1/streaming`, AnyCable at `/cable`, Laravel Reverb at `/app` and `/apps`, a Socket.IO server at `/socket.io`), make it a `port = true` worker with those prefixes in `paths`. ox's proxy sends each prefix and everything under it to the worker's `$PORT`, websockets included, on every domain and the preview link, with the path kept (`/cable/x` stays `/cable/x`); the rest goes to the app. A prefix starts with `/`, is not `/`, has no trailing `/`, and belongs to one worker; the longest wins, and a `[static]` `paths` or `api` prefix under a worker's prefix is refused, as is a worker's prefix at or under a folder the app's `.htaccess` closes (it would answer 403). Every deploy, even an app-only one, restarts the worker in place (it has no second side like the app), so its paths answer 502 for those seconds and its open websockets drop: the clients must reconnect. ox never strips the prefix. - **A second site on a name of its own.** When the app serves a second site only on an address of its own, not under a path (Logto's Admin Console answers only on the host in `ADMIN_ENDPOINT`), run that listener as a `port = true` worker with `public = true` and point the app's setting at `${_PUBLIC_URL}` (`ADMIN_ENDPOINT = "${ADMIN_PUBLIC_URL}"`). Every request for the worker's names goes to its `$PORT`, whole. Its names are its preview link `--.