Tools

Small command-line tools shared across Inky Labs projects live in this repository’s tools/ directory. Each is a single file with no dependencies beyond a standard interpreter, and documents itself in --help.

Installing

Clone the repository and link each tool onto your PATH, so a git pull updates them:

git clone https://gitlab.com/inkylabs/developers.inkylabs.com.git ~/projects/developers.inkylabs.com
mkdir -p ~/.local/bin
ln -sf ~/projects/developers.inkylabs.com/tools/porkbun ~/.local/bin/porkbun
ln -sf ~/projects/developers.inkylabs.com/tools/inkylabs-audit ~/.local/bin/inkylabs-audit
ln -sf ~/projects/developers.inkylabs.com/tools/bazel-cache-sync ~/.local/bin/bazel-cache-sync
ln -sf ~/projects/developers.inkylabs.com/tools/inkylabs-mr ~/.local/bin/inkylabs-mr
ln -sf ~/projects/developers.inkylabs.com/tools/inkylabs-review-bot-enroll ~/.local/bin/inkylabs-review-bot-enroll
ln -sf ~/projects/developers.inkylabs.com/tools/site-compare ~/.local/bin/site-compare
ln -sf ~/projects/developers.inkylabs.com/tools/inkylabs-new-site ~/.local/bin/inkylabs-new-site

Tools never contain credentials. They read them from the environment or from ~/.env.

`porkbun`

A CLI for the Porkbun v3 API: domains, DNS records and nameserver delegation. It’s written rather than installed because what a wrapper is handed is registrar credentials, and a short, dependency-free script is easier to trust than somebody else’s package.

porkbun ping                                   # which account am I acting as?
porkbun domains --all                          # every configured account
porkbun check <domain>                         # why won’t it resolve?
porkbun dns list <domain> [--type T] [--name N]
porkbun dns set <domain> <type> <name> <content> [--ttl 600]   # an upsert
porkbun dns delete <domain> --id <id>
porkbun ns get|set <domain> [<host> …]         # registry delegation

Credentials. The default account’s keys are PORKBUN_API_KEY and PORKBUN_SECRET_KEY. Each further account is a profile: PORKBUN_<NAME>_API_KEY and PORKBUN_<NAME>_SECRET_KEY, selected with --profile <name>. Inky Labs’ domains are in the default account, so Inky Labs work needs no --profile. That account also holds personal domains, and its keys reach those too (Porkbun’s per-domain API access may narrow that; unverified). See Domains.

Wrong account. When a command fails because the domain is in another configured account, porkbun names that account. It never switches by itself; rerun with the --profile it suggests.

Beyond the CLI. Porkbun’s API covers far more than these commands (DNSSEC, URL forwarding, SSL, glue records), and every write accepts dryRun. The whole reference is greppable at https://porkbun.com/llms-full.txt. Email forwarding has no API and is control-panel only.

`inkylabs-mr`

How a change reaches an Inky Labs project: push a branch to the project’s fork in inkylabs/forks and open a merge request (Source code), and, to release it too, ship it (Releases). Run it on a branch in a clone of the project:

inkylabs-mr                    # send: push this branch to the fork, open (or show) its merge request
inkylabs-mr --ship [minor|major]  # ship: bump the version (patch by default), open it, set auto-merge, watch it deploy
inkylabs-mr --ship --no-watch  # ship, but stop once auto-merge is set
inkylabs-mr --watch [IID]      # watch this branch’s merge request (or !IID) through merge, tag and deploy
inkylabs-mr --title T
inkylabs-mr --no-build         # push without building, testing and uploading first
inkylabs-mr --no-rebase-check  # push even when the branch is behind `master`

It creates the fork the first time anyone needs it (and the remote fork), refuses to push master, and refuses a branch that isn’t rebased onto the project’s current master (merges are fast-forward), checking that before it builds anything. With --no-rebase-check it pushes the branch anyway, and the review bot rebases it once someone sets auto-merge. It never pushes the fork’s master: the review bot keeps that current. Credentials: GITLAB_TOKEN, or glab’s login.

--ship takes the version on master (package.json’s, or VERSION’s), bumps it into the branch’s last commit, whose subject gains “; release ” (a branch with no commits of its own gets a commit of just the bump), and builds after that, so the release is what’s verified. A branch that already raised the version keeps it, so shipping again changes nothing. It refuses a project whose CI doesn’t include ci/release.yml, where a merge deploys nothing. Watching exits 0 once the tag’s pipeline passes (or once merged, for a merge request that didn’t change the version), 1 when something failed, 3 when it’s waiting for a person (an owner’s approval, or a Maintainer to merge), and 4 after --timeout minutes (60).

In a Bazel project, it builds and tests the commit first (bazel test //...), and on a trusted workstation (one with the inkylabs AWS profile) uploads the results with bazel-cache-sync, so the merge request’s pipeline finds every step already done. It builds in an output base of its own (~/.cache/inkylabs-mr/<project>/) with an empty disk cache, so it uploads only what this commit needed, never your everyday disk cache. Last, it records the commit as verified, so the review bot runs no pipeline for it. It pushes nothing if the working tree has changes, if the build or tests fail, if the upload fails, or if the AWS sign-in has expired (it says to run aws sso login --profile inkylabs). inkylabs-mr --no-build skips all of that, for a change too heavy to build locally.

`review-bot`

Not yet set up for Inky Labs: there is no inkylabs_bot user, deploy key, Lambda function or SSM parameter yet. What follows is how it works once there is.

The review bot. Where people merge (review mode), it posts the commit statuses review-bot/approved (the owners in the target branch’s .gitlab/CODEOWNERS have approved) and review-bot/verified (a verified commit with no pipeline of its own), which GitLab’s Pipelines must succeed requires; starts the pipeline itself when a commit isn’t verified; and keeps the auto-merge queue moving: it rebases the first approved merge request someone set auto-merge on and sets auto-merge on it again. It never merges itself. After a merge into master it releases, when the version has no tag yet: it creates the tag and starts its pipeline, with INKYLABS_PREVIOUS_RELEASE (Releases). And it fast-forwards the fork’s master; each sweep finishes either if a run didn’t. It runs as a Lambda function, inkylabs-review-bot: a container image of tools/review-bot/ (worker.py is its entry point), invoked by the webhook receiver tools/review-bot/hook/ and every 15 minutes by a schedule, all described in infra/review-bot/. The projects it looks after are tools/review-bot/projects.txt, one a line: the name, then the branches in review mode (review for master, <branch>=review for another). The design and its history are SVRBC’s, from which the bot is ported: they are in svrbc/review-bot’s PLAN.md.

tools/review-bot/review_bot.py check <project> <iid>        # what it would decide, and why; reads only
tools/review-bot/review_bot.py owners <project> <path>…     # who owns these paths
tools/review-bot/review_bot.py run --dry-run <project>…     # comment and label, never push
tools/review-bot/review_bot.py run --review master <project> # treat master as in review mode
bazel test //tools/review-bot/...                          # the tests, and the image built and checked

Credentials, run by hand: REVIEW_BOT_TOKEN or GITLAB_TOKEN, or glab’s login; merging and releasing also need REVIEW_BOT_DEPLOY_KEY. REVIEW_BOT_USER names whose statuses count as the bot’s (default inkylabs_bot), for trying it with your own login. In Lambda, all of it comes from SSM (/inkylabs/review-bot/token, /deploy-key, /token-expires).

Its token renews itself. The token is a fine-grained token of inkylabs_bot, and /inkylabs/review-bot/token-expires records when it expires. At the end of a run, when fewer than 330 days are left (so about monthly), the bot rotates the token for another year and writes both parameters. So it never nears expiry, and a broken rotation leaves about eleven months to notice. If one rotates but can’t store the new token, the old one is already revoked: the run fails saying so (in the function’s log), and someone makes a new token by hand. To rotate it now, whatever the date, invoke the function with {"ROTATE": true} (review_bot.py rotate prints the command); only the function can store the new token.

Its token needs Project: Read for a private project. A fine-grained token reads public projects without it, so nothing shows the gap until the first private project under the bot. SVRBC found it that way (its tools page): without it, every read of the project and its fork got 403 (insufficient_granular_scope), and the bot took the merge request for one from outside the forks group. Give Inky Labs’ token the permission from the start: many Inky Labs projects are private. GitLab can’t add a permission to an existing token: to make one by hand, sign in as inkylabs_bot, use Duplicate on the old token in Access tokens (it copies the group and permissions), and store the new one in /inkylabs/review-bot/token and /token-expires.

`inkylabs-review-bot-enroll`

Puts projects under the review bot, or checks them. It is idempotent: it sets up whatever is missing of the project’s fork, the bot’s deploy key (with write access), the two webhooks (merge request events from the project, pipeline events from its fork), and what the bot does after a merge: it may create the protected v* tags (in a project that includes ci/release.yml, with inkylabs_bot made a direct member there) and push the fork’s master. It also sets the project’s merge checks (Pipelines must succeed, skipped pipelines not successful, automatic rebase off, fast-forward), adds push events to the webhook, deletes the tests:: labels, and, last, protects master (and any branch projects.txt adds) for people to merge (merge Maintainers, push No one), with the * rule merging No one. It never writes CODEOWNERS or tools/review-bot/projects.txt: those are merge requests, and it says when they’re missing.

inkylabs-review-bot-enroll --check --all          # where every project stands
inkylabs-review-bot-enroll <project>…             # set them up

It needs a token with Owner rights on the projects, and AWS access (AWS_PROFILE, default inkylabs) for the webhook receiver’s URL and secret. Until the bot exists it refuses to run: its KEY_ID and BOT_USER_ID are unset (as are inkylabs-audit’s BOT_KEY_ID and BOT_USER_ID), to be filled in once inkylabs_bot has added the key.

`inkylabs-new-site`

Starts a new site: a static Astro site (Stack, tier 1) on S3 and CloudFront, with everything every Inky Labs site has already in place, so that only what is the site’s own is left to write. It writes from this repository’s templates/, which came from SVRBC’s recipes.svrbc.org, the first site made this way. No Inky Labs site has been made with it yet.

inkylabs-new-site site example.inkylabs.com --name "Example Site" --description "An example Inky Labs site" \
  --color '#1d4e89' --dir ~/projects/example.inkylabs.com    # a fresh clone of the project, or empty
inkylabs-new-site infra example.inkylabs.com --commit --site ~/projects/example.inkylabs.com
                                                          # then inkylabs-mr --ship here; four times

site writes the site’s repository from templates/astro-site: the Bazel build (the site, its icons, //:site_test and the vitest suite), CI with releases and verified commits, a PWA, the special pages, the CloudFront router, the deploy (//:deploy_aws), AGENTS.md and a README, with the domain, names, tagline (--tagline, for the home page’s title; the description by default) and color put in, and inkylabs_bazel pinned to this repository’s newest bazel/.
infra writes the site’s AWS resources and DNS records here, in infra/<domain>-site and the zone’s DNS stack. They come up in four merge requests, each shipped before the next can be written, since each needs what the one before made: the certificate; its validation CNAME; the bucket, distribution and deploy role (CloudFront won’t take an alias whose certificate isn’t issued); and the site’s CNAME. Each run works out which comes next, from the files and from AWS, and writes it; it exits 3 while the step before is still to be applied or ACM is still validating. --commit puts it on a branch from origin/master, committed, for inkylabs-mr --ship; --site puts the distribution’s id in the site’s .gitlab-ci.yml; --check only says which step is next.

It doesn’t make the GitLab project (do that as New projects says), a site at a domain’s apex (that is moving a site), or a zone’s DNS stack. It reads AWS with AWS_PROFILE (default inkylabs), and changes nothing there: the releases apply what it writes.

The template is built and tested here: templates/astro-site is a package of this repository’s workspace, so bazel test //... (and so every inkylabs-mr) builds it and runs its tests, and //:new_site_test writes a site from it and walks the four infra steps against a stand-in. A change to what every site has goes in the template too, so the next site starts with it.

`site-compare`

Proves a new build of a site equal to the old one before it replaces it (Builds and CI). It opens each route on both in headless Chrome, at each width and in each color scheme, and compares full-page screenshots and the page’s text:

site-compare 'https://{host}.inkylabs.com' 'http://{host}.localhost:4322' \
    www/ lofi/ qev/
site-compare https://developers.inkylabs.com http://localhost:4321 / /builds --widths 1280

{host} in the two URL templates stands for each route’s host (<host>/<path>). For each page it prints whether the screenshots match, the share of rows that differ and where the first is, and how the text changed; into --out (default ./site-compare) it writes both screenshots and, for a page that differs, the two side by side from just above the first difference. Exit status is 1 when anything differs beyond --threshold (0.5% of rows). It needs Chrome or Chromium, nothing else: the pixels are compared in the browser.

`bazel-cache-sync`

Uploads a machine’s Bazel disk cache to the shared cache. Bazel only ever reads https://bazel-cache.inkylabs.com; a trusted machine builds into ~/.cache/bazel-disk as usual and then runs this. The bucket and bazel-cache.inkylabs.com exist (2026-10-26), but this hasn’t been run for Inky Labs yet.

AWS_PROFILE=inkylabs bazel-cache-sync --dry-run   # what would go up
AWS_PROFILE=inkylabs bazel-cache-sync
bazel-cache-sync --disk-cache DIR --bucket NAME

It flattens the disk cache’s ac/d6/d6fa… layout to the ac/d6fa… that Bazel’s HTTP protocol reads, and uploads every cas/ blob before any ac/ entry, so a result never appears before its outputs. Entries already in the bucket are skipped.

Credentials are whatever the AWS CLI finds: an SSO profile on a trusted workstation, or in CI the short-lived keys the template gets through GitLab OIDC. It needs only s3:ListBucket and s3:PutObject, and the aws CLI (the CI base image has it).

`inkylabs-audit`

Checks every project in the inkylabs GitLab group, and the site each one serves, against the rules in this handbook. Rules written down drift from what is actually configured; this finds where.

inkylabs-audit                        # every project; only problems are shown
inkylabs-audit --all                  # passing checks too
inkylabs-audit -p inkylabs.com          # one project (repeatable)
inkylabs-audit --no-sites             # skip the Porkbun and HTTP checks
inkylabs-audit --json
inkylabs-audit --list-checks

Check	Rule
`group-default-branch`	New projects default to `master` (Source code)
`group-2fa`	Members must use two-factor authentication. A warning while it’s an open question
`no-stray-forks`	Every fork in `inkylabs/forks` belongs to a project that exists and isn’t archived (Source code)
`default-branch`	The default branch is `master`
`branch-protection`	Nobody pushes to `master` or force-pushes, counting every rule that matches `master` (the most permissive wins), and Maintainers merge it once the review bot’s statuses pass; a `*` rule letting Maintainers merge, or the bot’s key on `master`’s push list, fails it. A warning for a project the bot doesn’t look after yet (Source code)
`gitlab-avatar`	The project has a GitLab avatar, and no other project has the same one (a warning; Source code)
`t3-icon`	A `t3.json` at the root names the project’s logo as its `iconPath`: a committed image that doesn’t draw another file (a warning, or a failure when the image isn’t there; Source code)
`codeowners`	`.gitlab/CODEOWNERS` exists, and its `*` rule names known users (Source code)
`review-bot`	The review bot looks after the project (a warning on every project until Inky Labs has the bot): its deploy key, both webhooks, its line in `tools/review-bot/projects.txt`, the protection, and its key on the protected `v*` tags (in a project that takes releases, with `inkylabs_bot` a direct member, which GitLab requires of a key’s owner to create tags) and on the fork’s `master`. A warning listing what’s missing (`inkylabs-review-bot-enroll` sets it up)
`merge-checks`	Where people merge (review mode): Pipelines must succeed on, skipped pipelines not successful, automatic rebase off, fast-forward (The review bot)
`fork-protection`	The project’s fork, if it has one, takes branches but not pushes to its `master` (Source code)
`fork-merged-branches`	The fork keeps no branch already merged into its `master` (a warning; Source code)
`tag-protection`	Existing tags are covered by a protected-tag rule creatable by maintainers at most
`release-gate`	A project that deploys from CI does it from release tags: it includes `ci/release.yml`, so only a merged version bump (tagged by `inkylabs_bot`) deploys. A warning for one that deploys on every merge to `master` instead, such as GitLab Pages from `master` (Releases)
`fork-pipelines`	A push to the project’s fork starts no pipeline (except for paths `bazel test //...` doesn’t cover), so the review bot can take a verified commit as tested (a warning; Builds)
`master-pipelines`	In a project that releases from tags (includes `ci/release.yml`), a push to `master` starts no pipeline: the merged commit already passed, and the review bot starts each release’s pipeline (a warning; Builds)
`ff-only`	The project and its fork merge fast-forward only, with squashing off (Source code)
`merge-settings`	Source branches deleted and a passing pipeline required (a warning)
`bazel-lazy-tools`	A Bazel build gets its tools from `inkylabs_bazel`’s `nix_tools`, not `rules_nixpkgs` (Builds and CI)
`bazel-shared-cache`	`.bazelrc` reads the shared cache and never uploads through Bazel (Builds and CI)
`bazel-ci-downloads`	CI’s `bazel build //...` passes `--remote_download_minimal` (a warning; Builds and CI)
`bazel-tests`	Tests are Bazel tests: CI calls no test runner (`npm test`, `cargo test`, …) itself, and a `package.json` with tests has a Bazel test that runs them (a warning; Builds and CI)
`bazel-npm-step`	A Bazel repository with `package-lock.json` installs it as a Bazel step (Builds and CI)
`bazel-site-build`	A Nuxt site in a Bazel repository is built by Bazel, from copied inputs (Builds and CI)
`bazel-deterministic-tars`	Tarballs made in build steps are sorted, with zero timestamps
`bazel-glob-ignored`	Globs exclude Git-ignored (generated) paths (Builds and CI)
`bazel-script-symlinks`	Scripts Bazel runs find the repository without following symlinks
`hosting-default`	The site is deployed to AWS (or, for developers, GitLab Pages), not Vercel or Netlify (Hosting)
`lambda-traps`	For a Lambda site: Nuxt Content’s database in `/tmp`, no `aws s3 sync` of a zero-timestamp build, Nitro’s package links followed when zipping (Hosting)
`ci-pinned-tools`	Nothing in CI is fetched `@latest`
`group-runners`	At least two of the `inkylabs` group’s own runners are online (a warning; Builds)
`own-runners`	No CI job is pinned to GitLab’s hosted runners (`saas-linux-…` tags), whose minutes run out (a warning; Builds)
`shell-nix-nocc`	`shell.nix` uses `mkShellNoCC`
`bazel-one-nixpkgs`	`shell.nix` pins the same nixpkgs as `MODULE.bazel` (a warning)
`no-other-branches`	Nobody can push another branch, apart from exceptions `AGENTS.md` explains (Source code)
`tofu-central`	Only developers.inkylabs.com has OpenTofu stacks, in `infra/` (a warning; AWS)
`tofu-lockfile`	Each OpenTofu stack commits its `.terraform.lock.hcl` (AWS)
`tofu-no-state`	No OpenTofu state, saved plan or `.terraform/` is committed (AWS)
`tofu-remote-state`	Each stack keeps its state in S3, locked with `use_lockfile` (a warning)
`tofu-pinned`	A `shell.nix` pins OpenTofu (a warning)
`agents-md`	`AGENTS.md` exists and names this handbook (Agents files)
`claude-md`	`CLAUDE.md` is a one-line pointer to `AGENTS.md` (a warning)
`analytics`	A site with analytics uses PostHog, cookieless, without session replay (a warning; Sites). Inky Labs has no shared PostHog project yet; once it does, its key goes in `POSTHOG_KEY` and the check also warns about a site sending elsewhere
`pages-domain`	A GitLab Pages domain is verified and has a valid certificate
`project-name`	A project that publishes a site on GitLab Pages is named after its domain (a warning; New projects)
`porkbun-account`	The domain is in the default Porkbun account, not one reached with `--profile` (Domains)
`site-up`	The site answers over HTTPS; a Pages site for project members only by sending a visitor to GitLab’s sign-in, and the checks that read its pages are skipped
`page-titles`	The home page’s title is `<site name> — <tagline>`, under about 60 characters, and `/_/`’s is `<page> · <site name>` (or the manifest’s `short_name`, or on a multi-tenant site the name its tenants share), with no other separator (a warning; Sites)
`special-pages`	The site serves `/_/` and `/_/architecture` (Sites)
`source-commit`	The site’s `/_/` links the commit it was deployed from, on GitLab (a warning; Sites)
`pwa`	The site is a PWA: its home page links a manifest with a name, a start URL, a standalone display and 192, 512 and maskable icons, and registers a service worker (a warning; Sites)
`stack-front-end`	A site is built with Astro: static, or rendering on the server through `@astrojs/node` for a reason `AGENTS.md` gives. A warning for anything else, Nuxt included (Stack)
`pages-audience`	A project publishing GitLab Pages keeps it to project members, or says in `AGENTS.md` that its public site is developer-facing (a warning; Hosting)
`stack-api`	A site’s API is a Rust function on Lambda, not JavaScript (a warning; Stack)
`stack-islands`	Islands are Solid, not Svelte, and a component doesn’t destructure its props, which then never update (a warning; Stack)

Which sites. Most Inky Labs sites are hosted elsewhere than GitLab Pages (inkylabs.com’s subdomains are on Vercel today), so GitLab doesn’t know about them. A project counts as serving a site if it has a Pages domain or if its name is a domain (inkylabs.com, lofi.inkylabs.com).

It only reads. Fixing a finding usually affects other people, so the tool reports and a person decides. Exit status is 1 when a check fails; warnings don’t count.

Credentials. A GitLab token from GITLAB_TOKEN (environment, then ~/.env), or else the one glab is logged in with. The domain check runs porkbun and is skipped if that isn’t installed.

Adding a rule to the handbook? Add a check in the same commit: one function in tools/inkylabs-audit, registered in CHECKS with the page that states the rule, and a row in the table above.

The inkylabs-audit skill teaches an agent to run it and work through the findings.