Guide: Cargo via Docker on macOS

This guide describes what is in place for running Cargo (build, test, check) via Docker on macOS and how to use it when native Cargo is problematic (e.g. Rust version mismatch, tss-esapi or other platform issues).

What’s in place

All commands below are run from the repository root unless noted.

Prerequisites

1. Docker
   Docker Desktop for Mac (or another Docker runtime) installed and running.

2. cargo-make
   Used to run the Makefile tasks. Install if needed:

   $
   cargo install cargo-make

3. CARGO_HOME (optional but recommended)
   So the container can reuse your Cargo cache:

   $
   export CARGO_HOME="${CARGO_HOME:-$HOME/.cargo}"

Colima configuration (Apple Silicon M1/M2/M3)

If you use Colima on an M3 (or M1/M2) Mac, these settings can make cargo-docker-minimal and tests run faster.

Recommended start command (one-liner)

$
colima start --arch aarch64 --cpu 4 --memory 12 --disk 60 --vm-type=vz --mount-type=virtiofs

Use colima stop first if Colima is already running, then run the command above. This gives native arm64, 4 CPUs, 12 GiB RAM (avoids linker OOM when building carbide-api tests), VZ virtualization, and virtiofs mounts. Adjust --cpu and --memory to match your machine.

1. Use native arm64 (default)
Do not start Colima with --arch x86_64. The minimal image is built for your Mac’s architecture (arm64), so it runs natively. Forcing x86_64 would use emulation and be much slower.

2. Give the VM more CPU and memory
Rust builds are CPU- and memory-heavy. Defaults (e.g. 2 CPU, 2 GiB RAM) are too low. Example for an M3:

$
colima stop
$
colima start --cpu 4 --memory 8 --disk 60

Adjust to your machine (e.g. 6–8 CPU, 8–12 GiB RAM if you have it).

3. Use VZ runtime and virtiofs (macOS 13+)
Apple’s VZ virtualization and virtiofs mounts are faster than the default QEMU + 9p/sshfs setup:

$
colima stop
$
colima start --cpu 4 --memory 8 --disk 60 --vm-type=vz --mount-type=virtiofs

Or edit the config and start:

$
colima start --edit

In the editor, set (or add) something like:

1
cpu: 4
2
memory: 8
3
disk: 60
4
vmType: vz
5
mountType: virtiofs

Then save, exit, and run colima start.

4. Rebuild the minimal image after changing Colima
If you change CPU/memory or VM type, rebuild so the image uses the new VM:

$
cargo make build-cargo-docker-image-minimal

Summary

The minimal image uses the lld linker (lower RAM use than default ld). If linking still fails with signal 9, give Colima more memory (e.g. --memory 12), then colima stop and colima start again, and rebuild the image: cargo make build-cargo-docker-image-minimal.

Tests to run before build

Use these to validate changes (e.g. IB partition update) before building the Docker image or doing a full build.

When to run what

Use the smoke test for quick feedback; run the IB partition tests when you need to validate that feature (e.g. before a PR). The minimal image uses sccache and a persistent cache dir (~/.sccache or SCCACHE_DIR), so the second and later runs of the same (or similar) commands are much faster.

Option A: Quick check (no database)

Verifies that the API and deps compile. No Postgres needed.

$
# From repo root
$
cargo make cargo-docker-minimal -- check -p carbide-api --no-default-features

If you haven’t built the minimal image yet, build it first: cargo make build-cargo-docker-image-minimal.

Option A2: Postgres connectivity smoke test (small, fast)

Verifies that the minimal Docker setup can reach Postgres: DATABASE_URL is passed into the container and the DB is reachable (e.g. via host.docker.internal). Uses the small postgres-smoke-test crate so it compiles quickly.

1. Start Postgres (e.g. docker-compose up -d postgresql or use your existing Postgres). If you get an error about the loki logging plugin, start Postgres without it: docker-compose -f docker-compose.yml -f docker-compose.no-loki.yml up -d postgresql.

2. Run the smoke test:

$
export DATABASE_URL="postgres://carbide_development:notforprod@host.docker.internal:5432/carbide_development"
$
cargo make cargo-docker-minimal -- test -p postgres-smoke-test

Use the correct host/port (e.g. host.docker.internal:30432 if Postgres is on port 30432 on the host). The cargo-docker-minimal task adds --add-host=host.docker.internal:host-gateway so the container can reach the host (required on Colima; harmless on Docker Desktop). If the test passes, Postgres connectivity from the container is working.

Quick one-liner (recommended for a fast sanity check):

$
export DATABASE_URL="postgres://carbide_development:notforprod@host.docker.internal:5432/carbide_development"
$
cargo make test-docker-postgres-smoke

Takes ~10–30 seconds. Use this instead of the long carbide-api DB tests when you only need to confirm Docker + Postgres work.

Option B: IB partition tests (need PostgreSQL)

These tests use a real database and take a long time (first run compiles most of the workspace). Prefer Option A2 (smoke test) for a quick check; run these when you need to validate IB partition (or other API) behavior.

1. Start Postgres (if not already running):

$
docker-compose up -d postgresql

If you get an error about the loki logging plugin, use: docker-compose -f docker-compose.yml -f docker-compose.no-loki.yml up -d postgresql.

2. Run IB partition tests:

Locally (Rust 1.90 + DATABASE_URL):

$
export DATABASE_URL="postgres://carbide_development:notforprod@localhost:5432/carbide_development"
$
cargo test -p carbide-api ib_partition --no-default-features --no-fail-fast

Via minimal Docker image (after build-cargo-docker-image-minimal):

$
export DATABASE_URL="postgres://carbide_development:notforprod@host.docker.internal:5432/carbide_development"
$
cargo make cargo-docker-minimal -- test -p carbide-api ib_partition --no-default-features --no-fail-fast

Use host.docker.internal so the container can reach Postgres on the host (Docker Desktop for Mac supports this; the minimal image includes libpq-dev for the Postgres client).

DATABASE_URL format: Use the full URL including the database name (e.g. .../carbide_development). The test harness connects to that database to create and drop per-test databases (e.g. db0_carbide__tests__...). The database you name in the URL must already exist (create it or use an existing one such as carbide_development).

Other test filters: The test name is a substring match. Examples:

$
# Examples (set DATABASE_URL first)
$
cargo make cargo-docker-minimal -- test -p carbide-api test_create --no-default-features --no-fail-fast
$
cargo make cargo-docker-minimal -- test -p carbide-api update_ib --no-default-features --no-fail-fast
$
cargo make cargo-docker-minimal -- test -p carbide-api update_instance_ib --no-default-features --no-fail-fast

Suggested order before a build

1. Run Option A (check) to ensure everything compiles.
2. If you changed API or DB code, run Option B (ib_partition tests) with Postgres and DATABASE_URL set.
3. Then run your full build (e.g. cargo make cargo-docker-minimal -- build -p carbide-admin-cli --release).

Quick start (recommended on Mac)

Use the minimal path. The workspace (including carbide-rpc) needs protoc to compile .proto files, so you build a small image once (~2–5 min) that adds only Rust + protoc.

First time only — build the minimal image:

$
# From repo root
$
cd /path/to/bare-metal-manager-core
$
$
cargo make build-cargo-docker-image-minimal

Then run Cargo as needed:

$
# Build admin-cli (default command)
$
cargo make cargo-docker-minimal
$
$
# Or pass a custom cargo command (each argument after --; do not wrap in quotes)
$
cargo make cargo-docker-minimal -- build -p carbide-admin-cli --release
$
cargo make cargo-docker-minimal -- check -p carbide-api --no-default-features

Output binaries appear under target/ in your repo as usual.

How to use: Minimal path (cargo-docker-minimal)

When to use: Day-to-day builds and checks on macOS when you don’t need the full API test stack (PostgreSQL, TSS, etc.).

What it uses: The image carbide-build-minimal (Rust 1.90 + protoc). You must build it once with build-cargo-docker-image-minimal (~2–5 min). The carbide-rpc crate needs protoc and the Google well-known proto files (libprotobuf-dev), so the bare rust:1.90.0-slim-bookworm image is not enough for workspace builds.

Usage

$
# Default: build admin-cli in release mode
$
cargo make cargo-docker-minimal
$
$
# Custom cargo command (pass each argument after --; do not wrap in quotes)
$
cargo make cargo-docker-minimal -- build -p carbide-admin-cli --release
$
cargo make cargo-docker-minimal -- check -p carbide-api --no-default-features
$
cargo make cargo-docker-minimal -- build -p carbide-api --no-default-features

What works

• Building carbide-admin-cli.
• Building/checking carbide-api with --no-default-features (avoids tss-esapi and other heavy deps).
• Other crates that don’t need PostgreSQL, protobuf, or TSS.

What doesn’t work

• Full carbide-api with default features (needs TSS/measured-boot stack).
• API tests that require a database (slim image has no PostgreSQL client libs). For those, use the full image path below or run tests elsewhere (e.g. CI).

How to use: Full image path (build-cargo-docker-image + cargo-docker)

When to use: When you need the full environment (e.g. run API tests with PostgreSQL, or build with default features). On Apple Silicon the image build is slow (45+ minutes); prefer the minimal path for routine work.

Step 1: Build the image (once)

$
cargo make build-cargo-docker-image

Or manually:

$
docker build -f dev/docker/Dockerfile.build-container-x86_64 -t carbide-build-x86_64 dev/docker

On Apple Silicon this runs under emulation and can take a long time; step 7 (installing cargo tools) alone often takes 45+ minutes.

Step 2: Run Cargo in the full container

$
# Build admin-cli
$
cargo make cargo-docker -- build -p carbide-admin-cli --release
$
$
# Run API tests (set DATABASE_URL if tests need Postgres)
$
export DATABASE_URL="postgres://carbide_development:notforprod@host.docker.internal:5432/carbide_development"
$
cargo make cargo-docker -- test -p carbide-api ib_partition --no-fail-fast

For tests that need Postgres, start it first (e.g. docker compose up -d postgresql) and use host.docker.internal in DATABASE_URL so the container can reach the host’s Postgres.

Command reference

When to use which option

Tips and troubleshooting

1. Cargo cache
   The tasks mount CARGO_HOME (e.g. $HOME/.cargo) into the container so dependency builds are cached on your Mac.

2. sccache (faster repeat builds)
   The minimal image uses sccache and mounts ~/.sccache (or SCCACHE_DIR) so compiled Rust artifacts are reused across runs. Rebuild the minimal image once (cargo make build-cargo-docker-image-minimal) to get sccache; then the second and later runs of test -p carbide-api ... (and other cargo commands) are much faster.

3. File ownership
   cargo-docker-minimal runs the container as root and runs chown -R on /code after each run, so target/ is left owned by your user and “Permission denied” when writing to target/ should not recur.
   If you still see “Permission denied” (e.g. after using another Docker task that writes to the repo as root), from the repo root run:

   $
   cargo make fix-target-permissions

   or manually: sudo chown -R $(id -u):$(id -g) target (or . for the whole repo).

4. Postgres for API tests
   Start Postgres (e.g. docker-compose up -d postgresql; if the Loki plugin is missing, use -f docker-compose.no-loki.yml as well), then set:

   $
   export DATABASE_URL="postgres://carbide_development:notforprod@host.docker.internal:5432/carbide_development"

   Use host.docker.internal (not localhost) so the container can reach Postgres on the host. cargo-docker-minimal passes DATABASE_URL into the container when set.

5. Docker or tests hang

   • DB tests: If DATABASE_URL is unset or uses localhost, the container cannot reach Postgres and tests can hang. Set DATABASE_URL with host.docker.internal as the host (see above).
   • Stuck containers: Stop them with docker ps then docker stop <container_name> or docker rm -f <container_name>.
   • Timeout: Run tests with a time limit so they don’t hang indefinitely:
     • Linux: timeout 300 cargo make cargo-docker-minimal -- test ...
     • macOS: Install GNU coreutils then use gtimeout, or run without timeout and set DATABASE_URL so DB tests don’t hang:

       $
       brew install coreutils   # one-time; provides gtimeout
       $
       export DATABASE_URL="postgres://carbide_development:notforprod@host.docker.internal:5432/carbide_development"
       $
       gtimeout 300 cargo make cargo-docker-minimal -- test -p carbide-api ib_partition --no-default-features --no-fail-fast
     (300 seconds = 5 minutes; adjust as needed.)

6. Apple Silicon (M1/M2/M3) and Colima
   The full build image is x86_64 and runs under emulation, so it is slow. Use cargo-docker-minimal for daily work; use the full image only when you need it.
   Colima: For faster runs on M3, give the VM more resources and use the VZ runtime with virtiofs (see Colima configuration below).

7. protoc required for workspace builds
   The carbide-rpc crate compiles .proto files and needs the Protocol Buffers compiler (protoc). The minimal image adds only that; the full image includes it as well.

8. Running without cargo-make
   You can run the same docker run locally. The minimal variant (after building carbide-build-minimal once); add -e DATABASE_URL="$DATABASE_URL" when running tests that need the DB:

   $
   docker run --rm \
   >
     -v "$(pwd)":/code \
   >
     -v "${CARGO_HOME:-$HOME/.cargo}":/cargo \
   >
     -w /code \
   >
     -u "$(id -u):$(id -g)" \
   >
     -e CARGO_HOME=/cargo \
   >
     ${DATABASE_URL:+-e DATABASE_URL="$DATABASE_URL"} \
   >
     carbide-build-minimal \
   >
     cargo build -p carbide-admin-cli --release