Docker Compose Environments#
Overview#
- Multi-container environments enable complex AI workflows that single containers cannot support.
You can build full RAG pipelines with ingestion, embedding, vector search, reranking, and generation services. You can create and use microservices architectures with frontend, backend, and supporting services.
- AI Workbench uses Docker Compose to run multi-container applications in a project.
Regardless of whether you are using Docker or Podman as the container engine, AI Workbench manages the entire compose environment with shared networking.
The compose application runs along side your project container, not inside it.
- You define the multi-container application with a compose file in the project.
The compose file specifies services, networks, volumes, and dependencies using standard Docker Compose syntax. It’s versioned along with everything else.
Key Concepts#
- Compose File
A YAML file (typically compose.yaml or docker-compose.yaml) defining services, networks, and volumes. Must use Docker Compose v2 syntax.
- Service
An individual container defined in the compose file. Has its own image, environment variables, ports, volumes, and resource constraints.
- Profile
A tag that enables conditional service activation. Lets you define multiple configurations in one compose file and selectively start services.
- Network
A virtual network that connects services so they can reach each other by name. AI Workbench compose environments use a single named network for all services.
- Volume
Persistent storage that survives container restarts. Used for model caches, database storage, or shared data between services.
When to Use Multi-Container Environments#
- Use compose for applications or services beyond your development environment.
The project container is for code, development tools, and interactive work.
Compose services can handle production-like setups that don’t fit well in the project container. Examples: inference APIs, databases, message queues, monitoring systems.
- Use compose for full application or AI pipelines.
Complex AI applications often need multiple specialized services working together. For example, RAG systems require embedding models, vector databases, reranking, LLM inference, and web frontends.
How Compose Integrates with AI Workbench#
- The project spec.yaml points to your compose file location.
Set the compose_file_path field in environment.compose_file_path.
The path is relative to the project root. AI Workbench reads this file and provides UI controls for compose operations.
- AI Workbench provides Desktop App controls for compose environments.
Start and stop the entire compose stack or individual profiles.
Monitor service status and view logs for each service. No need to use docker compose commands directly in the terminal.
- Compose services can access secrets defined in AI Workbench.
Reference secret variables in compose environment sections using ${VARIABLE_NAME} syntax.
AI Workbench passes configured secrets to compose when starting services. Keeps sensitive values like API keys out of version control.
- Web services in compose can integrate with AI Workbench’s proxy.
Set NVWB_TRIM_PREFIX environment variable to “true” for web services.
AI Workbench proxies these services and makes them accessible through the Desktop App. Users can access compose web UIs without managing port forwarding.
- Compose files are versioned with your project in Git.
Changes to services, configurations, and dependencies are tracked.
Teammates get the same multi-container environment when they clone the project. Container images are specified by tags in the compose file, not stored in Git.
- Services can request GPU resources and manage storage through volumes and networks.
Compose services can be configured with specific GPU assignments, memory limits, and persistent storage. Services communicate through shared networks using service names as hostnames. See Multi-Container Environments (Docker Compose) for detailed specifications and configuration syntax.
Limitations and Considerations#
- All services run on the same host as your project container.
Compose is not a distributed orchestration platform.
GPU and compute resources are limited to the single machine running AI Workbench. For distributed deployments, consider Kubernetes or other orchestration platforms.
- Compose services are not automatically backed up or versioned.
Only the compose file is in version control, not service data.
Volume data, downloaded models, and database contents persist on the host. Back up important service data separately from your project repository.
- Large container images increase disk space requirements.
NIM containers can be 10-20 GB or larger.
Multiple models multiply storage requirements quickly. Ensure adequate disk space before deploying multi-container environments.