AI Workbench Projects#

Diagram of the folder structure of a Workbench project showing the .project folder with spec.yaml, requirements.txt, apt.txt, preBuild.sh, and postBuild.sh files, as well as the src folder with the project code.

An AI Workbench project contains everything you need to do your work: code, data, models, environment configuration, metadata, and work history.

Overview of Workbench Projects#

Projects are Git repositories#

  • They have a few extra files that provide metadata that drive automation and the overall user experience.

  • Any Git repository can be an AI Workbench project by adding the correct files.

  • Projects necessarily imply a containerized development environment, i.e. the project container.

  • Projects necessarily imply Git versioning (git vs git-lfs vs git-ignore) for individual files and folders.

Workbench makes projects portable and reproducible#

  • Workbench handles the Git and container commands to build, run and version the project.

  • The trade-off is that you don’t do things like git clone or docker run.

  • Instead, you use the Desktop App or the Workbench CLI for those kind of things.

  • However, nothing is reproducible by default. You may need to manage some things yourself.

    • Making sure the system has the correct hardware (GPUs, etc.) to run the project.

    • Making sure you have any secrets, like API keys, properly configured in the project.

Note

The best way to understand how projects work is to work through the exercises in the Onboarding Project.

Default AI Workbench Project Structure#

AI Workbench projects are specially-formatted Git repositories. When you create a new project, AI Workbench creates the following default structure. These files together represent the AI Workbench project.

Use the project filter bar to filter your file browser view. To duplicate, edit, rename, or delete a file, right-click it and select either Duplicate, Edit, Rename, or Delete. You can also select these commands from the … menu on the right.

Note

  • To make a Git repository into an AI Workbench project, create the .project/spec.yaml file. For the full spec file reference, see The Project Specification.

File or Folder

Description

Required?

Git?

.project/configpacks

Tracks when configpacks are run to avoid duplication.

Yes.

Tracked by Git.

.project/spec.yaml

Metadata defining the project. For the full reference, see here.

Yes

Tracked by Git.

code/

Suggested location for your code files.

No

Tracked by Git.

data/

Suggested location for your data files.

No

Tracked by Git LFS.

data/scratch

Suggested location for intermediate data products.

No

Ignored by Git.

models/

Suggested location for your models.

No

Tracked by Git LFS.

.gitattributes

Defines Git attributes for relevant folders. For more information, see here.

Yes

Tracked by Git.

.gitignore

Lists files and folders that you don’t want tracked by Git. The default targets typical python development. For more information, see here.

Yes

Tracked by Git.

apt.txt

Adds apt packages to your project. Specify 1 package per line. For more information, see Packages.

No

Tracked by Git.

postBuild.bash

Runs after installing packages. Runs as the container user with passwordless sudo access. For more information, see Scripts.

No

Tracked by Git.

preBuild.bash

Runs before installing packages. Runs as the container user with passwordless sudo access. For more information, see Scripts.

No

Tracked by Git.

README.md

The README file for your project, written in markdown.

No

Tracked by Git.

requirements.txt

Adds pip packages to your project. Specify 1 package per line. For more information, see Packages.

No

Tracked by Git.

variables.env

Non-sensitive environment variables set in the container at runtime. Variables are not built into the container image. For more information, see Environment Variables.

No

Tracked by Git.

AI Workbench Projects and Git#

We recommend that you don’t commit intermediate data, or other large files that you can easily recreate, to avoid bloating your Git repository. You can use the data/scratch directory, which is ignored by Git, to store temporary data. You can also create your own directory for this purpose and add it to the .gitignore file.

AI Workbench warns you when large files are about to be tracked by Git. You can move the files to an untracked location, to a Git LFS tracked location, or delete them altogether. Git LFS tracked locations are listed in the .gitattributes file.

For more information, see Git in AI Workbench.

Persist Data Between Container Restarts#

Screenshot of the mount model with volume mount selected.

When you work in a container, files written to the container’s filesystem do not persist when you restart the container. AI Workbench removes containers when they stop, which means that any changes you make to the container are lost.

To persist project data between restarts, use a mount. A mount allows you to persist data even after the container is stopped or removed. A host mount binds a directory from your host computer into the container. A volume mount creates a Docker/Podman volume to store data.

For more information, see AI Workbench Mounts.

Note

By default, each project has a project mount for the container so that changes you make to project code itself (like postBuild.bash) persist across container restarts.

FAQs#

Common questions on projects#

Can I manage projects using the CLI?#

Yes. The CLI has feature parity with the Desktop App.