> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/openshell/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/openshell/_mcp/server.

# Run Local Inference with Ollama

> Run Ollama inside a community sandbox or attach a profile for a host-level Ollama server.

This tutorial covers two Ollama deployments:

1. The self-contained Ollama community sandbox. This is the recommended path.
2. A host-level Ollama service shared by explicitly authorized sandboxes.

## Prerequisites

Complete the [Quickstart](/get-started/quickstart) before proceeding.

## Option A: Ollama Community Sandbox

The community image bundles Ollama and supported coding agents:

```shell
openshell sandbox create --from ollama
```

Inside the sandbox, run a local or cloud model:

```shell
ollama run qwen3.5
ollama run kimi-k2.5:cloud
```

Or launch a coding agent against Ollama:

```shell
ollama launch claude
ollama launch codex
ollama launch opencode
```

For an automated workflow:

```shell
ollama launch claude --yes --model qwen3.5
```

Useful starter models include `qwen3.5:0.8b` for smoke tests and `qwen3.5`
for coding and tool use. Check the
[Ollama model library](https://ollama.com/library) for current model details.

Update the bundled Ollama installation with `update-ollama`, or request an
update at sandbox start:

```shell
openshell sandbox create --from ollama -e OLLAMA_UPDATE=1
```

## Option B: Host-level Ollama

Use this path when Ollama runs on the same machine as the OpenShell gateway and
selected sandboxes should share it.

### Start Ollama on a Reachable Address

```shell
curl -fsSL https://ollama.com/install.sh | sh
OLLAMA_HOST=0.0.0.0:11434 ollama serve
```

In another terminal, pull a model:

```shell
ollama pull qwen3.5:0.8b
```

### Import an Endpoint-bearing Profile

Save this as `ollama-openai.yaml`:

```yaml
id: ollama-openai
display_name: Ollama
description: Host-level Ollama OpenAI-compatible API
category: inference
inference_capable: true
credentials: []
endpoints:
  - host: host.openshell.internal
    port: 11434
    protocol: rest
    access: read-write
    enforcement: enforce
binaries:
  - /usr/bin/curl
  - /usr/local/bin/curl
  - /usr/bin/python3
  - /usr/local/bin/python
  - /sandbox/.uv/python/**
  - /sandbox/.venv/**
```

```shell
openshell provider profile lint -f ollama-openai.yaml
openshell provider profile import -f ollama-openai.yaml
openshell provider create --name ollama --type ollama-openai
```

The provider has no secret because this Ollama server does not authenticate.
Its attachment still carries the endpoint and binary policy.

### Attach and Verify

```shell
openshell sandbox create \
  --name ollama-client \
  --provider ollama \
  -- \
  curl http://host.openshell.internal:11434/v1/chat/completions \
    --json '{
      "model":"qwen3.5:0.8b",
      "messages":[{"role":"user","content":"hello"}],
      "max_tokens":10
    }'
```

For an OpenAI SDK client, set its base URL to
`http://host.openshell.internal:11434/v1`, use any non-empty API key value the
SDK accepts, and select the real Ollama model in the request.

## Troubleshooting

* Bind host-level Ollama to `0.0.0.0`, not `127.0.0.1`.
* Use `host.openshell.internal`, not `localhost`, from a sandbox.
* Confirm attachment with `openshell sandbox provider list ollama-client`.
* Inspect the effective policy with `openshell policy get ollama-client --full`.
* Run `ollama ps` and `ollama pull <model>` when the model is unavailable.
* If the gateway is remote, the hostname refers to the remote gateway host,
  not your laptop. Use a shared service address or tunnel instead.

## Next Steps

* [Provider-backed Inference](/sandboxes/inference-routing)
* [Profiles](/providers/profiles)
* [Customize Sandbox Policies](/sandboxes/policies)