Get StartedTutorials

Use Local Inference with LM Studio

View as Markdown

This tutorial exposes an LM Studio server on the gateway host to selected sandboxes. The provider attachment grants access; the workload calls LM Studio’s OpenAI- or Anthropic-compatible endpoint directly.

Prerequisites

For a headless installation:

curl -fsSL https://lmstudio.ai/install.sh | bash

Start the daemon and server:

lms daemon up
lms server start --bind 0.0.0.0

Load a model:

lms get qwen/qwen3.5-2b
lms load qwen/qwen3.5-2b

Create the Provider Profile

Save this as lmstudio.yaml:

id: lmstudio
display_name: LM Studio
description: Host-level LM Studio compatible APIs
category: inference
inference_capable: true
credentials: []
endpoints:
  - host: host.openshell.internal
    port: 1234
    protocol: rest
    access: read-write
    enforcement: enforce
binaries:
  - /usr/bin/curl
  - /usr/local/bin/curl
  - /usr/bin/python3
  - /usr/local/bin/python
  - /sandbox/.uv/python/**
  - /sandbox/.venv/**
openshell provider profile lint -f lmstudio.yaml
openshell provider profile import -f lmstudio.yaml
openshell provider create --name lmstudio --type lmstudio

The profile intentionally has no credential. LM Studio does not require one by default, but the provider attachment still supplies its policy boundary.

Attach and Call the Native Endpoint

OpenAI-compatible request:

openshell sandbox create \
  --name lmstudio-client \
  --provider lmstudio \
  -- \
  curl http://host.openshell.internal:1234/v1/chat/completions \
    --json '{
      "model":"qwen/qwen3.5-2b",
      "messages":[{"role":"user","content":"hello"}],
      "max_tokens":10
    }'

Anthropic-compatible request:

openshell sandbox exec lmstudio-client -- \
  curl http://host.openshell.internal:1234/v1/messages \
    --json '{
      "model":"qwen/qwen3.5-2b",
      "messages":[{"role":"user","content":"hello"}],
      "max_tokens":10
    }'

Configure SDKs with the same native base URL. Some SDKs require a non-empty API key even when LM Studio ignores it; use a literal non-secret value such as unused. Keep the actual model ID in the client request.

Troubleshooting

  • Enable Serve on Local Network in the LM Studio Developer tab, or use lms server start --bind 0.0.0.0.
  • Use host.openshell.internal, not 127.0.0.1 or localhost, in the sandbox.
  • Confirm the model is loaded with lms ps.
  • Confirm the attachment with openshell sandbox provider list lmstudio-client.
  • Inspect endpoint and binary policy with openshell policy get lmstudio-client --full.
  • A remote gateway cannot reach an LM Studio process on your laptop without a tunnel or shared network path.

Next Steps