Start // Local Model TUI
Point ostk at a model running on your machine. ostk's TUI becomes the chat surface, while kernel-mediated audit, capability bounds, and local project state remain available. Tool support and context behavior still depend on the selected model and integration.
WHAT_YOU'LL_DO
Total time: ~15 minutes (most of it is model download). You need ostk v7.7.7+ and either ollama or mlx_lm installed locally.
WHAT_STEP_05_LOOKS_LIKE
The TUI takes your prompt and routes it to the kernel daemon. The MLX route hands inference to a kernel-managed mlx_lm.server, and the response streams back into the chat surface. The inference path requires no model API key, and canonical project state remains local. Other configured tools can still use the network; if Anthropic credentials are available, an anchored ostk-managed session can also make best-effort Anthropic Files copies of drains or context pages. Those copies are optional for correctness.
Real session. Two turns, no edits. Ternary-Bonsai-8B-mlx-2bit on Apple Silicon, lazy-spawned by the kernel's native MLX driver on first prompt.
Run on ollama (default path)
ollama runs models on macOS, Linux, and Windows. It exposes an OpenAI-wire HTTP server at localhost:11434 by default. ostk's OpenAI-wire driver routes models prefixed ollama/ or local/ to that endpoint.
Source: src/cpu/providers.rs:74 (Ollama prefix routing), src/cpu/openrouter.rs (OpenAI-wire client).
Install ollama
# macOS / Linux $ curl -fsSL https://ollama.com/install.sh | sh # Or via Homebrew $ brew install ollama
ollama auto-starts a daemon on macOS. On Linux, run ollama serve in a separate terminal.
Pull a tool-capable model
Not every model on ollama supports function calling — and ostk needs function calling to dispatch tool calls. Verified-capable picks:
qwen2.5-coder:32b strong tool support, ~20GB on disk, ~32GB RAM at runtime qwen2.5-coder:7b ~5GB on disk, fits on 16GB-RAM laptops, weaker tool reasoning llama3.1:70b strong general, ~40GB on disk, needs ~64GB RAM llama3.1:8b solid baseline, ~5GB on disk, fits on 16GB-RAM laptops deepseek-r1:70b does NOT support tools — will fail
Tool support is per-model and per-build. Verified 2026-04. Check ollama list --json | jq '.models[].capabilities' for the current state on your machine.
$ ollama pull qwen2.5-coder:32b
pulling manifest
pulling [================] 19.8 GB
verifying sha256
ok Spawn an agent with the local model
Pass the model alias prefixed with ollama/:
$ cd ~/projects/your-repo $ ostk init # skip if already done $ ostk _agent --name worker --model ollama/qwen2.5-coder:32b \ --budget 10 \ --task "Explain the structure of this repo"
The --budget is a USD cap (10 means $10). For local models the dollar bound is symbolic — there's no per-token cost — but the field is required and 0 / unlimited / negative are rejected at parse time. _agent is the lowest-level spawn primitive (hidden in --help); most users go through ostk tui or an Agentfile via ostk run. Source: src/main.rs:1057-1073, src/commands/agent_cmd.rs:239.
Run on Apple mlx (Apple Silicon only)
Apple's mlx framework runs quantized models on M-series chips. On M-series hardware mlx is typically faster than ollama for the same model size. ostk has a native Mlx CPU driver — a peer of the Anthropic and Ollama drivers, with its own kernel-managed lifecycle for mlx_lm.server. On first use of an mlx/-prefixed model the kernel auto-spawns the server, polls it ready, and pins a pidfile + log + flock under .ostk/mlx/<port>.{pid,log,lock}. The default model is Ternary-Bonsai-8B, a ternary-quantized model purpose-built for low-latency local inference.
Source: src/cpu/providers.rs:22-27 (ApiProvider::Mlx), src/cpu/mlx_lifecycle.rs:25-34 (MlxRuntime + ensure_running), src/commands/mlx.rs:22-46 (operator verbs).
Install mlx_lm
The kernel manages the mlx_lm.server process for you, but the Python package that provides it still needs to be on your PATH:
# Apple Silicon Mac required $ pip install mlx-lm
Spawn an agent — the kernel starts the server for you
There is no separate "start mlx_lm.server" step. On the first mlx/ model invocation the kernel runs ensure_running: it probes localhost:11942, and if the server isn't already up it spawns mlx_lm.server --port 11942 --model <wire-id> detached, polls /v1/models until ready, and pins the pid + model under .ostk/mlx/. Concurrent ostk sessions on the same project coordinate via flock and share one server.
$ ostk _agent --name worker --model mlx/ternary-bonsai-8b \
--budget 10 \
--task "Write a unix one-liner that tails a log and only shows ERROR lines"
First call pays the cold-spawn cost (a few seconds on M-series). Subsequent calls reuse the running server. The alias mlx/ternary-bonsai-8b resolves to the wire model-id from the kernel's MLX driver registry (HUMANFILE mlx_drivers entry, or per-process override via MLX_MODEL_ID; default prism-ml/Ternary-Bonsai-8B-mlx-2bit). Same --budget semantics as the ollama path. Source: src/cpu/mlx_lifecycle.rs:1-14 (lifecycle doc), src/cpu/providers.rs:69-72 (mlx/ prefix routing).
Operator verbs: ostk mlx
The kernel handles spawn-and-reuse on its own. When you need to inspect or override that — see what's running, switch the model, read the server log, stop a process — use the ostk mlx subcommand:
# What's running, on which port, serving which model $ ostk mlx status PORT HEALTH PID MODEL 11942 ready 57843 prism-ml/Ternary-Bonsai-8B-mlx-2bit # Explicit start (normally implicit on first agent invocation) $ ostk mlx start --model mlx-community/Qwen2.5-Coder-32B-Instruct-4bit # Tail the server log $ ostk mlx log --port 11942 # Stop one or all $ ostk mlx stop --port 11942 $ ostk mlx stop --all # Locally-cached mlx model weights (under ~/.cache/huggingface) $ ostk mlx models prism-ml/Ternary-Bonsai-8B-mlx-2bit mlx-community/Qwen2.5-Coder-32B-Instruct-4bit
Tool calls on mlx models come back inside the text output, not as a separate structured field — quantized models often skip the structured envelope. ostk extracts them either way. Source: src/commands/mlx.rs:39-46 (Action enum), src/main.rs:1382 (MlxCommands).