Stanford Researchers Launch OpenJarvis: A Native-First Framework for Constructing On-Machine Private AI Brokers with Instruments, Reminiscence, and Studying

Stanford researchers have launched OpenJarvis, an open-source framework for constructing private AI brokers that run solely on-device. The undertaking comes from Stanford’s Scaling Intelligence Lab and is offered as each a analysis platform and deployment-ready infrastructure for local-first AI methods. Its focus will not be solely mannequin execution, but additionally the broader software program stack required to make on-device brokers usable, measurable, and adaptable over time.

Why OpenJarvis?

In accordance with the Stanford analysis workforce, most present private AI initiatives nonetheless preserve the native part comparatively skinny whereas routing core reasoning by way of exterior cloud APIs. That design introduces latency, recurring value, and information publicity considerations, particularly for assistants/brokers that function over private recordsdata, messages, and protracted person context. OpenJarvis is designed to shift that steadiness by making native execution the default and cloud utilization non-obligatory.

The analysis workforce ties this launch to its earlier Intelligence Per Watt analysis. In that work, they report that native language fashions and native accelerators can precisely serve 88.7% of single-turn chat and reasoning queries at interactive latencies, whereas intelligence effectivity improved 5.3× from 2023 to 2025. OpenJarvis is positioned because the software program layer that follows from that end result: if fashions and client {hardware} have gotten sensible for extra native workloads, then builders want a regular stack for constructing and evaluating these methods.

The 5-Primitives Structure

On the architectural degree, OpenJarvis is organized round 5 primitives: Intelligence, Engine, Brokers, Instruments & Reminiscence, and Studying. The analysis workforce describes these as composable abstractions that may be benchmarked, substituted, and optimized independently or used collectively as an built-in system. This issues as a result of native AI initiatives usually combine inference, orchestration, instruments, retrieval, and adaptation logic right into a single hard-to-reproduce utility. OpenJarvis as an alternative tries to present every layer a extra specific function.

Intelligence: The Mannequin Layer

The Intelligence primitive is the mannequin layer. It sits above a altering set of native mannequin households and offers a unified mannequin catalog so builders would not have to manually observe parameter counts, {hardware} match, or reminiscence tradeoffs for each launch. The objective is to make mannequin selection simpler to check individually from different elements of the system, such because the inference backend or agent logic.

Engine: The Inference Runtime

The Engine primitive is the inference runtime. It’s a widespread interface over backends comparable to Ollama, vLLM, SGLang, llama.cpp, and cloud APIs. The engine layer is framed extra broadly as hardware-aware execution, the place instructions comparable to jarvis init detect obtainable {hardware} and suggest an acceptable engine and mannequin configuration, whereas jarvis physician helps keep that setup. For builders, this is likely one of the extra sensible elements of the design: the framework doesn’t assume a single runtime, however treats inference as a pluggable layer.

Brokers: The Conduct Layer

The Brokers primitive is the habits layer. Stanford describes it because the half that turns mannequin functionality into structured motion below actual machine constraints comparable to bounded context home windows, restricted working reminiscence, and effectivity limits. Slightly than counting on one general-purpose agent, OpenJarvis helps composable roles. The Stanford article particularly mentions roles such because the Orchestrator, which breaks advanced duties into subtasks, and the Operative, which is meant as a light-weight executor for recurring private workflows. The docs additionally describe the agent harness as dealing with the system immediate, instruments, context, retry logic, and exit logic.

Instruments & Reminiscence: Grounding the Agent

The Instruments & Reminiscence primitive is the grounding layer. This primitive consists of help for MCP (Mannequin Context Protocol) for standardized instrument use, Google A2A for agent-to-agent communication, and semantic indexing for native retrieval over notes, paperwork, and papers. It additionally help for messaging platforms, webchat, and webhooks. It additionally covers a narrower instruments view that features net search, calculator entry, file I/O, code interpretation, retrieval, and exterior MCP servers. OpenJarvis is not only a neighborhood chat interface; it’s supposed to attach native fashions to instruments and protracted private context whereas preserving storage and management native by default.

Studying: Closed-Loop Enchancment

The fifth primitive, Studying, is what offers the framework a closed-loop enchancment path. Stanford researchers describe it as a layer that makes use of native interplay traces to synthesize coaching information, refine agent habits, and enhance mannequin choice over time. OpenJarvis helps optimization throughout 4 layers of the stack: mannequin weights, LM prompts, agentic logic, and the inference engine. Examples listed by the analysis workforce embrace SFT, GRPO, DPO, immediate optimization with DSPy, agent optimization with GEPA, and engine-level tuning comparable to quantization choice and batch scheduling.

Effectivity as a First-Class Metric

A significant technical level in OpenJarvis is its emphasis on efficiency-aware analysis. The framework treats vitality, FLOPs, latency, and greenback value as first-class constraints alongside job high quality. It additionally emphasizes on a hardware-agnostic telemetry system for profiling vitality on NVIDIA GPUs by way of NVML, AMD GPUs, and Apple Silicon by way of powermetrics, with 50 ms sampling intervals. The jarvis bench command is supposed to standardize benchmarking for latency, throughput, and vitality per question. That is essential as a result of native deployment will not be solely about whether or not a mannequin can reply a query, however whether or not it could possibly accomplish that inside actual limits on energy, reminiscence, and response time.

Developer Interfaces and Deployment Choices

From a developer perspective, OpenJarvis exposes a number of entry factors. The official docs present a browser app, a desktop app, a Python SDK, and a CLI. The browser-based interface might be launched with ./scripts/quickstart.sh, which installs dependencies, begins Ollama and a neighborhood mannequin, launches the backend and frontend, and opens the native UI. The desktop app is offered for macOS, Home windows, and Linux, with the backend nonetheless working on the person’s machine. The Python SDK exposes a Jarvis() object and strategies comparable to ask() and ask_full(), whereas the CLI consists of instructions like jarvis ask, jarvis serve, jarvis reminiscence index, and jarvis reminiscence search.

The docs additionally state that all core performance works with no community connection, whereas cloud APIs are non-obligatory. For dev groups constructing native purposes, one other sensible characteristic is jarvis serve, which begins a FastAPI server with SSE streaming and is described as a drop-in alternative for OpenAI purchasers. That lowers the migration value for builders who wish to prototype towards an API-shaped interface whereas nonetheless preserving inference native.

Take a look at Repo, Docs and Technical particulars. Additionally, be happy to observe us on Twitter and don’t overlook to hitch our 120k+ ML SubReddit and Subscribe to our Publication. Wait! are you on telegram? now you may be part of us on telegram as effectively.