The rise of Large Language Models (LLMs) and autonomous agents is reshaping computing. Traditional operating systems, designed for deterministic execution, struggle to efficiently support probabilistic AI workloads. The Model Native Operating System paradigm emerges as a fundamental solution: an OS co-designed with models and hardware, treating models as first-class citizens to unify probabilistic intelligence with deterministic control.

A model-native OS reimagines the core abstractions:

• Interaction: From GUI to multimodal natural interaction (voice, gesture).
• Interface: Declarative, LLM-friendly APIs that separate “what” from “how”.
• Execution: Heterogeneous resource management (CPU/GPU/NPU) for efficient inference.
• Intelligence: Inherent OS-level models enabling cross-app agents and system self-optimization.
• Security: Multi-dimensional protection for data, model behavior, and agent actions.

Traditional imperative APIs force LLMs to specify how to perform a task, while LLMs excel at deciding what to do. This mismatch leads to inefficiency and errors.

DMI (Declarative Model Interface) [EuroSys'26] adopts a policy-mechanism separation. It provides three core primitives: Access (deterministic navigation), State (direct state setting), and Observation (structured data return). Evaluated on OSWorld-W, DMI improves task success rate by 67% and reduces interaction steps by 43.5% compared to Microsoft UFO2.

Symphony APIs (under submission) propose a “semi-open” design: APIs encapsulate core capabilities while exposing controlled extension points. LLMs can extend API logic either statically (predefined callbacks) or dynamically (API calls back the LLM at runtime based on intermediate state). In robotic pick-and-place tasks, dynamic extension doubles success rate over full API rewriting (from 40% failure to 20%).

Manually evolving OS code is costly: analysis of Linux Ext4 shows 82.4% of commits are for bug fixing and maintenance, not new features.

SysSpec [FAST'26 Erik Riedel Best Paper Award] introduces a new paradigm: developers write formal specifications (functional, modularity, concurrency) instead of low-level C code. An AI agent then generates the implementation from these specs. SpecFS, a full file system built this way, passes xfstests with correctness comparable to human-written code. Adding complex Ext4 features (e.g., delayed allocation) via spec patches reduces write operations by 99.9% and improves development efficiency by 3-5x.

Deploying large models on resource-constrained devices requires model-system co-design.

• PowerInfer [SOSP'24]: Fast LLM serving on consumer-grade GPUs using sparse activation and neuron caching.
• PowerInfer-2 [Arxiv'24]: Extends to smartphones with dynamic neuron caching (hot/cold neurons) and neuron-cluster heterogeneous computing. Achieves 11 tokens/s for Mixtral 47B on a phone, 29x faster than llama.cpp.
• HeteroInfer [SOSP'25]: First mobile inference engine to fully utilize CPU, GPU, and NPU in parallel. Reaches >1000 tokens/s prefill and >50 tokens/s decoding on Snapdragon 8Gen3, 1.34–6.02x faster than SOTA.
• DiffKV / LeanKV [SOSP'25]: Unifies KV cache compression via heterogeneous quantization (Key high-precision, Value low-precision) and dynamic sparsity. Achieves 3–5x compression without accuracy loss and 1.9–2.5x throughput gain.

Multitasking on accelerators demands fine-grained preemption and scheduling.

• REEF [OSDI'22]: Enables microsecond-scale kernel preemption on GPUs by exploiting kernel idempotence. Kills and restores best-effort kernels for real-time tasks, with <2% latency overhead and up to 7.7x throughput gain.
• XSched [OSDI'25]: Provides a unified preemptive scheduling framework for diverse XPUs (GPU, NPU, FPGA). A multi-level hardware model adapts to both mature and emerging XPUs; evaluated on 10 different platforms.

Enabling autonomous agents on end devices requires OS support for:

• Cross-app data access with privacy controls.
• Task persistence and state recovery.
• Secure sandboxing and permission management.

Our Symphony API robotics case study demonstrates how runtime extensibility allows agents to adapt to dynamic environments (e.g., avoiding obstacles during movement).

2026

• [ASPLOS'26] History Doesn't Repeat Itself but Rollouts Rhyme: Accelerating Reinforcement Learning with HistoRL. Jingkai He et al.
• [ASPLOS'26] PipeWeaver: Addressing Data Dynamicity in Large Multimodal Model Training… Zhenliang Xue et al.
• [FAST'26] Sharpen the Spec, Cut the Code: A Case for Generative File System with SYSSPEC. Qingyuan Liu et al. (Best Paper)
• [FAST'26] SolidAttention: Low-Latency SSD-based Serving on Memory-Constrained PCs. Xinrui Zheng et al.
• [EuroSys'26] LLMFolder: Revisiting Constant Folding in Large Language Models. Gansen Hu et al.
• [EuroSys'26] KunServe: Parameter-centric Memory Management… Rongxin Cheng et al.

2025

• [SOSP'25] PhoenixOS: Concurrent OS-level GPU Checkpoint and Restore… Xingda Wei et al.
• [SOSP'25] Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference. Le Chen et al. (HeteroInfer)
• [SOSP'25] DiffKV: Differentiated Memory Management for Large Language Models… Yanqi Zhang et al.
• [OSDI'25] Preemptive Scheduling for Diverse XPUs using Multi-level Hardware Model. Weihang Shen et al. (XSched)
• [OSDI'25] BlitzScale: Fast and Live Large Model Autoscaling… Dingyan Zhang et al.
• [ASPLOS'25] PipeLLM: Fast and Confidential Large Language Model Services… Yifan Tan et al.

2024 & Earlier

• [SOSP'24] PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU. Yixin Song et al.
• [OSDI'22] Microsecond-scale Preemption for Concurrent GPU-accelerated DNN Inferences. Mingcong Han et al. (REEF)

Preprints

• PowerInfer-2 (Arxiv'24) // TurboSparse (Arxiv'24) // Symphony APIs // DMI (EuroSys'26 to appear)

We believe the model-native OS will become the foundational substrate for AGI, enabling on-device intelligence, self-evolving systems, and seamless human-AI interaction. Researchers and students interested in shaping this future are welcome to collaborate.