===== Model Native Operating System =====

The rise of Large Language Models (LLMs) and autonomous agents is reshaping computing. Traditional operating systems, designed for deterministic execution, struggle to efficiently support probabilistic AI workloads. The **Model Native Operating System** paradigm emerges as a fundamental solution: an OS co-designed with models and hardware, treating models as first-class citizens to unify probabilistic intelligence with deterministic control.

{{ :model_native_os_arch.png?direct&600 |Figure 1: Model Native Operating System Architecture}}

===== Vision =====

A model-native OS reimagines the core abstractions:
  * **Interaction**: From GUI to multimodal natural interaction (voice, gesture).
  * **Interface**: Declarative, LLM-friendly APIs that separate "what" from "how".
  * **Execution**: Heterogeneous resource management (CPU/GPU/NPU) for efficient inference.
  * **Intelligence**: Inherent OS-level models enabling cross-app agents and system self-optimization.
  * **Security**: Multi-dimensional protection for data, model behavior, and agent actions.

===== Key Technical Pillars =====

==== LLM-Friendly Interfaces ====

Traditional imperative APIs force LLMs to specify **how** to perform a task, while LLMs excel at deciding **what** to do. This mismatch leads to inefficiency and errors.

**DMI (Declarative Model Interface)** [EuroSys'26] adopts a //policy-mechanism separation//. It provides three core primitives: ''Access'' (deterministic navigation), ''State'' (direct state setting), and ''Observation'' (structured data return). Evaluated on OSWorld-W, DMI improves task success rate by **67%** and reduces interaction steps by **43.5%** compared to Microsoft UFO2.

{{ :dmi_comparison.png?direct&600 |Figure 2: DMI eliminates multi-step GUI interaction}}

**Symphony APIs** (under submission) propose a "semi-open" design: APIs encapsulate core capabilities while exposing controlled extension points. LLMs can extend API logic either statically (predefined callbacks) or **dynamically** (API calls back the LLM at runtime based on intermediate state). In robotic pick-and-place tasks, dynamic extension doubles success rate over full API rewriting (from 40% failure to 20%).

{{ :symphony_extensions.png?direct&600 |Figure 3: Static vs. dynamic extension in Symphony APIs}}

==== OS Generation & Evolution ====

Manually evolving OS code is costly: analysis of Linux Ext4 shows **82.4%** of commits are for bug fixing and maintenance, not new features.

**SysSpec** [FAST'26 **Erik Riedel Best Paper Award**] introduces a new paradigm: developers write **formal specifications** (functional, modularity, concurrency) instead of low-level C code. An AI agent then generates the implementation from these specs. **SpecFS**, a full file system built this way, passes xfstests with correctness comparable to human-written code. Adding complex Ext4 features (e.g., delayed allocation) via spec patches reduces write operations by **99.9%** and improves development efficiency by **3-5x**.

{{ :sysspec_flow.png?direct&600 |Figure 4: SysSpec toolchain and evolution}}

==== On-Device LLM Inference ====

Deploying large models on resource-constrained devices requires model-system co-design.

  * **PowerInfer** [SOSP'24]: Fast LLM serving on consumer-grade GPUs using sparse activation and neuron caching.
  * **PowerInfer-2** [Arxiv'24]: Extends to smartphones with **dynamic neuron caching** (hot/cold neurons) and **neuron-cluster heterogeneous computing**. Achieves **11 tokens/s** for Mixtral 47B on a phone, **29x** faster than llama.cpp.
  * **HeteroInfer** [SOSP'25]: First mobile inference engine to fully utilize CPU, GPU, and NPU in parallel. Reaches **>1000 tokens/s** prefill and **>50 tokens/s** decoding on Snapdragon 8Gen3, **1.34–6.02x** faster than SOTA.
  * **DiffKV / LeanKV** [SOSP'25]: Unifies KV cache compression via heterogeneous quantization (Key high-precision, Value low-precision) and dynamic sparsity. Achieves **3–5x** compression without accuracy loss and **1.9–2.5x** throughput gain.

{{ :powerinfer2.png?direct&600 |Figure 5: PowerInfer-2 neuron caching and heterogeneous execution}}
{{ :heteroinfer_soc.png?direct&600 |Figure 6: HeteroInfer parallelizing across CPU/GPU/NPU}}

==== Dynamic GPU/XPU Sharing ====

Multitasking on accelerators demands fine-grained preemption and scheduling.

  * **REEF** [OSDI'22]: Enables **microsecond-scale kernel preemption** on GPUs by exploiting kernel idempotence. Kills and restores best-effort kernels for real-time tasks, with <2% latency overhead and up to **7.7x** throughput gain.
  * **XSched** [OSDI'25]: Provides a unified preemptive scheduling framework for **diverse XPUs** (GPU, NPU, FPGA). A multi-level hardware model adapts to both mature and emerging XPUs; evaluated on 10 different platforms.

{{ :reef_preemption.png?direct&600 |Figure 7: REEF microsecond-scale GPU preemption}}
{{ :xsched_abstraction.png?direct&600 |Figure 8: XSched unified abstraction for diverse XPUs}}

==== On-Device Agents ====

Enabling autonomous agents on end devices requires OS support for:
  * Cross-app data access with privacy controls.
  * Task persistence and state recovery.
  * Secure sandboxing and permission management.

Our Symphony API robotics case study demonstrates how runtime extensibility allows agents to adapt to dynamic environments (e.g., avoiding obstacles during movement).

===== Selected Publications =====

**2026**
  * [ASPLOS'26] History Doesn't Repeat Itself but Rollouts Rhyme: Accelerating Reinforcement Learning with HistoRL. Jingkai He et al.
  * [ASPLOS'26] PipeWeaver: Addressing Data Dynamicity in Large Multimodal Model Training... Zhenliang Xue et al.
  * [FAST'26] Sharpen the Spec, Cut the Code: A Case for Generative File System with SYSSPEC. Qingyuan Liu et al. (Best Paper)
  * [FAST'26] SolidAttention: Low-Latency SSD-based Serving on Memory-Constrained PCs. Xinrui Zheng et al.
  * [EuroSys'26] LLMFolder: Revisiting Constant Folding in Large Language Models. Gansen Hu et al.
  * [EuroSys'26] KunServe: Parameter-centric Memory Management... Rongxin Cheng et al.

**2025**
  * [SOSP'25] PhoenixOS: Concurrent OS-level GPU Checkpoint and Restore... Xingda Wei et al.
  * [SOSP'25] Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference. Le Chen et al. (HeteroInfer)
  * [SOSP'25] DiffKV: Differentiated Memory Management for Large Language Models... Yanqi Zhang et al.
  * [OSDI'25] Preemptive Scheduling for Diverse XPUs using Multi-level Hardware Model. Weihang Shen et al. (XSched)
  * [OSDI'25] BlitzScale: Fast and Live Large Model Autoscaling... Dingyan Zhang et al.
  * [ASPLOS'25] PipeLLM: Fast and Confidential Large Language Model Services... Yifan Tan et al.

**2024 & Earlier**
  * [SOSP'24] PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU. Yixin Song et al.
  * [OSDI'22] Microsecond-scale Preemption for Concurrent GPU-accelerated DNN Inferences. Mingcong Han et al. (REEF)

**Preprints**
  * PowerInfer-2 (Arxiv'24) // TurboSparse (Arxiv'24) // Symphony APIs // DMI (EuroSys'26 to appear)


===== Prospects =====
We believe the model-native OS will become the foundational substrate for AGI, enabling on-device intelligence, self-evolving systems, and seamless human-AI interaction. Researchers and students interested in shaping this future are welcome to collaborate.