Mesh LLM: distributed AI computing on iroh

What happened

Mesh LLM (July 11, 2026) presents a decentralized approach to running large language models by pooling GPU resources across multiple machines over the iroh P2P network.

Context and impact

iroh is a QUIC-based P2P networking library. Mesh LLM aggregates GPU capacity from multiple computers and exposes it as a single localhost:9337/v1 OpenAI-compatible endpoint. For developers without access to powerful single-GPU servers, it offers an alternative to expensive cloud APIs.

Details

• Supports models from 500M to 235B parameters
• Three execution modes: local, route to a peer with the model loaded, or split across multiple nodes
• No central server — direct authenticated QUIC connections between nodes
• Open source, available on GitHub