SubQ: First Sub-Quadratic LLM with 12M-Token Context (2026)

On May 5, 2026, a 13-person Miami startup called Subquadratic came out of stealth with $29 million in seed funding and a claim that could fundamentally change how frontier language models scale. Its model, SubQ, is described as the first fully sub-quadratic frontier LLM — a system that processes context at linear compute cost rather than the quadratic cost every transformer has paid since 2017. The practical headline: a 12-million-token context window at research scale, and a production API that handles 1 million tokens at approximately 1/300th the cost of Claude Opus for equivalent retrieval accuracy on standard long-context benchmarks. If the numbers hold under independent scrutiny, SubQ represents the most significant architectural shift in LLM infrastructure since FlashAttention. This guide covers the Subquadratic Sparse Attention (SSA) architecture, all published benchmark numbers, the developer products in beta today, and the researcher skepticism worth weighing before committing production workloads.

The Attention Tax: AI’s Hidden Infrastructure Ceiling

Every transformer-based language model pays the same tax: attention compute scales with the square of context length. Process a sequence of 1,000 tokens and attention requires roughly one million operations per layer. Scale to 128,000 tokens — Claude Opus’s standard window — and that cost jumps by four orders of magnitude. This quadratic relationship is why larger context windows are almost always priced at a premium, why frontier labs charge $15–$30 per million tokens for long-context calls, and why well-resourced teams still think carefully before indexing a full codebase or legal repository into a single prompt.

The research community has attacked this ceiling from multiple directions. FlashAttention (2022) and FlashAttention-2 (2023) made quadratic attention dramatically more memory-efficient by fusing operations in GPU SRAM — but the compute complexity remained O(n²). Linear attention models like RWKV and Mamba-style state-space models (SSMs) traded some accuracy for compute efficiency. Mixture-of-Experts (MoE) reduced cost per token through selective routing. None of these approaches delivered a full frontier-quality model running at linear attention cost until SubQ’s claim last Tuesday.

Subquadratic Sparse Attention (SSA): How the Architecture Works

SSA replaces standard attention at the architectural level, not as a post-training optimization. The core mechanism is content-dependent token selection: for each query token, SSA learns to select a small, fixed-size subset of positions in the sequence that are actually relevant to that query, then computes exact attention only over those selected positions.

In standard self-attention, every query attends to every key-value pair. At one million tokens, that means one trillion attention operations per layer. SSA’s selection step reduces this to a fixed budget k of attended positions per query, making effective compute O(k × n) where k is a constant. Compute scales linearly with sequence length rather than with its square.

The engineering challenge is the selection step itself: ranking positions by relevance without first attending to all of them. Subquadratic describes a learned hierarchical scoring mechanism trained jointly with the language model that produces an approximate relevance ranking fast enough that selection overhead does not negate the attention savings. The company has not published a full technical paper, which is the main point of researcher pushback discussed below.

The architecture ships in two configurations. The research model supports the full 12-million-token context window referenced in launch materials. The production API exposes a 1-million-token window — still 5× larger than Claude Opus’s standard context and 8× larger than GPT-5.5’s default — at pricing designed to undercut the frontier market substantially.