SubQ: First Sub-Quadratic LLM with 12M-Token Context (2026)

On May 5, 2026, a 13-person Miami startup called Subquadratic came out of stealth with $29 million in seed funding and a claim that could fundamentally change how frontier language models scale. Its model, SubQ, is described as the first fully sub-quadratic frontier LLM — a system that processes context at linear compute cost rather than the quadratic cost every transformer has paid since 2017. The practical headline: a 12-million-token context window at research scale, and a production API that handles 1 million tokens at approximately 1/300th the cost of Claude Opus for equivalent retrieval accuracy on standard long-context benchmarks. If the numbers hold under independent scrutiny, SubQ represents the most significant architectural shift in LLM infrastructure since FlashAttention. This guide covers the Subquadratic Sparse Attention (SSA) architecture, all published benchmark numbers, the developer products in beta today, and the researcher skepticism worth weighing before committing production workloads.

The Attention Tax: AI’s Hidden Infrastructure Ceiling

Every transformer-based language model pays the same tax: attention compute scales with the square of context length. Process a sequence of 1,000 tokens and attention requires roughly one million operations per layer. Scale to 128,000 tokens — Claude Opus’s standard window — and that cost jumps by four orders of magnitude. This quadratic relationship is why larger context windows are almost always priced at a premium, why frontier labs charge $15–$30 per million tokens for long-context calls, and why well-resourced teams still think carefully before indexing a full codebase or legal repository into a single prompt.

The research community has attacked this ceiling from multiple directions. FlashAttention (2022) and FlashAttention-2 (2023) made quadratic attention dramatically more memory-efficient by fusing operations in GPU SRAM — but the compute complexity remained O(n²). Linear attention models like RWKV and Mamba-style state-space models (SSMs) traded some accuracy for compute efficiency. Mixture-of-Experts (MoE) reduced cost per token through selective routing. None of these approaches delivered a full frontier-quality model running at linear attention cost until SubQ’s claim last Tuesday.

Subquadratic Sparse Attention (SSA): How the Architecture Works

SSA replaces standard attention at the architectural level, not as a post-training optimization. The core mechanism is content-dependent token selection: for each query token, SSA learns to select a small, fixed-size subset of positions in the sequence that are actually relevant to that query, then computes exact attention only over those selected positions.

In standard self-attention, every query attends to every key-value pair. At one million tokens, that means one trillion attention operations per layer. SSA’s selection step reduces this to a fixed budget k of attended positions per query, making effective compute O(k × n) where k is a constant. Compute scales linearly with sequence length rather than with its square.

The engineering challenge is the selection step itself: ranking positions by relevance without first attending to all of them. Subquadratic describes a learned hierarchical scoring mechanism trained jointly with the language model that produces an approximate relevance ranking fast enough that selection overhead does not negate the attention savings. The company has not published a full technical paper, which is the main point of researcher pushback discussed below.

The architecture ships in two configurations. The research model supports the full 12-million-token context window referenced in launch materials. The production API exposes a 1-million-token window — still 5× larger than Claude Opus’s standard context and 8× larger than GPT-5.5’s default — at pricing designed to undercut the frontier market substantially.

Benchmark Claims at Launch

Subquadratic published the following performance numbers at the May 5 launch event:

• RULER 128K: 95.0% accuracy at a processing cost of $8 per evaluation run. Claude Opus achieves 94% on the same benchmark at approximately $2,600 — a 325× cost reduction for slightly higher accuracy.
• MRCR v2 at 1M tokens: 65.9% accuracy, which the company states beats OpenAI’s best published MRCR v2 result by 9 percentage points.
• SWE-Bench Verified: 81.8%, placing SubQ near the top tier of published coding agent benchmarks.
• Needle-in-a-haystack at 12M tokens: 92.1% retrieval accuracy across the full 12-million-token research window.
• Speed vs. FlashAttention: 52× faster throughput at 1 million tokens. The 1,000× efficiency figure referenced in launch materials compares specifically to dense attention without FlashAttention at 12M tokens — a meaningful distinction from the headline number.

These are entirely self-reported numbers. No independent third-party evaluation was published alongside the launch, which is unusual for a claim of this architectural significance. Understanding why matters before you architect anything on top of this API.

The Skepticism: What Researchers Are Questioning

VentureBeat’s May 6 report quoted several ML researchers questioning the efficiency claims directly. The concerns are specific and worth taking seriously:

• No arXiv preprint: For a claim that the fundamental architecture of transformers can be replaced while maintaining frontier accuracy, the research community expects a public methodology before accepting the numbers. Subquadratic has stated a paper is in preparation but has not published one.
• Benchmark selection bias: RULER and needle-in-a-haystack measure retrieval accuracy, which is not equivalent to full comprehension or reasoning across long contexts. A model that selects which tokens to attend to might excel at finding specific needles while degrading on tasks requiring understanding of diffuse relationships spread across millions of tokens.
• Selection overhead not accounted for: Fast approximate nearest-neighbor selection at 12M-token scale is itself a significant compute operation. Critics note that if selection latency is included in the full cost accounting, the 1,000× efficiency claim may not survive end-to-end measurement.
• The research-to-production gap: The 12-million-token context window is a “research configuration.” What developers can actually access today is 1 million tokens. The technical and reliability gap between a controlled research demo and a production-stable API at that scale is real, and the launch materials do not fully address it.

Subquadratic’s founder response — that independent benchmarks will follow and that the $29M seed round involved investors who performed technical due diligence — neither resolves the methodology questions nor is it meant to. The appropriate developer posture is systematic testing against your actual workloads, not deferred trust.

Developer Products Available in Beta

SubQ API

The production API exposes SubQ with a 1-million-token context window via a standard HTTP API using OpenAI-compatible endpoint structure. That compatibility is practically significant: it means SubQ is a drop-in replacement for most applications already built on OpenAI or Anthropic’s client libraries, with no SDK changes required. Pricing is consumption-based at approximately $1.50 per million input tokens at 1M-token scale, placing it well below the $15–$30 per million token range where Claude Opus and GPT-5.5 operate for long-context calls. The 12-million-token research window is on a waitlist for research partners and enterprise pilots. Sign up at subq.ai.

SubQ Code

SubQ Code is a CLI coding agent built on the SubQ model, designed to compete directly with Claude Code, Cursor, and GitHub Copilot Workspace. The differentiating feature is whole-codebase context: SubQ Code claims to ingest an entire large-scale repository — including every file, function definition, and git history — into a single context window without vector databases, chunking, or retrieval augmentation. For teams currently frustrated by the practical 200K-token limits of competing coding agents, which force manual curation of which files the model sees during a session, this is the use case that most directly validates SSA’s value proposition. SubQ Code is in private beta; the waitlist opened on May 5.

SubQ Search

SubQ Search is a deep research tool using the 1M-token context to process large document corpora — SEC filings, research paper collections, legal discovery sets, technical documentation libraries — and generate structured research reports. It is positioned as a direct competitor to Perplexity Pro’s deep research mode and Google NotebookLM, with the differentiator being raw in-context analysis capacity rather than retrieval-augmented generation over a chunked index. For workflows where semantic chunking loses cross-document relationships, SubQ Search’s single-context approach may produce qualitatively different outputs.

Use Cases Worth Testing Right Now

Given open questions about independent verification, the right posture is early evaluation, not immediate production migration. Four workloads are worth prioritizing for direct A/B testing:

• Large codebase search and refactoring: If your repository exceeds 200K tokens and you currently use chunking or vector retrieval for AI-assisted analysis, SubQ Code’s whole-repo claim deserves a direct test against your actual codebase. Results will tell you more than any benchmark.
• Legal and compliance document analysis: Full contract review, regulatory filing analysis, and legal discovery spanning millions of words are the core use case SubQ was built for. Even at 1M tokens, this is 5× the practical window of most current frontier APIs. See also: our guide on AI for legal and compliance workflows.
• Scientific long-sequence data: Genomic sequences, climate model outputs, or time-series datasets that exceed current context limits in a single inference call are natural fits for the SSA architecture’s claimed strengths.
• Cost-sensitive retrieval applications: If your workload is dominated by long-context lookups — finding specific information across large corpora — the RULER cost numbers suggest SubQ’s price profile deserves a direct comparison at production volume before you renew frontier API contracts.

Four Milestones Before Committing Production Workloads

Track these four developments to calibrate your production timeline for SubQ:

1. arXiv preprint: The technical paper Subquadratic has committed to publishing. If the SSA architecture and training methodology survive peer review, this resolves the methodology objections definitively and is the single most important signal for production adoption decisions.
2. Independent benchmark runs: Third-party evaluation from LMSYS, Scale AI, or a comparable organization running MMLU, GPQA, extended-context reasoning, and full-task SWE-Bench against SubQ independently of the company. Self-reported benchmarks on launch day have a poor track record predicting production behavior.
3. 12M-token API availability: If the research configuration transitions to general API availability and demonstrates production-stable reliability, it validates the headline claim operationally rather than in controlled conditions.
4. SWE-Bench independent reproduction: The 81.8% SWE-Bench Verified figure is the most consequential claim for developer adoption. Independent reproduction of this number matters most for SubQ Code decisions, since it directly measures coding task performance rather than long-context retrieval.

The Bigger Picture: The Context Window Arms Race Just Changed Scale

SubQ’s launch comes one day after Google I/O 2026, where Google announced plans for a 2-million-token Gemini 3.1 Ultra context window — already the largest in general availability before SubQ’s 12M claim. Gemini 3.2 Flash surfaced in the Gemini app with 1M-token support weeks before. The pattern is clear: the frontier labs are in a context length war, and the cost of that war, under quadratic attention, falls entirely on the API consumer.

SubQ’s architectural bet is that the arms race is the wrong frame. Instead of building bigger quadratic attention systems, Subquadratic is claiming the quadratic tax itself can be eliminated. If SSA proves out, the competitive pressure it places on OpenAI, Anthropic, and Google is not just about context length — it is about infrastructure cost structure. A model that processes 1M tokens for $1.50 instead of $25 makes entire categories of applications economically viable that were not before: continuous monitoring of large codebases, real-time analysis of live document streams, always-on research assistants with whole-library context.

Whether SubQ specifically delivers at that scale in production — which remains unverified — the architectural direction it represents is inevitable. The quadratic attention ceiling has always been a temporary constraint, not a physical law. The developer action today is straightforward: run your actual workloads against the SubQ beta API, form a position from measured results, and watch for the preprint. Don’t build production-critical systems on unverified benchmarks, but don’t ignore a $8 vs. $2,600 cost ratio either.