Papers
10 open-access works on agent-web infrastructure: research and benchmark reports, a working paper, specifications, proposals, position papers, and a framework paper.
Labels describe the artifact type, not peer-review status. Read the research and evidence standards.
April 2026 · Artificial intelligence / Software engineering · Framework paper
The Retention LayerSelf-Evolving Agents as Compounding Competitive Advantage in Agent-Native Businesses
Presents the Adaptive Convergence Protocol (ACP), a two-layer architecture (versioned Resource Substrate + Self-Evolution loop) that transforms agents into self-improving systems. Models switching cost dynamics, automated customization economics, and a four-stage retention flywheel. Evaluates vertical fit across 7 industries and assesses 6 failure modes with mitigations.
March 2026 · Information retrieval / Artificial intelligence · Research report
The Semantic Object ModelA Token-Efficient Web Representation for AI Agents
Introduces SOM, a structured format that compiles supported web pages into semantic JSON for agent consumption. The paper records an early public-web evaluation; current retained benchmark evidence is linked below.
Evidence note: Historical paper measurements predate the current retained-artifact and denominator policy. The current claim registry permits scoped serialized-byte observations, not universal token or cost claims.
March 2026 · Artificial intelligence / Computers and society · Position paper
The Agentic WebRethinking Web Infrastructure for Machine Consumption
A position paper arguing that the web is entering a fourth state and proposing three infrastructure primitives: SOM, Agent Web Protocol, and cooperative content negotiation via robots.txt directives.
March 2026 · Networking / Software engineering · Protocol specification
Agent Web ProtocolA Purpose-Built Communication Protocol for AI Agent-Web Interaction
Deep technical specification of AWP, a protocol designed for AI agents interacting with web content. Covers all 7 MVP methods, intent-based interaction via semantic element targeting, SOM integration, WebAssembly skill extensibility, and a detailed comparison with CDP.
March 2026 · Computers and society / Information retrieval · Proposal
Cooperative Content Negotiation for the Agentic WebExtending robots.txt for AI Agents
Proposes SOM directives for robots.txt that let publishers offer structured semantic representations to AI agents instead of blocking them entirely. Covers the publisher-agent conflict, directive syntax, complementary signaling mechanisms, security considerations, and an adoption pathway.
March 2026 · Artificial intelligence / Computers and society · Research report
The Hidden TaxQuantifying Token Waste in Agent-Web Interaction
Estimates the annual economic cost of HTML presentation noise in agent workloads at $1B to $5B per year. Combines Cloudflare crawl volume data, HTTP Archive page sizes, WebTaskBench token measurements, and a survey of 10 agent frameworks.
Evidence note: The $1B to $5B figure is a modeled industry estimate, not observed spend or a customer result.
March 2026 · Information retrieval / Artificial intelligence · Benchmark report
Does Format Matter?Agent Task Performance Across Web Representations
Introduces WebTaskBench, a task-based benchmark that measures how page representations affect agent cost and speed. Reports token and latency results for HTML vs markdown vs SOM across GPT-4o and Claude Sonnet 4, and specifies the rubric framework for accuracy and hallucination evaluation in follow-up revisions.
March 2026 · Computers and society / Artificial intelligence · Research report
The Publisher's CalculusA Cost-Benefit Analysis of Serving Structured Representations to AI Agents
Presents a comprehensive cost-benefit framework for web publishers evaluating SOM adoption. Models four publisher strategies across three tiers (10K to 50M agent requests/month), finding that SOM-first serving reduces per-request infrastructure cost by 60 to 80% with break-even at approximately 50,000 to 170,000 agent requests per month.
Evidence note: The savings and break-even figures are modeled scenarios, not measured publisher or customer outcomes.
March 2026 · Artificial intelligence / Information retrieval · Working paper
Information Fidelity Under Semantic CompressionMeasuring Task Accuracy, Hallucination, and Grounding Across Web Representations for AI Agents
Defines an evaluation design for testing whether semantic compression preserves the information agents need for correct task completion. Introduces a web-agent hallucination taxonomy, a grounding verifiability score, and a planned evaluation across 150 tasks, 4 models, and 3 representations.
Evidence note: Figures labeled projected and final results pending in the paper are not empirical findings.
March 2026 · Software engineering / Artificial intelligence · Research report
Agent Compliance with robots.txt SOM DirectivesEmpirical Evidence of the Discovery Gap
Tests whether AI agent frameworks discover and use proposed SOM robots.txt directives. Six experiments across 5 frameworks, 3 parsers, 12 content negotiation scenarios, and 2 LLMs reveal a universal discovery gap: 0 of 5 frameworks check robots.txt, yet the server infrastructure works in 100% of tests and SOM achieves equal accuracy at 55% fewer tokens.
Evidence note: The discovery result applies to the five tested framework versions and documented test corpus; it is not a universal claim about all frameworks or later releases.