Blog — OpenRouter Blog

AI Agent Regression Testing After a Prompt or Model Change

An agent's behavior can change when you edit a prompt, swap a model, change a tool schema, or change what retrieval returns. This guide covers the locked case set, the per-case behavioral contract, and how to run the same suite against two concrete model slugs through OpenRouter so the diff shows what the change did.

September 30, 2026

Building a Golden Eval Dataset from Production Traffic

A golden eval dataset is a curated set of production inputs with reviewed expected outputs, versioned in Git and run before every deploy. This guide covers the five steps to build one from live traffic and how to run the same set against many candidate models through one API.

September 30, 2026

How to Test Tool-Calling Accuracy in AI Agents

An agent can call the wrong tool, or call the right tool with the wrong arguments. This guide covers three ways to test each failure mode, a Python harness that grades both, and how to run the same test cases against several tool-capable models through OpenRouter.

September 30, 2026

Image-to-Video AI Models Compared: Cost, Resolution, and Control

If you already have the image a video should start from, the model choice comes down to what has to happen after that frame. This post compares the Veo 3.1, Seedance, Kling, and Grok Imagine Video lines on duration, resolution, first-frame and last-frame control, generated audio, reference inputs, and current per-second price, then shows how to submit an image-to-video job with the TypeScript SDK.

September 29, 2026

Best Embedding Models in 2026

An embedding model decides what your retrieval system can find. We shortlisted the embedding models in our catalog for English RAG, multilingual retrieval, code search, text-and-image retrieval, and low-cost indexing, sent live requests to each one, and recorded their prices, context windows, and default vector sizes so you can pick a candidate and test it on your own data.

September 23, 2026

How to Use Jev: Moderation with the Jev API in TypeScript

A method for putting Jev to work on a new problem, worked end to end on marketplace listing moderation: what stays in code, what Jev sees, how to phrase the questions, and how to turn its probabilities into publish, hold, or reject.

September 23, 2026

Batch API: half-price inference by bundling requests

Send a whole workload in one POST, collect the results within 24 hours, and typically pay half the per-token price. Across 230k+ batches that completed over our two week beta period, the median finished in 7 minutes.

September 22, 2026

Is Jev as Accurate as Frontier Models at Classification?

Claude Opus 5 leads OpenRouter's classification task ranking by spend. We sent the same 3,080 Banking77 utterances to it and to Jev 1.13 through the Decisions API. Opus scored 84.4% to Jev's 81.0%, and Jev answered in 175 ms at $0.11 per thousand requests against 2.3 seconds and $2.42 for Opus.

September 22, 2026

What Is Nemotron 3.5 Lightning

Nemotron 3.5 Lightning is NVIDIA's open-weight 30B mixture-of-experts model with about 3B active parameters per token, built for the high-volume execution calls in an agent run. This post covers what the architecture means, how the model compares with Nemotron 3 Ultra, what context and features each OpenRouter endpoint exposes, and how to call it with structured outputs.

September 22, 2026

Two Hours of Work That Takes a Week

For Descript, testing a new model was a couple of hours of work and a week of waiting. The team removed the waiting. Evaluations run in an hour or two now, and nobody has to ask an engineer for time. This post covers the queue, what replaced it, and what changed once evaluation ran several times a week.

September 21, 2026

View all posts RSS

AI Agent Regression Testing After a Prompt or Model Change

OpenRouter is Joining Stripe

Ori Eval: Find the Best Model for What You're Building

Surpassing Frontier Performance with Fusion

Building a Golden Eval Dataset from Production Traffic

How to Test Tool-Calling Accuracy in AI Agents

Image-to-Video AI Models Compared: Cost, Resolution, and Control

Best Embedding Models in 2026

How to Use Jev: Moderation with the Jev API in TypeScript

Batch API: half-price inference by bundling requests

Is Jev as Accurate as Frontier Models at Classification?

What Is Nemotron 3.5 Lightning

Two Hours of Work That Takes a Week