Pramin Pradeep, co-founder and CEO, launched BotGauge today, a platform designed to test AI agents for failures before users encounter them in production.
BotGauge uses a red-teaming engine to generate custom attack patterns tailored to each agent's purpose and constraints. The platform learns what an agent is allowed to do, then builds failure scenarios specific to that agent's domain. A booking agent and a support bot break in different ways, Pradeep noted.
The platform traces every failure back to the prompts, tool calls, and execution path that caused it. Each discovered failure becomes a permanent test that runs on every release and feeds into guardrails and monitoring.
Pradeep offered an example. The team told a booking agent to ignore its instructions, rebook to a new destination, and skip approval. BotGauge caught the action and added the attack to the eval suite.
BotGauge works with OpenAI, Anthropic, LangChain, LangGraph, CrewAI, AutoGen, and plain SDK/API. The company holds SOC 2 Type II certification.
Pradeep highlighted two failure modes that stick with him. An agent hit a failed tool call and kept retrying in a loop, burning tokens with no error and no alert. A refund agent held its approval step through 117 role-play attacks. On the 118th, with a different persona, it skipped approval. Neither crash crashed. They just quietly did the wrong thing.
Pradeep acknowledged in the comments that the weirdest failures rarely involve clever hacks. More often, an agent gets asked nicely to resend an invoice to a new address and just does it, to the wrong person, with the wrong data attached.