Direct answers about testing one prompt across Claude, GPT, and your local models — privately, on your own hardware.
A prompt debugger lets you see how a single prompt behaves across different language models, side by side, so you can find where it silently breaks, drifts, or produces inconsistent answers. Amagra is a local, self-hosted prompt debugger: it runs one prompt across Claude, GPT, and local Ollama models at once and scores how much they agree.
In Amagra, paste your prompt into the editor, select the models to compare, and run. Each model's output appears in its own card with latency and length, plus a divergence score (Aligned, Mixed, or Divergent) summarizing how much the answers agree. Programmatically, send a POST /debug/prompt request with your prompt and the list of models.
Amagra's divergence score quantifies how much the models' outputs actually agree, measured as the average pairwise word-set overlap across the successful responses. It is reported as a verdict (Aligned, Mixed, or Divergent) with an agreement percentage and word-count spread, so you can spot inconsistent prompt behavior at a glance instead of reading every output.
Yes. Amagra is self-hosted and runs entirely on your own hardware via Ollama, with no account and no internet required for local models. Cloud models (Claude, GPT, Gemini) work too if you add your own API key, but nothing is required to leave your machine.
Amagra is local-first and private by default. When you use local Ollama models, your prompts and data never leave your machine. Cloud models are opt-in: prompts are only sent to a provider like Anthropic or OpenAI if you choose a cloud model and supply your own API key.
LangSmith is a hosted, cloud-based tracing and evaluation platform. Promptfoo is open-source but CLI and config-file driven. Helicone is primarily an observability and logging layer. Amagra is a self-hosted GUI workspace that compares models on one prompt, scores their divergence, keeps persistent project memory, and runs fully private on your own hardware.
Local models via Ollama (the default), plus Anthropic Claude, OpenAI GPT, Google Gemini, and any OpenAI-compatible endpoint such as vLLM or LM Studio.
Yes. Self-hosting is free and MIT licensed, always. There is also a managed free tier (100 requests per day, no card required), with paid tiers for managed hosting and enterprise features.