A self-hosted GUI that runs one prompt across Claude, GPT, and your local models side by side — and scores how much they agree. Here's where it fits next to the tools you already know.
| Feature | Amagra | Promptfoo | LangSmith | Open WebUI |
|---|---|---|---|---|
| Runs locally | ✓ | ✓ | ✗ | ✓ |
| Visual GUI workspace | ✓ | CLI | ✓ | ✓ |
| Cross-model side-by-side | ✓ | Config | Partial | Partial |
| Divergence score | ✓ | ✗ | ✗ | ✗ |
| Pre-run prompt checks | ✓ | Asserts | ✗ | ✗ |
| Persistent project memory | ✓ | ✗ | Partial | Partial |
| Self-hosted | ✓ | ✓ | Limited | ✓ |
| License | MIT | Open source | Proprietary | Open source |
Feature availability reflects each tool's core, self-hostable offering at time of writing. Corrections welcome — open an issue.
LangSmith is a hosted, cloud-based tracing and evaluation platform built for observing production LLM apps. It is proprietary and your data lives on their servers. Amagra is the opposite posture: local-first, self-hosted, and MIT licensed. It runs entirely on your own hardware, and prompts only leave your machine if you deliberately pick a cloud model and supply your own key. If you want hosted production observability, LangSmith fits; if you want to debug prompts privately on your own machine, Amagra does.
Promptfoo is open source and local, but CLI- and config-file-driven — excellent for scripted regression tests in CI. Amagra covers the interactive half of the same job: paste a prompt, pick models, and read every answer side by side with a divergence score, no YAML required. Many teams use both — Promptfoo for automated assertions, Amagra for hands-on cross-model debugging.
Open WebUI is a polished self-hosted chat front-end for local models. Amagra is not a chat app: its job is to run one prompt across several models at once and quantify their agreement, then keep that work in persistent memory with every routing decision logged and replayable. Chat is for conversation; Amagra is for comparison and debugging.
The divergence score is the differentiator none of the others have: the average pairwise word-set overlap across every model's answer, reported as Aligned, Mixed, or Divergent. It turns "did these models actually agree?" from a manual read of five outputs into a single number — so a prompt that silently breaks on one model is visible at a glance.