AI is changing. It now creates complex, step-by-step AI agents. But there is a big problem: AI can be unpredictable. Unlike normal code that works the same way every time, AI agents using LLMs can act in many different ways.
LangWatch is a new, free tool. It helps fix this. It gives AI agents a standard way to be tested. This includes checking their actions, running tests, and watching them. LangWatch helps make AI development more organized and based on facts, not just guessing.
Test AI Agents First for Better Results
If you use tools like LangGraph or CrewAI, you want to know why your AI agent makes mistakes. LangWatch lets you run full tests. These tests go beyond just checking what goes in and what comes out.
By running complete tests, you can see how different parts of the AI work together:
- The Agent: This is the main AI logic and how it uses tools.
- The User Simulator: This fake user tests different needs and tricky situations.
- The Judge: This AI checks the agent's choices. It follows rules to see if the agent did well.
This setup helps you find the exact moment an AI failed. It could be a step in talking or using a tool. This means you can fix problems before your AI goes live.
Improve AI by Checking Its Work
It is hard to move data from checking tools to training data for AI. LangWatch makes this easier. It has one place called Optimization Studio for this. This helps improve your AI.
Improve AI Step by Step
LangWatch helps you change and improve your AI instructions step by step. This happens in a clear cycle:
| Step | What to Do |
| Trace | Record everything the AI does. See how its state changes and what it outputs. |
| Dataset | Turn specific actions, especially wrong ones, into saved test cases. |
| Evaluate | Run automatic tests on the saved cases. See how well the AI works and if it is safe. |
| Optimize | Use the Optimization Studio. Change AI instructions and settings to make it better. |
| Re-test | Check if the changes fixed the problem. Make sure no new problems happened. |
This system makes sure that every change you make to AI instructions is based on real data. It is not just a guess.
Built on Open Standards, Works with Many Tools
LangWatch does not lock you into one company's tools. It is built using OpenTelemetry (OTel). OTel is a standard way to send data. This means LangWatch can work with your current systems. You do not need special code from us.
LangWatch works with today's top AI tools:
- AI Group Tools: LangChain, LangGraph, CrewAI, Vercel AI SDK, Mastra, and Google AI SDK.
- AI Model Companies: OpenAI, Anthropic, Azure, AWS, Groq, and Ollama.
Because LangWatch works with many tools, you can easily switch AI models. For example, you can move from GPT-4o to a local Llama 3 model using Ollama. Your AI testing tools will still work the same way.
Manage AI Instructions Like Code with Git
A very useful feature for developers is the direct link to GitHub. Often, AI instructions are treated like simple settings. This makes it hard to track changes. LangWatch connects AI instruction versions to the tests they ran.
This allows you to work like this with Git:
- Keep your AI instructions in your code files.
- Tag the AI tests in LangWatch with the code version used.
- See how changes in your code affected your AI's performance by comparing tests from different versions.
Ready for Businesses: Easy Setup and Security
If your company has rules about where data must be kept, LangWatch can be set up on your own servers. You can run it with one command using Docker Compose. This keeps your AI test data and private information safe within your company's network.
Key features for businesses:
- ISO 27001 Certified: This shows a high level of security. It is good for industries with strict rules.
- Supports Model Context Protocol (MCP): This helps your AI work better with Claude Desktop for more complex information.
- Add Notes and Use Queues: Experts can add notes to label special cases. This helps combine automatic tests with human checks.
Summary
Moving AI from a test idea to a working product needs the same careful work as making normal software. LangWatch provides the tools to check AI workflows. It helps test AI systems in a big way.