I built a fintech AI agent for discussing financial performance of publicly traded stocks using natural language. I have data covering ~1,500 public companies from scraping I’d already been doing for years.
The AI agent is built with LangChain, Ollama, Python, and DeepEval. It has
- input guardrails
- tool-calls, limits, and tool guardrails
- llm-judge output guardrails
- context summarization
- sets of single-turn and multi-turn evals
- persisted chat histories
LLMs are incredible, but they can still fail at the simplest reasoning tasks even when the facts are given to them in a very minimal context history.
Using Meta’s open-weight Llama 8b model, it says $306B is greater than $391B. So is LLM superintelligence less realistic than a Zuck vs Musk cage fight? And will Zuck challenge Amodei to a cage match next? I’d buy tickets to either of those.
Adding output guardrails helps to ensure output is grounded, backed by evidence and facts provided by tools. It improves reliability, so be sure to include output guardrails to catch these LLM mistakes. Using a stronger model improves reliability as well. After switching from Llama 8b to Qwen 8b, the evals performed noticeably better. The agent model can be switched to Qwen, Deepseek, Mistral, or about a dozen others instantly through configuration.
I’ve built reatime chat systems in the past using SocketIO, Postgres, and React and plan to deploy my agent to run interactively in a web app as well.