ajay150313/agentsre-langchain

When you run an AI agent in production, standard monitoring tools tell you whether the server is up and whether API calls are succeeding. They do not tell you whether the agent is making good decisions. A payment-routing agent might return HTTP 200 on every request while quietly making wrong decisions 30 percent of the time. This library addresses that gap. agentsre-langchain is a Python package that adds four behavioral metrics to LangChain agents. The first is the Decision Quality Rate, the percentage of executions where the agent produces a high-confidence result. The second is Tool Invocation Efficiency, a ratio comparing actual tool calls made to a baseline, which catches agents that keep retrying the same tool unnecessarily. The third is the Human Escalation Rate, the percentage of runs where the agent fails and hands control back to a human. The fourth is Queue Depth Drift, a count of how many pending items are accumulating. Integrating the library requires wrapping an agent function with a decorator. The decorator captures each execution, extracts the confidence score and tool-call count from the agent output, calculates the four metrics, and stores them for retrieval. Configuration lets you set alert thresholds for each metric so you are notified when performance degrades. The README includes a table of recommended starting targets by environment. Development allows more slack (Decision Quality Rate above 75%, escalation rate below 10%). Production tightens those expectations (Decision Quality Rate above 92%, escalation rate below 2%). The project recommends running a 30-day observation window before committing to specific targets. The package installs via pip and optionally integrates with a companion service called agentsre that receives the metrics. Example scripts cover basic agents, multi-tool routing, ReAct-pattern agents, agents with conversation memory, and cost tracking.