traceloop/openllmetry

OpenLLMetry is an open-source library for monitoring and tracing AI applications that use large language models. When you build something with an LLM provider like OpenAI or Anthropic, it can be hard to see what is happening inside: which prompts went out, what the model returned, how long it took, and where failures occurred. OpenLLMetry adds that visibility by automatically recording those interactions as structured trace data. It is built on top of OpenTelemetry, an industry-standard format for collecting observability data. Because it uses that standard, the traces it produces can be sent directly to monitoring platforms you may already use, including Datadog, Honeycomb, Grafana, New Relic, Splunk, and over twenty others. If you have OpenTelemetry set up in your application already, you can add OpenLLMetry's instrumentations to it without replacing anything. Setup is minimal. You install the SDK via pip, add two lines of Python code to your application, and tracing begins. The library then automatically captures calls to supported LLM providers, vector databases, and AI frameworks without requiring you to manually wrap each function call. The list of supported providers is broad: OpenAI, Anthropic, Google Gemini, Groq, Mistral, Cohere, Ollama, AWS Bedrock and SageMaker, Hugging Face, and others. On the vector database side, it covers Chroma, Pinecone, Qdrant, Weaviate, Milvus, and a few more. It also instruments popular agent and orchestration frameworks. A JavaScript and TypeScript version of the library is maintained in a separate repository. The project is built by Traceloop, a Y Combinator-backed company, and released under the Apache 2.0 license. The semantic conventions it defines for LLM tracing have been proposed as part of the broader OpenTelemetry standard.