← back
OpenLLMetry is all you need
Takeaway
OpenLLMetry brings the OpenTelemetry standard and ecosystem to LLM apps so you get vendor-neutral, drop-in observability across providers, vector DBs and agent frameworks.
Summary
- Nir (CEO Traceloop) introduces OpenLLMetry, an extension of OpenTelemetry (CNCF's second-largest project after Kubernetes) for LLM observability.
- Reviews OTel primitives: logs (arbitrary events), metrics (aggregate counters like CPU, latency, plus GenAI-specific token usage and error rate), and traces (multi-step process tracking — essential for chains, workflows and agents).
- OpenLLMetry adds 40+ instrumentations covering foundation-model SDKs (OpenAI, Anthropic, Cohere, Gemini, Bedrock), vector DBs (Pinecone, Chroma) and frameworks (LangChain, LlamaIndex, CrewAI, Haystack), monkey-patching client libraries to emit OTel data automatically.
- Output flows into any backend supporting OpenTelemetry — Datadog, Sentry, New Relic, Grafana Tempo, Dynatrace, Honeycomb — so users avoid vendor lock-in; collectors can pre-filter PII before export.
- Example: the Pinecone instrumentation auto-emits queries, indexing, returned vectors, distances, scores and latencies in OTel format.
observabilityopentelemetrytracing
Original description
OpenLLMetry (https://github.com/traceloop/openllmetry) is an open-source project for tracing and monitoring GenAI agents and apps anywhere you want - whether it's Datadog, New Relic, Dynatrace, Grafana, or even Langsmith.