As GenAI technologies continue to proliferate across diverse industries, achieving end-to-end observability from GenAI applications to AI models is essential for maintaining operational efficiency and reliability. This session introduces a practical approach to constructing a comprehensive GenAI observability stack, leveraging OpenTelemetry (OTel) as the foundational framework. Attendees will discover: ● The enhancements made to the OTel agent to facilitate zero-code instrumentation for GenAI frameworks such as Dify, LangChain, Spring AI, and etc. This enables seamless tracing and metrics collection for GenAI applications, including support for MCP protocols. ● Improvements to the OTel Python agent facilitating effective observation of large language model (LLM)-serving frameworks, including vLLM and SGlang, ensuring full-spectrum observability of AI model performance and interactions. ● How our AI Studio harnesses OpenTelemetry to efficiently monitor millions of queries per day during model serving. Attendees will gain actionable insights and strategies to apply OTel-based observability effectively across their entire GenAI stack, enhancing both application and model monitoring.
Huxing Zhang is a Staff Engineer of Alibaba Cloud working on observability. He is also member of Apache Software Foundation, PMC member of Apache Tomcat and Apache Dubbo. He speaks at ApacheCon, OTel Community Days, etc.
Minghui is a software engineer at Alibaba Cloud. He is specializing in APM's auto instrumentation tools for Java LLM applications, utilizing the OpenTelemetry standard to deliver ready-to-use solutions. He is also a PMC member of Spring AI Alibaba, actively contributing to the observability... Read More →
