← back
E-Values Evaluating the Values of AI: Sheila Gulati and Nischal Nadhamuni
Takeaway
Evals must become multifaceted, dynamic, and values-aware before agentic systems become fully self-sufficient, or we'll lose the ability to course-correct.
Summary
- Tola Capital's Sheila Gulati and Clarity (recently raised $70M Series B) CTO Nischal Nadhamuni argue agentic AI makes evals seminal—we're moving from trained datasets to self-taught self-sufficient models.
- Evaluation expanded from narrow (image classification) to broad multi-axis radar: capability, domain, values, safety, end-user awareness, across all modalities.
- Benchmark hacking and saturation are real—models that ace MMLU fail real-world Finance Bench questions; Microsoft Research's dynamic synthetic benchmarks (object moves, spatial reasoning) prevent memorization.
- Clarity uses layered evals on contract-automation flows; the talk frames evals as a values question because agents represent us individually and collectively.
evalsbenchmarksalignment
Original description
Join us for a thought-provoking session on the future of AI evaluations, led by Sheila Gulati, founder of Tola Capital, and Nischal Nadhamuni, cofounder and CTO of Klarity. We'll delve into the importance of rethinking and innovating evaluations at this pivotal time, highlight the limitations of current methods, and offer practical steps for the AI community. As AI continues to play a significant role in our society, the need to develop the right evaluation tools is more important than ever. Learn how to create smarter, fairer, and more effective AI systems that align with our collective values and societal goals. Sheila will cover the thematic trends of this industry and Nischal will show how his company Klarity applied them as they have grown to serve the largest companies in the world and achieved their recent $70M Series B! This is a unique opportunity to engage with leading experts and shape the future of AI—don't miss out! Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at https://www.ai.engineer/worldsfair/2024/schedule & join us at the AI Engineer World's Fair in 2025! Get your tickets today at https://ai.engineer/2025 About Sheila Sheila Gulati is the founder and managing director of Tola Capital, a venture capital firm that invests globally in artificial intelligence (AI) and enterprise software. For more than two decades, Sheila has been at the forefront of innovation in software and investing, and her leadership has resulted in some of the most impactful software platform businesses in the world and numerous successful investments and exits across a range of companies. At Tola, Sheila combines business acumen with her vision of emerging trends in enterprise software and a sense of partnership and candor to guide forward-thinking founders at every stage – from inception to scaling. Some of her notable portfolio investments include OSISoft (acquired by Schneider), hybris (acquired by SAP), and Clipchamp (acquired by Microsoft). Prior to founding Tola Capital in 2010, Sheila spent over a decade at Microsoft where, most recently, she led the company’s enterprise IT strategy and launched the company’s cloud platform, Microsoft Azure, now the company’s largest business at $100B in revenue (inclusive of cloud, database, and developer tools). Before that, Sheila had various leadership roles within the Microsoft Developer business unit and spent time on the Corporate Development team where she was responsible for acquisitions and investments. About Nischal Nischal Nadhamuni is the CTO and Co-Founder of Klarity, a company on a mission to automate document centric workflows. Prior to Klarity, he worked on a AI applications at Airware, Flipkart and Mass General Hospital. Nischal completed his undergrad at the Massachusetts Institute of Technology. After meeting Harvard Law student, Andrew Antos in an entrepreneurship class at MIT, an idea sparked between the two: automating document review would simplify workflows for millions of workers around the world. In 2017, the two formally launched Klarity.