← back
LLM Safeguards: Security Privacy Compliance Anti Hallucination: Daniel Whitenack
Takeaway
Production LLM safety requires a layered checklist covering hallucination, supply chain, server resilience, PII leakage, and prompt injection — not a single guardrail.
Summary
- Daniel Whitenack (Prediction Guard) walks through an enterprise LLM risk checklist: hallucinations (e.g. field-medic advisory apps), supply-chain vulnerabilities in Transformers/model assets, flaky model servers, PII leakage via prompts/logs, prompt injection.
- Argues against the naive 'just add RAG' fix — knowledge-base documents themselves can leak PII (e.g. doxxing employees in support contexts).
- Frames mitigation as layered: ground-truth retrieval, output validation, prompt-injection defense, PII detection in/out, scalable resilient model serving.
- Builds case for open-access LLM deployment given enterprise compliance trends; cautions that data scientists typically aren't infra/distributed-systems experts.
safetysecuritycompliance
Original description
Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at https://www.ai.engineer/worldsfair/2024/schedule & join us at the AI Engineer World's Fair in 2025! Get your tickets today at https://ai.engineer/2025 About Daniel Daniel Whitenack (aka Data Dan) is a Ph.D. trained data scientist and founder of Prediction Guard. He has more than ten years of experience developing and deploying machine learning models at scale, and he has built data teams at two startups and an international NGO with 4000+ staff. Daniel co-hosts the Practical AI podcast, has spoken at conferences around the world (ODSC, Applied Machine Learning Days, O’Reilly AI, QCon AI, GopherCon, KubeCon, and more), and occasionally teaches data science/analytics at Purdue University.