← back
Lessons from building GenAI based applications — Juan Peredo
Original: Lessons from building GenAI based applications — Juan Peredo
Takeaway
Building GenAI apps adds a whole AI architecture (model, hosting, eval, guardrails) on top of your stack — and ongoing model churn means evaluation never stops.
Summary
- Juan Peredo (consultant) shares 18 months of lessons building GenAI apps — adding AI introduces a parallel architecture (model choice, hosting, eval, hallucination control).
- Hosting trade-offs: Ollama locally, Modal/SkyPilot for cloud cost optimization across providers; HF transformers for quick local experimentation.
- Validation toolbox: prompt engineering precision, secondary LLM guardrail classifiers (add latency/cost), RAG (only as good as the data), fine-tuning (expensive, can degrade quality).
- Concrete failure: an LLM answered '3*3=33' — chatbots are easy to build but hard to make correct.
- Model churn (new models every other week) forces continual eval of both your current model and incoming candidates.
llm-appsraghosting
Original description
Integrate GenAI into your applications to deliver value without hidden complexities or budget surprises. Join us for a fast-paced session packed with hard-won insights. We will discuss topics like model hosting dilemmas, cost surprises, and the critical role of validating outputs. We will also talk about strategies to minimize these challenges, including: - Chatbots to agents: Supercharge your apps with AI chatbots and agents. - Cost control: How AI impacts your development cost and tips on how to control them. - Pro moves: Prompt management, observability, and evaluations. Perfect for managers, engineers, and builders: walk away with strategies to ship AI apps faster, cheaper, and smarter. Cut through the hype—let’s talk results.