Claude plays Minecraft!

1.3K views · Feb 15, 2025 · 18:15 min · Watch on YouTube ↗

Takeaway

Minecraft is a great agent playground; Amazon Bedrock Agents with Return-of-Control let you wire LLM tool use into Mineflayer for emergent in-world behavior.

Summary

Demo builds 'Rocky', a Minecraft agent that takes chat commands, runs through Claude Haiku via Amazon Bedrock Agents, and acts in the world.
Architecture evolution: started with LangChain on Lambda + Cohere on SageMaker, moved to Amazon Bedrock Agents calling Claude Haiku with the Mineflayer bot framework running on ECS.
Bedrock Agents provide Return-of-Control, knowledge bases for RAG, guardrails, and traceable chain-of-thought — useful for tool dispatch like jump/move/find/dig.
Tools: jump, move_to_position, locate_player, dig (with size param), find entity — Rocky unexpectedly dug itself out of holes despite not being trained to.
Code is open source; demo runs Minecraft + Mineflayer locally, agent calls Bedrock over the internet.

minecraftbedrockagents

Original description

Not Chatbots again! Not today, thank you!

Imagine a world where AI does more than just chat—it thinks, solves, and acts. Enter the realm of Minecraft, where virtual landscapes become the perfect proving ground for the next generation of autonomous agents.

This deep dive session takes you on an adventure beyond the ordinary, showcasing how we've engineered an AI entity capable of reasoning, problem-solving, and executing tasks in Minecraft.

Leveraging the power of Amazon's Bedrock, Lambda, SNS, and a suite of AWS serverless technologies, paired with the latest Claude 3 model from Anthropic, we reveal the full potential of AI beyond the confines of chatbots.

Discover how these serverless solutions empower our AI to interact with, adapt to, and transform its environment in real-time, providing a glimpse into the future of autonomous agents in both virtual and real-world applications. Join us as we navigate the intricacies of serverless AI architecture, demonstrating its practical implications, block by block.

Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at https://www.ai.engineer/worldsfair/2024/schedule & join us at the AI Engineer World's Fair in 2025! Get your tickets today at https://ai.engineer/2025

About Derek
Derek is a Developer Advocate for Developer Relations at Amazon Web Services, focused on helping developers across Australia and New Zealand to build their applications on AWS. He is focused on engaging technical audiences like developer, community, and user groups to share the latest AWS cloud services in blog posts, social media, and public speaking. He has a special interest in cloud-native architecture and front-end / mobile development.

In prior roles he has worked primarily as a Software Developer with over 18 years’ experience designing, architecting and building complex solutions across a range of industries including health, telecommunications, insurance, finance and defence.

He is currently enjoying the adoption of agile development principles across the industry, that is finally recognizing that it’s critical to release early and frequently to maximize learning and validate assumptions.