Your Agent Can Now Train Models — Merve Noyan, Hugging Face

14.2K views · May 13, 2026 · 19:10 min · Watch on YouTube ↗

Takeaway

Hugging Face Hub plus open coding agents and inference routing let you spin up, fine-tune, and serve open models from chat with near-zero glue code.

Summary

Merve Noyan (HF open source) demos how HF Hub puts an AI engineer 'at your fingertips' — open models like GLM 5.1 now top SWE-Bench rankings vs closed peers.
Hub now ships a remote MCP server and skills so agents can train models from chat (e.g. 'train Qwen 3.5 on this dataset'); HF Inference Providers route queries across Groq, Cerebras, etc.
Local agents: Llama-CPP-baked 'llama agent' binary launches any HF model; OpenCode and Hermes agent recommended for memory management and Slack/WhatsApp integration.
New 'traces' dataset repo type stores Codex/Claude Code/OpenCode session traces parsed for inspection and downstream training.
GGUF and 'use this model' panel auto-show local runtimes (LM Studio, Jan, Ollama, llama.cpp) and hardware fit (e.g. Gemma 4 4-bit fits 24GB L4).

agentsopen-sourcehuggingface

Original description

Open-source models have caught up. GLM 5.1 is leading the Artificial Analysis intelligence index over closed models, and the gap is closing fast with each release cycle. The practical upside beyond benchmarks: full weight access means you can quantize, fine-tune, and deploy to edge devices or browsers without data leaving your infrastructure.

@MerveNoyan walks through the Hugging Face ecosystem built around this: inference providers that route to the fastest or cheapest option per model, benchmark datasets for filtering by SWE-bench or AIME scores directly on Hub, a traces repository type for storing and exploring agent sessions, and skills that plug into coding agents. The closer is a live demo where she asks Claude Code to fine-tune a vision-language model on a dataset by name. The agent calculates VRAM requirements, selects an instance, and kicks off the job. What used to be a day of napkin math is now a prompt.


Speaker info:
- https://x.com/mervenoyann
- https://www.linkedin.com/in/merve-noyan-28b1a113a/
- https://github.com/merveenoyan

Timestamps
0:00 Introduction to Open Agent Ecosystem
0:39 Importance of Open Source in Machine Learning
2:36 Hugging Face Hub overview
3:06 Agentic models and Vision-LMs
4:24 Benchmark datasets and model filtering
5:16 Inference providers and model routing
6:50 Local coding agents and tools
7:46 Hermes agents for memory management
9:20 Traces repository for agent sessions
10:22 Tips for finding and serving local models
12:07 Supercharging agents with Hugging Face skills
13:41 Live demonstration of agent-driven fine-tuning
14:41 Training vision models (object detection/segmentation)
15:00 Using Model Context Protocol (MCP) for agents
16:30 Case study: OCR processing for AI papers