← back
The Hidden Life of Embeddings: Linus Lee
Takeaway
Treating embedding space as a steerable canvas with discoverable semantic directions unlocks UI patterns prompts alone can't deliver.
Summary
- Prompting feels like 'steering a car from the back seat with a pool noodle' — looking inside latent spaces offers more direct control than tokens alone
- Demonstrates a custom encoder/decoder model where 248-dim embeddings can be decoded back to text, then perturbed (blur radius) to sample semantic neighborhoods
- Identified directions in embedding space that mean specific things (length, sentiment) and built spatial-canvas UIs where users drag-and-drop to vary text along those axes
- Projecting two pieces of text onto the model's negative-sentiment and length axes gives a quantitative readout of how the model sees them
- Mixing embeddings between two passages produces interpolated outputs that capture both styles — pointing to richer reading/writing interfaces than chat
embeddingslatent-spaceinterfaces
Original description
We love text embeddings as a critical pillar of LLM applications, but there's so much to text embeddings beyond their value in vector search. This talk will be a grand tour through a series of experimental projects from my last two years of research for visualizing, manipulating, and interpreting embeddings. We'll start with the basics (t-SNE, UMAP, and PCA), talk about how language models can be used to manipulate and interpret embeddings, and end by using a new tool I've built that lets us directly observe which features popular embedding models like to encode into their embeddings. Recorded live in San Francisco at the AI Engineer Summit 2023. See the full schedule of talks at https://ai.engineer/summit/schedule & join us at the AI Engineer World's Fair in 2024! Get your tickets today at https://ai.engineer/worlds-fair About Linus Lee Linus is a Research Engineer at Notion prototyping new software interfaces for augmenting our collaborative work and creativity with AI. He has spent the last few years experimenting with AI-augmented tools for thinking, like a canvas for exploring the latent space of neural networks and writing tools where ideas connect themselves. Before Notion, Linus spent a year as an independent researcher, during which he was Betaworks's first Researcher in Residence.