← back
New York Times' Connections: A Case Study on NLP in Word Games — Shafik Quoraishee, NYT Games
Original: New York Times' Connections: A Case Study on NLP in Word Games — Shafik Quoraishee, NYT Games
Takeaway
Connections is a reproducible NLP benchmark for abstract reasoning where graph-coloring formulations outperform raw semantic similarity.
Summary
- NYT Games engineer Shafik analyzes Connections (16 words into 4 groups of 4) as an AI benchmark — yellow easiest, purple has decoy-overlap that trips LLMs.
- Independent (not internal NYT) research — Connections puzzles are human-made and game mechanics intentionally test abstract reasoning, decoys, and Kahneman System 1/2 thinking.
- Random-guess analytics: ~0% chance to win cold, 1/5000-1/6000 after one category solved, ~1/35 after two — humans rely on intuition + deliberate reasoning.
- Models Connections as a graph-coloring problem — words are vertices, edges weighted by semantic similarity strength, with 4 colors for the 4 categories.
- Notes pure semantic similarity is insufficient; needs additional structure to handle decoys reliably.
nlpbenchmarksword-games
Original description
This session will examine the interplay between human intuition and artificial intelligence in puzzle-solving, using the popular New York Times Connections game as a practical case study.
We'll investigate how gameplay can be systematically evaluated through AI algorithms, exploring machine learning strategies such as clustering, semantic mapping, and natural language processing.
Attendees will gain insights into building AI-driven puzzle solvers, learn methods for quantitatively assessing gameplay complexity, and discuss the potential impacts of AI on puzzle game design and player engagement.
Timestamps [00:00]:
[01:45] Introduction to Connections
[03:50] Why Connections is Interesting for AI
[05:18] Human vs. AI Problem Solving
[06:55] AI Analysis and Methodology
[09:26] Semantic Similarity
[10:45] Relational Alignment
[12:16] Multi-dimensional Analysis
[15:44] Graph Neural Networks (GNNs)
[17:42] Motivation and Next Steps