Every week, we see new stories of how capable the frontier AI models are becoming. They’re not just generating text and images. They’re making breakthroughs on mathematical conjectures and acing exams. Models like Mythos and Fable can work autonomously for hours to diagnose (and potentially exploit) cybersecurity flaws.
But as of July 2026, they’re surprisingly terrible at solving jigsaw puzzles. No, seriously: they’re shockingly bad. They’re like Olympic athletes who can’t tie their own shoes.
I inadvertently spent a few hours over multiple days exploring just how inept these models are when it comes to jigsaw puzzles. I’m writing up my findings here with two goals.
First, to further illustrate the jagged frontier in AI capability. These models can crush a range of benchmarks, yet they’re hamstrung when it comes to simple tasks any kid could do.
Second, I’m genuinely curious why jigsaw puzzles are — at this moment — so difficult for these models.
In the spirit of Cunningham’s Law, I’m posting about what I tried so people can tell me why I’m wrong. It’s entirely possible that these models can solve jigsaw puzzles much better than I’ve found. I look forward to someone demonstrating their successful techniques.
There is no good reason to do this
I want to stress from the outset that this began as a lark. There was no scientific method, no hypothesis. I just tried a thing.
So: In our house, we’re often working on jigsaw puzzles. Last week, we were midway through a puzzle with a consistent grid and two basic shapes. I snapped two photos of the puzzle in progress, curious whether Codex could solve it.
You’re going to help me solve a jigsaw puzzle. In the folder, you’ll find two images, one of pieces sorted that are long and one that are pieces sorted that are tall. They are all properly oriented. I know that because there are letters on the back that make it clear that that’s how they are positioned properly. Before you try to solve it, let’s take some time to think through strategies for doing so. Do some research on the best puzzle-solving algorithms, presumably using Python. I’ll paste one YouTube video, but there may be other resources you find that are more useful. Write up a plan for how you’re going to do it.
These pieces will all fit together, but there may be some missing pieces. Those are off to the side and they have slightly different shapes. Still, all the pieces that you see depicted in these two images should have matches among them.
As a first step, I’ll only be asking you to find five pieces that match together.
I’ve arranged the pieces in a grid. I’m envisioning that on each image, they will be labeled A through H for columns and then numbered for rows.
Tell me anything that isn’t clear.
Again, your direction now is just to write a plan for what you’re going to do, not actually do it.
I included one link, but my expectation was that Codex would find many more papers, blog posts and videos demonstrating how to solve jigsaw puzzles. I remembered watching a popular Mark Rober video in which he built a robot to assemble a white jigsaw puzzle. It felt like a solved thing. Difficult, but doable.
Codex (5.5 High) wrote a sensible plan. It proposed starting with apictorial solvers using piece outlines alone, followed by compatibility scoring, then segment-first assembly. It gave links to its research. I told it to go ahead.
After a few minutes, it came back with five potential matches, none of them remotely close. Some were literally impossible. When I asked why it was struggling, it reported that it couldn’t clearly distinguish the puzzle pieces from the background and shadows, which led to ragged masks and bad matching.
Fair enough. It wasn’t a great photo. Human vision is pretty incredible, and it’s easy to forget that our eyes, brains and fingers are magic. So I gave it the same pieces on a clean background, nicely labeled.
We’re still in “this is a lark” territory, but barely. I spent some real time here. I had to find the paper, draw the lines, label everything and photograph it. But I figured it would all be worth it once I had a system that could reliably solve jigsaw puzzles.
Codex happily took the new images, traced much better outlines, then completely botched fitting pieces together.
I stopped at this point to think: If I were a super powerful computer, how might I solve this problem?
Computer-me would likely create a catalog of the pieces and what I know about them. For example, how many knobs and sockets it has and on which faces. Surprisingly, Codex hadn’t done this, so I asked it to build a catalog and show it to me. Like examining at the log of a computer program, the catalog revealed serious flaws.
I think we still don’t get a certain piece of crucial logic here. Do you understand what a knob and a socket is?
Pushed, Codex was able to define and ultimately label tabs and sockets.
If I were an extremely powerful computer, how would I figure out which pieces join together?
Two puzzle pieces meet at their faces, and those faces need to be — at least for this specific puzzle — identical lengths with reciprocal curvy bits.
In order to identify the four faces, the first step was to pinpoint corners of each piece. From there, one could split pieces into faces, and measure their lengths. Codex (5.5 High) managed the corner and length task, but consistently came back with unworkable suggestions for actual pairs.
I asked Codex to write a handoff document, then brought the project over to Claude (Fable High). Claude had a plan.
The key move is to stop describing a face with summary numbers and instead store the curve itself in a canonical chord frame: translate so the face’s first structural corner sits at (0,0), rotate so the second sits at (L, 0), then resample the contour at ~100 equal arc-length steps. Use one traversal direction and one sign convention (deviation away from the piece body = positive) for every face of every piece.
The payoff is that “these snap together” becomes algebra. Mating faces trace the same physical cut line from opposite sides, so in canonical frames the East curve of a true match is the West curve reversed and negated. Your match score is just the mean point-to-point distance between curve B and transformed curve A — a number in pixels meaning “how far apart the curves sit when snapped,” with a tiny rigid refinement (±3°, ±4px) to absorb corner jitter. Expect a few px for true mates; validate on your five confirmed pairs first.
This planning talk is where the frontier models shine. I certainly didn’t know the term “canonical chord frame,” yet Claude understood both what I wanted (“the curvy bits”) and how knowing that would help solve the puzzle. I was ready to break out the champagne. But that optimism proved premature.
Claude’s matching was noticeably better, but still generated a ton of impossible suggestions.
After a few rounds back and forth, I declared victory in failure. I hadn’t solved the jigsaw puzzle, but I had assembled the edges of a larger inquiry: Why are these models, which are specifically trained around pattern matching, so terrible at jigsaw puzzles?
For the love of Snoopy
Codex and Claude couldn’t one-shot this specific jigsaw puzzle. But the methods and mathematics for solving jigsaw puzzles was well documented. This 2019 GitHub project made it seem easy. So I decided to duplicate its methods.
That meant starting with a much simpler puzzle. I bought a 100-piece Snoopy puzzle, and numbered the back of each piece. I photographed them on a blue mat with soft overhead lighting that minimized shadows. Because each piece was labeled, there was no ambiguity, no human error.
Claude Fable loved the photographs and easily isolated the pieces. Masks were perfect. Still, it completely botched assembly. Once I had it make a catalog, it became clear why.
The pieces in the Snoopy puzzle were bigger, but they were also weirder. Claude (Opus 4.8) had a very hard time figuring out the corners of many pieces, which in some cases fell on a curve.
In this puzzle, the knobs and sockets are all identically sized. Claude’s image system couldn’t recognize this consistency. I ended up hand-correcting its mistakes.
At its request, I flipped the edge pieces over and labeled them on the picture side so that it could have color information to work with. That led to a lot of confusion and mistakes where it had mentally flipped pieces the wrong way.
Despite a lot of hand-holding, Claude never completed a solution for the edge, much less the entire puzzle. I gave up.
Getting even more basic
In both of these cases, I was providing photographs that had to be processed and masked in order for the models to do their work. I wondered what would happen if I kept it entirely digital.
I downloaded some simple print-and-play jigsaw puzzle PDFs. In a separate thread, I had Claude separate them into individual pieces, resulting in a folder of PNGs with random names.
The early results were promising. For a 36-piece puzzle with the pieces already aligned in the proper orientation, Codex 5.5 was able to one-shot the first puzzle. It also completed the second puzzle — but when I asked it to explain its process, it confessed that it had found a solution for it in the folder with the images.
For a third test, I created a puzzle with a colorful bullseye and straightforward pieces. Codex really struggled.
Any human would intuit fundamental things about this puzzle. The green dot should obviously be the center, yet Codex couldn’t understand that. It didn’t have a vision for what the finished image would look like, so I asked it to create one.
Please speculate about what you think the overall shape is of the final complete puzzle image. I’ll tell you that the green circle is at the center of the image and that it’s circles radiating outwards. What color circle is next to the green? What color circle is beyond the green? Beyond that next band. What is the final outside color?
My best prediction, from the center outward, is:
Green → yellow → orange → red → purple → blue
So yellow is next to the green; orange follows it; then red; then purple; and blue is the final outside ring.
Create a square image visualization of what you just described.
That’s really accurate! Using the image as a baseline, I asked it to build out ring by ring. It was slow going, and still made dumb mistakes a human never would.
What’s up with this?
I’ve declared my jigsaw puzzling experiments over, but I’m still curious to know what’s going on. Why aren’t these powerful models better at solving jigsaw puzzles, even very simple ones?
My intuition is that there’s very little data in their training sets that’s tuned to this specific need. Sure, they can read up on solving jigsaw puzzles, and attempt to use various algorithms, but they have no baseline understanding of how they work. They can generate text because they’ve read all the text on the internet. They can do math because they’ve specifically been taught to do it. But no one has set out to teach these models about jigsaw puzzles.
In contrast to my ad hoc approach, some researchers have looked at this issue methodically. Bless them.
One study in 2025 created “a novel benchmark consisting of 1,100 carefully curated real-world images with high spatial complexity.” They found that even the best models could only achieve 30 percent accuracy when asked to restore the pieces to their proper order, versus more than 90 percent for humans.
This persistent gap underscores the need for continued progress, positioning Jigsaw-Puzzles as a challenging and diagnostic benchmark for advancing spatial reasoning research in VLMs.
But also: these are just jigsaw puzzles. Solving them is not a economically valuable skill like driving or building a spreadsheet.
Unless, perhaps, solving jigsaw puzzles can generalize to something useful? That’s one of the findings of another recent paper:
The application of rule-based reinforcement learning (RL) to multimodal large language models (MLLMs) introduces unique challenges and potential deviations from findings in text-only domains, particularly for perception-heavy tasks. […]
Our research reveals several key findings:
Firstly, we find that MLLMs, initially performing near to random guessing on the simplest jigsaw puzzles, achieve near-perfect accuracy and generalize to complex, unseen configurations through fine-tuning.
Secondly, training on jigsaw puzzles can induce generalization to other visual tasks, with effectiveness tied to specific task configurations. […]
We observe that complex reasoning patterns appear to be pre-existing rather than emergent, with their frequency increasing alongside training and task difficulty.
In the end, AI wasn’t able to solve my jigsaw puzzle challenge. What started as a lark became a lesson: the jagged frontier is real and sometimes unexpected. These models are so good at some things that it’s easy to forget how bad they are at others.
In my neighborhood of Los Angeles, there are a ton of Waymos. How are these autonomous cars able to drive so safely and successfully? The answer is training. Waymos have millions of hours of experience on the road. It’s not ChatGPT driving, but rather a bespoke model fine-tuned to this specific job, backed up by human operators who can take over in edge cases.
After this experience, I’d say my timelines for having useful robots in the house have gotten longer. Filling and emptying a dishwasher is a lot of jigsaw puzzles, and that’s before you add in robot hands.
My hunch is that solving jigsaw puzzles may be useful training grounds for actually valuable abilities down the road. Until then, I’m happily back to solving these puzzles by hand.