What problems fall with more intelligence?

A little over 3 years ago before joining DAT, I was working with a friend on some startup ideas. It was around the time of going from GPT 3.0 to GPT 3.5 a point where the model seemed to shift from simply being able to semantically respond to free text to being able to reason about real world objects. A lot of potential started coming into focus fast.

Despite all this it was hard to use AI for much beyond chat. Context windows were small, RAG was in its infancy, MCP was nearly two years away, and it seemed like getting AI better tools and context was going to be required for it to do more useful work. To do that I spent a lot of time trying to figure out how to get LLMs talking with APIs. Its level of determinism in handling structured data was a lot like its ability to render Will Smith eating spaghetti: vaguely recognizable as having done the thing and deeply unsettling in its failures.

It hallucinated a lot, often in subtle ways. The equivalent of a bit of randomness in choosing the right next word was a disaster when trying to interface with JSON, as that right next word might be the key in a dictionary object. Human language can handle a creative flourish, APIs cannot. Success looked like getting it right 9 out of 10 times. But one 9 of reliability on transactional processing is foobar; no system can survive on that. I remember thinking at the time the answer to this would be a framework, or protocol or combination of those things that would force LLMs into compliance.

Fast forward 3 years and I was wrong. The issue was largely fixed by more intelligence. True, MCP came into being. But as time has gone on I would argue MCP is better suited for uniformity of tool exposure than for compliance. For the most part, these days agents handle DB schema, JSON objects and API interfaces just fine. They inspect the interface and implement it correctly. They reliably use the correct keys because they understand that exact matches are required for the interface to work.

These were hard stop issues three years ago, and they weren’t solved primarily by better engineering around AI, they were solved by better AI. So what happens in 2029? How many of today’s struggles are solved by three more years of improvement in base intelligence? Here are some that I think could fall:

1) Long horizon coherence

This gets better with each release but after a few hours of work of an AI working on its own things begin to fall apart. A million-token context window? Load it up! But imagine asking someone to read ten volumes of mathematics before attempting a proof. Some of it is relevant; some offer approaches that look applicable but lead nowhere. Compare that with giving them exactly the mathematical background the proof requires. With context, more is not always better.

Today, my harness does a lot of that context filtering. This may not be needed in the future; a far more intelligent model will probably be capable of knowing what to remember, what to discard, when to revisit an assumption, and how to recognize that the work has drifted from its objective.

2) Self verification and prudence

In the same way that GPT 3.5 seemed to usher in reasoning about real-world objects, the current generation of frontier models seem to be starting to exhibit something like judgement. How do you know when you are right? How do you know that you are right? It’s one of the hardest human skills.

Right now with AI this is not a trust zone, it requires engineering. But the goal with prudence is not perfection. There is no such thing. Let’s say there is an arbitrary scale for wise judgement: my golden retriever is a 1 and the wisest human is a 10. GPT 3.5 was probably a 2. Current models may be a 4. Getting to a 6 or 7 probably means being able to make a much broader range of consequential business decisions unsupervised. Perfection is a fallacy with prudence, but incremental improvements vastly open up delegation.

3) Specification

A lot of the dark matter “glue” on human teams is simply understanding what was meant and what is really needed. With AI there are a lot of elaborate efforts to get very precise on the spec. I have agentic loops dedicated to specification management. A meaningful chunk of that scaffolding is for a model that can’t yet infer intent or notice when a request is underspecified. Intelligence solves this the way a senior colleague does: by asking the two right questions and then not needing the other twenty.

Looking Forward

We haven’t ever had a technology revolution built on rapidly expanding intelligence itself. In the past we had to learn to put the pieces together and engineer solutions to leverage new capabilities. The electricity didn’t get smarter; we got better at wiring. But today the “wiring” itself thinks! And it gets better at it by the month. The engineering question stops being “how do I make this work” and becomes “which of the problems I am facing today will still be relevant by the time they’re finished?”

Today we have to anticipate the ever expanding radius of more intelligent models. It makes for a lot of conjecture. Yet the stakes are real for anyone building today. Will we have a hyper realistic 8k video of Will Smith eating whatever he wants? Bet the farm. Which problems will we no longer need to engineer around? That’s the harder bet.

How are you betting on the state of artificial intelligence in 2029?