Talk · Santa Clara University · 2026
Beyond What AI Can Do
Why what AI can't do matters more than ever — a talk I gave to students at Santa Clara.
I was invited to Santa Clara to talk about what it's actually like to build AI for a living. I could have given the usual demo reel. Instead I wanted to make one argument, and make it honestly: the interesting frontier is no longer what these systems cando — that curve is taking care of itself. The interesting frontier is what they still can't, because that's where all the leverage, and all the careers, are about to concentrate.
To get there, though, you have to actually understand the machine. So I started with the least mystical possible description of it.
What an LLM actually is
Strip away the marketing and a language model is a remarkably simple shape. Words become numbers. Those numbers flow through a very large neural network. Out the other side comes a probability distribution over the next token. That's it. It was trained on exactly one job — predict the next token — and everything else, reasoning, conversation, code, translation, emerges from that single objective carried out at enormous scale.
The other thing worth internalizing is that everything becomes vectors. Text, images, audio, video — different modalities collapse into the same underlying language: points in a shared space where similar meanings cluster together. “A golden retriever,” a photo of one, the sound of a bark — pushed close together by training. Once you see that, a lot of the magic turns into geometry.
How we got here, and the Bitter Lesson
Modern AI has three eras, each defined by what stopped being the bottleneck. Before 2012 it was classical ML — you hand-crafted the features and told the model what to look at. From 2012 the deep-learning era let networks learn their own features, and it ended with the 2017 Transformer paper, Attention Is All You Need— arguably the most important paper in the field. Since then we've been in the era of scale, where you turn three knobs — more data, more compute, more parameters — and capability stops being incremental.
This is Rich Sutton's Bitter Lesson: across seventy years of AI research, the general methods that leverage computation win, over and over, against the clever hand-built ones. Past a certain scale you get emergence— qualitatively new abilities that simply weren't present in smaller models. It's bitter because it keeps humbling our cleverness. It's a lesson because it keeps being true.
How a model gets shaped — and where I work
A model is made in two acts. Pre-training reads the internet and produces raw capability with zero alignment. Post-training gives it shape: supervised fine-tuning teaches tasks by demonstration, and reinforcement learning teaches preferences through reward.
RL isn't mystical — you've trained a dog. Say “sit.” The dog tries something. A treat for sitting, nothing for the rest. Its behavior shifts toward what got rewarded. Same loop in an LLM: it drafts an answer, something scores that answer — a reward model, a unit test, a human — and its weights shift toward what scored well.
Which brings me to my actual job. In 2024, OpenAI's o1 showed that RL can train deep reasoning — when the reward is verifiable. And that word splits the whole world in two.
Verifiable
There's a clear right answer.
Code — do the tests pass?
Math — does the proof check out?
Tools — did the call return?
Subjective
Taste. Judgment. Context.
Is this poem any good?
Is this the right strategy?
Is this argument honest?
How do you build a reward signal for tasks that have no answer key?
That's the frontier I work on — at Google on Search AI Mode and AI Overviews, and in my own research. There are two broad ways to steer a model toward doing more. You can change the weights — my work on agentic RL, like ARPO and its successor AEPO, which branch sampling at the model's most uncertain moments to get better tool use for half the budget. Or you can change what surrounds the model — the arc from prompt engineering to context engineering to harness engineering: skills, tools, retrieved memory, everything you fit into a million-token window. Both answer “how do we make AI do more?” The rest of the talk was about the opposite question: what should it never do, even when it can?
My job didn't disappear. It changed shape.
Software engineering used to mean deterministic systems: same input, same output, test it once and ship it. Bugs were reproducible; behavior was specified; the compiler was the only thing arguing with you. Now I build probabilistic systems — same input, adistributionof outputs, with the model as a new and moody dependency. You stop designing for correctness and start designing for variance: eval harnesses, guardrails, fallbacks, prompt versioning, trace logging. AI engineering is software engineering for systems that don't behave the same way twice.
Students always ask: “so do you still code?” Constantly — just not the way you might think. I direct the model and review the diff instead of typing every line. I still read stack traces myself, because AI can't debug what it can't reproduce. I still whiteboard architecture with humans. The skill that compounds was never typing speed. It's knowing what good code looks like, so you can tell the moment the AI hands you bad code.
A hot take about “coding is dead”
“AI replaces engineers” gets the shape exactly backwards. What's really happening is a capability inversion around a moving line.
And the demo-to-production gap is where this bites hardest. A demo is the easy 80% — one user, the happy path, localhost, failures that are silent or even charming. Production is the other 20%: millions of users, weird inputs, race conditions, failures that cost money or trust or lives. Vibe coding is generative — it produces something plausible. Production is directional— each step has to land in a specific place, not merely a plausible one. If you can't read the code, you can't fix it, and that's exactly where careers get made.
There's also a piece of math that people underestimate:
Every agent handoff is a multiplication, not an addition. Four agents that are each individually reliable chain down to a coin flip. This is the real reason long autonomous workflows are so much harder than the demos suggest — and it's why the honest work is in the evaluation and the guardrails, not the generation.
So what's left for you?
The things AI can imitate but not actually do:
- Taste— knowing what's worth making, not just what's possible to make.
- Judgment— calling the model's output wrong, and trusting yourself to override the consensus.
- Direction — choosing the problem. The model can answer; only you can ask the right question.
- Deep thinking — sitting with a hard problem long enough that something new actually appears.
And this isn't a tech-only conversation. In finance, models read 10-Ks faster than you and the edge becomes judgment under uncertainty. In consulting, slide-making and frameworks are free but client trust isn't. In medicine and law, recall gets commoditized and diagnosis under ambiguity doesn't. In every field the floor rises, the bar rises faster, and the work that compounds is the work AI can't grade.
Step back, way back
I ended by zooming out as far as I could. Life on Earth is the longest-running reinforcement learning game we know of. The reward is survive and replicate; the mechanism is variation and selection. Single cells, then multicellular life, then nervous systems, then us — each level unlocked a new kind of intelligence. We may be the first level that builds the next one on purpose.
Which reframes what's scarce. Access used to be the bottleneck — top universities, expensive textbooks, patient mentors, being in the right room. That's mostly gone. A kid with a $50 phone has what an elite university had in 2005: any question answered any time, a tutor that never tires, papers turned into podcasts, remarkable free tiers. If access was the old privilege, what's the new one?
The answer is curiosity.The bar to think deeply is lower than it has ever been. The reward for doing it is higher than it has ever been. In an age of cheap generation, discernment is the rarest skill — and it's the one thing no model can want on your behalf.