Tech built the most elaborate hiring process of any profession. But in our obsession with metrics, did we forget to measure the right things?
Picture a journalist interviewing for a new role. She brings her published work articles that can be read, critiqued, discussed. Her portfolio travels with her. Her track record speaks before she does. Now imagine that same journalist is handed a grammar puzzle and given 45 minutes to solve it, while her portfolio sits ignored on the table. This isn’t hypothetical. It’s exactly how software engineering interviews work today. A few top companies decided that data structures and algorithms (DSA) puzzles were a good proxy for intelligence — and the rest of the industry followed. Yet surveys from real engineers in top tech companies have shown these DSA algorithm questions have very little to do with the real job of software engineering.
The cost isn’t just “false negatives” losing talented engineers who didn’t grind LeetCode. The real cost is brain drain and time drain at an industry scale.
Community-driven estimates based on LeetCode usage suggest that engineers spend 5 million hours a month on coding puzzles. That’s 60 million hours a year of human potential diverted from building real products to solving artificial riddles.
The mismatch between what we test in interviews vs what we expect in the actual software engineering job always existed, AI makes it even more pronounced. Today, software engineering is shifting from writing code to building systems. In the age of AI, the job is no longer syntax; it is judgment. It’s the ability to navigate a codebase you didn’t write, debug a distributed failure, and audit AI-generated output for architectural coherence. None of these critical skills are tested in a 45-minute coding interview. The model is no longer just imperfect, it is misaligned with the job itself.
How Should Engineers Prepare Instead?
What if we created space for engineers to invest time in building visible proof of their skills?
Open-source contributions. Startups. Research. Hackathons. Mentorship. Teaching. These have historically been undervalued because they were difficult to evaluate consistently. But that constraint is disappearing. With AI, it is now possible to analyze repositories, contributions, and systems at scale. Even if the github repo code is AI-assisted, it still reflects a candidate’s real-world behavior: how they structure problems, make trade-offs, and iterate. Its quality can be measured and its impact based on forks and stars can be judged.
This is far closer to the actual job than solving isolated puzzles.
How Should Interviews Change in the AI Era?
With AI coding assistants writing boilerplate, catching syntax errors, and suggesting implementations, writing code from memory is increasingly not the job.
The interview needs to mirror the actual work. Give candidates a real, messy codebase and ask them to find a bug. Make code review a formal round, show a pull request with subtle issues and see what they catch. Ask them to build a minimal MVP with all tools available, including AI, and watch the decisions they make under ambiguity.
“Interviewers must stop expecting a perfect complete identical solution from each candidate. This requirement is what leads us to design puzzles as questions.
The signal is never whether they finish and show a certain output. It’s how they navigate, what questions they ask, where they look when stuck, what they cut when time is short. This reasoning is the job and it is what should be measured in an interview. “— Payal Fofadiya, previous founder of an AI agent interview company and currently an Engineering Manager working on an Agent Platform team in big tech.
The infrastructure problem
Most companies default to LeetCode not because they believe it works, but because it’s operationally easy to administer. Running a realistic debugging session requires prepared environments, senior engineering time, and consistent evaluation frameworks, resources most companies struggle to scale. Hiring often involves interviewing 20–30 candidates across multiple rounds for a single role, making efficiency a hard constraint.
This is the ‘Infrastructure Gap’ that is being bridged by a new category of interviews that encourage transparent use of AI tools. One of the San Francisco-based companies, Fulloop is developing AI-assisted interview infrastructure, structured technical environments that allow use of AI in a transparent manner, adaptive assessments, and AI based comparative evaluation frameworks that gives startups the setup they couldn’t build themselves. The goal isn’t to automate the interviewer out of the room. It’s to remove the operational friction that forces companies to run the wrong interview in the first place, and to help define what the right one looks like as the job continues to change.
The interview that reflects the work
Every other serious profession figured this out. You hire on the work. You judge the thinking. You look at what someone built and how they reasoned when the answer wasn’t obvious. The technical interview is the last holdout of a model that never made as much sense as we convinced ourselves it did.
Give candidates real problems. Let them use every tool available. Watch how they think, not whether they finish. A developer who has shipped a product has already proven more than a thousand algorithmic puzzles ever could. It’s time our interviews reflected the work.
Media Contact:
Company Name: Fulloop.ai
Website: https://www.fulloop.ai/
Contact Person: Payal Fofadiya
Email: payal@fulloop.ai
Country: United States
