Engineering
How we learned to stop worrying and love the AI (in coding interviews)
April 8, 2025

How we learned to stop worrying and love the AI (in coding interviews)

John Wang
Co-Founder and CTO
JUMP TO SECTION

I remember my 5th grade math teacher told me I needed to memorize square roots because “you won't always have a calculator with you".

Technology changes quickly, and banning AI from technical interviews in 2025 feels eerily similar to banning calculators from math tests in the 90s. Ironically, many well known AI companies are still preventing candidates from using AI in technical assessments (see Anthropic for example).

At Assembled, we actively encourage AI tools in certain technical interviews. Candidates have really enjoyed these interviews as they offer an authentic preview of our work environment. It also provides us with much clearer signals about a candidate’s day-to-day engineering work.

To be clear, we haven't abandoned traditional interviews entirely — we still have dedicated sessions focusing on core CS fundamentals without AI assistance. But we believe a balanced engineer needs both skills: the ability to understand concepts deeply and the ability to leverage modern tools effectively.

The reality of modern engineering

Our engineers don't write most of their code from scratch anymore. We use Cursor, Claude Code, GitHub Copilot, and other AI tools for a significant portion of our day-to-day coding. This isn't a dirty secret, it's just how modern engineering works.

I’ve personally seen my productivity skyrocket since incorporating these tools. They’ve sped up everything from mundane tasks like writing tests to more complex but boilerplate-y work, like distributing WebSocket connections across servers.

A few years ago, we evaluated candidates on their ability to search Google or use an IDE effectively. We’ve now done the same with today’s generation of tools, which just so happen to be largely powered by AI.

New types of interviews

Many of our past questions that prioritized coding fundamentals can be one-shotted with AI (which makes for a pretty boring interview experience). When we first introduced AI into our interviews, a candidate used ChatGPT to solve our standard algorithm challenge in under three minutes, leaving us with 40 minutes of awkward small talk.

We’ve redesigned our interviews to better reflect the qualities we care about. Instead of FizzBuzz-type challenges, we now ask candidates to solve a scoped problem end to end, by building the backend and frontend. These problems have open ended design requirements—there's no right or wrong answer to most of them.

Finding great engineers

When you join Assembled’s engineering team, we don’t really care if you can implement Quicksort from memory or if you can balance a red-black tree without documentation: we care about whether you can write maintainable, well-tested code that solves real user problems.

After running dozens of AI-assisted interviews, we've identified a few patterns that distinguish exceptional engineers. The strongest candidates:

  • Have used AI tools extensively to write better, faster code. One candidate impressed us by using AI to set up a complex data structure, but then customized the critical sections with their own optimizations. Another used AI to summarize unfamiliar API documentation, allowing them to focus on the architecture decisions in the interview.
  • Actually understand the problem before building. This sounds obvious, but you'd be shocked how many people jump to implementation without understanding what needs to be done. A recent standout candidate asked a handful of piercingly thoughtful questions about use cases and then quickly incorporated that nuanced understanding into the end product.
  • Apply critical judgment to code, regardless of source. Strong engineers don't blindly trust AI output but rather methodically evaluate, debug, and verify the generated solutions. One impressive candidate identified subtle bugs in an AI-generated algorithm that even our interviewers initially missed.
  • Make good product decisions. The best candidates aren’t satisfied with code that merely meets the criteria, they’re generally thinking about what elevates the experience: from engineering specific things like performance, fault tolerance, and scalability to more general usability concerns like how a user will interact with their design. As AI handles more of the implementation details, engineers will make more product decisions than ever before. Having good "taste", i.e. knowing where to invest your effort for maximum impact and which details matter for user experience, is a crucial skill we're evaluating.

On the other end of the spectrum, weaker candidates reveal consistent patterns that limit their effectiveness with AI tools. They often treat AI as a magic solution generator by feeding in prompts and accepting whatever comes back without much scrutiny. We’ve also seen candidates lack the technical judgment to evaluate AI-generated solutions or get stuck on bugs that AI couldn’t solve out of the box.

Raising the bar

We've significantly increased our expectations for what candidates should accomplish during the interviews that allow AI. Where we once might have been satisfied with a basic backend implementation, we now expect candidates to build a relatively complete frontend, backend, and make significant progress on edge cases and optimizations within the same time frame.

Understanding computer science fundamentals remains important, but the emphasis has shifted from "can you implement this from scratch" to "can you guide AI and then recognize whether an implementation is correct and efficient." This mirrors the reality of modern engineering, where prompting and evaluating code often outweighs writing it from scratch.

At Assembled, we believe technical interviews should evaluate how candidates will perform in the actual job — not in an artificial environment with arbitrary constraints. Our engineers use AI tools in their daily work, and our interview process reflects that reality.

If you're interested in a technical role at Assembled where we're building AI systems to transform customer support, check out our open positions.