My thesis is that the true measure of reading code is still the ability to fix and extend it. So my current ideal interview problem is still a tiny toy codebase, where the interviewee is tasked with adding some (relatively trivial) feature to it. Like ten lines of code, but where those lines require you to have grokked the other couple hundred or so. Any downsides?
One thing I like about this (as opposed to a “blank slate” coding challenge) is that it shows how well the candidate is able to fit into the broader design style of the surrounding code. Not just the superficial things (like naming conventions), but also the structural patterns.
Back when I was a hiring manager for programmers I liked setting a "code challenge" where we gave the candidate a simple game and he had to transform it into another game.
For example, we gave you a tetris game, you had to change the code to make a snake game. Or change an Othello game to a "number sliding puzzle" game.
It was very interesting comparing people that really made an effort to use the game code vs candidates that just deleted most of the stuff to implement their logic.
>It was very interesting comparing people that really made an effort to use the game code vs candidates that just deleted most of the stuff to implement their logic.
What does "very interesting" mean here? What does this particular comparison tell you? It's genuinely unclear to me which you prefer, why, and what signal you get beyond willingness to follow an instruction in a peculiar context.
Your comment came up as a bit aggressive for me (non native so I may have misinterpreted) but I'll try to answer in the spirit of HN:
We had candidates that were 'frustrated' because of the code. Some were frustrated that the code was low-quality, others that it used Canvas for drawing and they have never used canvas. Others about the way the logic was implemented originally.
The best candidates worked with what they had, and used the "tools" at their disposal: for example, use the code that draws a circle in the canvas already, dont try to reinvent it. At the end of the 3 hours, we talked about their experience doing the project and they had good feedback.
The bad candidates just deleted most of the code and added their own logic. My thought process is that, I could imagine them on their first day of in the team: the code startups have is never "the best one". I dont want someone who wants to throw the baby with the bathwater, but someone who can improve what we have.
A detail is that the challenge lasted 3 hours. Of those 3 hours , 1.5 hours wage candidate was "pair programming" with some developers (segments of 30 min spaced to give the candidate time to work alone). And during the pair programming , the devs were instructed to help him, give them hints and ask "revealing" questions.
I do think there is some signal based on how well the developer can utilise existing structures without getting stuck in "all you have is a hammer". That said the tradeoff is mostly time-based so if it is a take home without constraints a lot of that signal is lost.
> fit into the broader design style of the surrounding code[, including] the structural patterns.
That might require one to be familiar with those ahead of time. If that's what you're hiring for, that's perfect of course, but I personally haven't had much experience with design patterns outside of a few C# ones in school five years ago. Yet I see myself as a competent amateur programmer (and C# isn't even my strong suit), just not one that often works on large code bases. These patterns seem like something I'd learn in a matter of days on the job... but that would not show in an interview.
I read the "patterns" part as idioms of the codebase, e.g. what a route handler looks like if this is an HTTP server, or the standard way to make an entity in a game. I think that's one of the more critical parts of the test. Certainly not GoF style "Design Patterns™".
It's kind of like learning a subject from the final exam.
If you can pick up on the patterns of an unfamiliar language and/or codebase and make your code look like it fits in, you're probably good at both reading and writing code, and working with a team.
I can't speak for the parent commenter, but by "patterns" I am referring to multiple layers:
Detail -- Indentation, spacing, line length, naming convention, placement of braces and parentheses.
Flow -- pacing, grouping of blocks of code, levels of nesting, when and where to break up functions into spaced blocks or helper methods.
Architecture -- class boundaries, interfaces, cross-file organization.
There are higher layers but I haven't named them yet in mind and they aren't as well tested by this kind of interview.
Fair enough, if that is meant then I imagine any competent programmer should be able to do that. I kind of figured that indentation and nesting levels were what they considered to be "superficial things [like] naming conventions". Cross-file organisation might be too large to adequately demonstrate in an interview, presuming it's not a whole-afternoon type of thing that seems popular in the USA (or at least I haven't seen them in Europe, I'm sure they must exist if only I post a comment saying they're not popular here).
You do miss out on the 'sit down and create a project' aspect though. Which shows things like familiarity with project structure, (python specific) virtual environments and paths etc.
These are good skills to have when a dev is debugging as they could get tied up for a long time dealing with something as simple as a relative path issue.
The things you mention can easily be standardized in the company, and starting a new project or setting up a dev environment should be automated to the point that you don’t even have to think about it.
>should be automated to the point that you don’t even have to think about it.
heh I like it. I'm going to put you as an intermediate skill level dev.
So these things break and need to be fixed. They also have to be setup by someone. You also need to know when something breaks, can I rule out that it's not a setup issue, a git merge that's broken a path, a user that's pushed a local dev config file accidentally.
In order to be effective, you need to have a feel and justification for the way things are. Getting someone to setup a project correctly should be a quick and easy way to do this.
For example, I've had candidates submit tech tests with code in the top level directory instead of having it in a src or some other sub folder. Which is obviously a red flag.
Only one I can think of is that a couple hundred lines (depending on content of course) can be quite a bit to grok within a 50 minute span. Especially if there are a number of places where improvements might make sense, but you have one or two in mind specifically...
I've been in that sort of position tbh, where a 200 line codebase has maybe 20-30 obviously wrong parts (creds in code, calls to 3rd party service in code w/o cacheing, maybe that could just be a service? why is this not in a queue, ...) and it's sort of weird that getting 11/14 and talking over pros and cons of the others can be considered a barely-pass.
Still, you're not wrong. Find a good enough, real-life enough example and it'd make for a great question.
You would have to fine-tune the size, taking into account that narrowing your focus to the relevant parts of the codebase is an integral part of your reading skills.
Yup, this is what my current take home (2 hours) project is. Read about 200 lines of code, implement about 30 somewhat complex lines in a real working project. And then there's a code review by a "junior" candidate for them to read and review, filled with about 10 bugs in 50 lines. I much prefer it to giving out algorithms questions.
By doing that you will be selecting for the "cracker" type. That is people who are able to get into any code, even under adversity, and do anything they want. It is a great skill to have even if you job doesn't involve removing copy protection, for example, he may be the only person who will be able to fix a bug in that old code base written in an obscure language when the last person to work on it left the company 10 years ago.
What you won't be testing is higher level skills. It may be easy to hack into a known codebase, and even do it cleanly, you just have to mimic the surrounding code. Getting the big idea and making long term choices is another set of skills, not exclusive with being a cracker, but not something you can see in 10 lines of code.
I've been at both ends. Building apps as a solo developer, as well as fixing some bug or implementing a single feature correctly for clients. I think it requires being able to zoom in and zoom out along the stack of abstraction that constitutes a code base, as well as either focusing on the now or a much larger timeframe.
The higher level skills can be somewhat tested by asking the candidate how it would build some module and the variation that some factors would bring in (like closer deadline, no third party service,...)
Uh, no. The skill here is one used by literally any programmer who starts work on a project they didn't start. Not that there's zero overlap, but come on.
I think there’s quite a bit of a difference between doing this very quickly and only feeling comfortable with a codebase after a while. It’s a good skill to have, but it might make low impact in certain circumstances.
