Every few days an article of this type is written and posted on HN. And all the comments are people either proposing another (same old) approach to hiring or people replying to those comments explaining why they'd hate that approach.
This is then done for every possible combination of approaches (homework questions, whiteboard interview, pair programming, fizzbuzz questions, algorithmic questions, work-history questions, career-progression questions, mini-project, magical incantations, and so on...)
It may sound cynical but at this point it feels like no one has any new ideas about it and the same content keeps getting recycled in the form of yet another article and yet another comments section.
Half of these articles are from company blogs which exist largely as just a marketing tool.
Since there is no actual new thought, everyone just cargo-cults the Google approach.
> Also, we chose to completely scrap whiteboard coding and assigned homework instead.
For an entry level engineer, this removes a bunch of variance and also because it gets you another scenario which happens way often in actual work.
You show up with the code and it gets reviewed & discussed on how/why of the implementation.
The most talented entry level people I've worked with have struggled with criticism instead of grading (i.e "grade the code" through a bunch of test-cases is different from "why did you do this and explain").
I was pretty nervous at my interview; no test was mentioned until the day, at which time (after the rest of the interview) I was presented with two sheets of xml-style configuration code for the company's proprietary build system.
The task was super simple - basically just duplicating some of the configuration and renaming some values to create another job, but at the time I just froze and couldn't see it. I managed to get some sort of answer out, but wasn't happy with my response. They claimed that they were meant to give this to me in advance as a homework question and somewhere the ball got dropped, but I had a feeling they wanted it to be a more on the spot thing considering the problem was so simple.
Thankfully I had the wherewithal to ask if I could take the problem home with me after the interview. When I got home and looked at the problem with fresh eyes I facepalmed so hard. I wrote up the correct answer and emailed it over to the company recruiter, hoping she would pass it along to my interviewers, but didn't get my hopes up.
I guess she did, since I ended up getting the job. I still cringe a bit about how I just blanked on that stupid question. Thankfully I got ramped up really quickly once I actually started the job and had no problem figuring things out after that.
This is generally fine but “homework” is a terrible idea (trend?) because it isn’t looking from the perspective of the new hire:
1. Unemployment can be extremely stressful and there is a very good chance the person needs a job NOW. Every little thing you do to dilly-dally instead of making an offer is another hurdle that may send the applicant elsewhere. Act as soon as possible when you see a résumé you like (get on the phone, find out what you want to know, make a decision; if you bring them to interview and they’re struggling, give feedback immediately; don’t have a system where you wait 3 weeks to tell them yes/no).
2. Any effort you expect an applicant to spend will be greatly multiplied. That person is not talking only to you, they’re looking at LOTS of job postings!! If every potential employer starts assigning “homework”, it becomes a major hurdle and you can bet they’ll look a lot closer at an employer that might provide a paycheck in a couple weeks instead of “homework”.
3. You don’t need complex problems to understand if somebody can code. Let them use whatever language their résumé claims they know, give them something to do for a few minutes and see what happens. If they get stuck, give them a solid hint and again, see what happens. You’re trying to assess things like (a) if they appear to know what their résumé claims, (b) if they are the kind of person you can interact with to get somewhere on a problem.
We hire mostly junior developers. I give a simple whiteboard interview with 3 questions, and encourage them to solve it with mostly pseudo-code but have a few "extra points" things that I'm looking for. Absolutely avoided any kind of "trick" questions.
I like the homework idea. I don't know about "coming back" but maybe send it to them a couple days prior to the tech interview.
I have failed many time at interviewing at technical interviews. I have never failed any 'real' world job project/endeavor. I've built an entire org's codebase and this company is now selling products worldwide successfully. Design,coded,built,tested and deployed code in 4 languages , each element being critical to the company.
I am an experienced dev, but couldn't tell you my birthday in a technical interview. All the rest of the interview is fine, I only 'freeze' up at technical questions.
I cannot count how many interviews I've done where I've shown up with 10+ years of industry experience and been given an interview catered toward juniors. If anything there is too much emphasis on interviewing entry level software engineers. There are others out there with a proven track record.
I'm just underselling myself, shooting too low. At this point do I need to be applying for management?
One thing I'd be interested in, but feel like I never see, is a post-post-mortem somewhere down the line, say a year or two later. This gets a little tricky because you don't want to air your company's dirty laundry or say negative things that could be easily traced to point to a specific person, but...
In theory, I like the approach they've taken to hiring associates, but at the end of the day, what really matters is: are the people they end up hiring successful at their jobs? It might take a year or more to really answer that question.
I don't feel like we're very good at this. As 'kinkrtyavimoodh points out in another comment, we see a lot of "how to interview engineer" type posts. But we never find out if, in the end, they were effective. This article claims that everything went really well because they had (or believed they had) such high-quality candidates that it was difficult to choose at the end, but they don't actually know that. They won't find this out after evaluating actual on-the-job performance. It might have been hard to pick from their candidate pool because their process failed to tease out flaws that are only going to become apparent later. We just don't know, and likely will never know.
> Notably, “potential” is nowhere on the scale. Many posts talk about hiring entry level engineers on their “potential,” but the term is seldom defined. Most people use “potential” to describe a sense that, even though a candidate currently cannot be an autonomous engineer, someday in the future they could be. We did not attempt to predict the future for any of our candidates but rather evaluated them on what they presented through the interviews.
> Some of the qualities we looked for in our associates in particular were:
> Resilience: Learning on the job is hard and we assumed that the associates would make mistakes and struggle through difficult concepts. We needed people who could endure these struggles and bounce back ready for the next challenge.
> Willingness to learn and the initiative to do so: Clover would assist the Associates in their growth, and provide teachers and mentors to help along the way, but any incoming Associate would need to be responsible for their own growth.
> Humility: This is an important trait for all Clover engineers, but we paid special attention to it in our Associates. They would have to learn from those around them, be respectful of others, and be able to take difficult feedback with grace.
> Deciding the technical skills to evaluate was a long process. We expected to teach our Associates most of the technical skills they would need to do their job, but we couldn’t accept candidates that were completely blank slates.
It really sounds like this is measuring the candidate's "potential".
I would argue they defined "potential" as "resilience, willingness to learn, and humility", which they feel is an atypical definition.
(Personally, I think that's a decent set of criteria to look for in entry-level engineers.)
We give homework and it has been a successful approach. Even if it's a trivial assignment one can gain a lot of insight on the candidate based on the overall professionalism of their submission.
entry level is bar far the easiest to interview for.
with experienced people they always think they are at or near the top of the scale, but dont understand that you have a different scale entirely.
with a junior person, ask them to describe a project they worked on. if they 'i dunno, school stuff', then you're done.
if you can engage with them meaningfully about their school work, or if they've done personal projects, have developed any kind of insight, show a spark of excitement or creativity then you're done - hired.
just keep an eye on them for a while to make sure they're engaged and absorbing culture. invest a good amount of time in their success (mentorship, pair debugging, whiteboard talks). if they dont start contributing more than they are costing in time after a few months, find them another role or fire them.
I would say, try to test the fundamentals. Do they understands how memory is accessed, have sense what kind of algorithm is used (though it would be too much to code in 45 minutes), ambitious, have general idea to solve a problem, can they see issues if things are not performant... Having a conversation, comfortable conversation is the key, imo.
It'll be interesting to see how it works out. Without some kind of real-time test of facility with abstract symbolic reasoning, I would expect them to get burned. It's way too easy for people to fool themselves and others about that kind of capability.
If they asked the subjects to modify the homework slightly during the debriefing, that might be enough.
It depends on how much effort we want to put into training entry level programmers. Different companies have different requirements. If we intend to train the programmer from the ground up, then we can probably look for programmers who are smart, resourceful and self-sufficient. We can train the rest, like good coding, etc.
If our expectations are higher, then what I look for is code sensibility. Do they cut and paste the code without thinking "hey, this is kind of dumb to do"? Do they see repeated code and think "this can be collapsed into something more succinct"? This is the hardest thing to train for, and the most time consuming from a mentor point of view, so if we have a high bar for entry level programmers, then code sensibility is one of the factors on top of the 3 above that I look for.
I've rejected interns that worked for me from a permanent position because after 4 months, they still had terrible code sensibility. I had to pour through their code with a fine tooth comb because it was riddled with tiny bugs, and it needed constant rewrites because they just didn't improve over the course of the internship. I've had other interns that just "got it" and wrote great code from the start, or only needed a few reviews and then "got it". Those are the ones that will be productive quickly and won't hamper the rest of the team with lengthy reviews.
You with your sensible guidelines, your calibrated metrics, your well-defined endpoints. Where's all the VOODOO!??! Where's the WITCHCRAFT!??!?!
I have only one modest proposal: No. More. Whiteboard interviews!!