Forums

Looks like we're about to hire again!  I have very limited experience in hiring and didn't do such a great job last time.  One of the problems we have with the current hire is that she doesn't think critically.  This is a major factor in success or failure in our company.

What kinds of questions can we ask in an interview to find a critical thinker?

Can anyone recommend a good book for Interviewing/Hiring?

I am committed to improving our process and appreciate all the help we can get.

The very first thing we are doing is taking a hard look at the job, the requirements, expectations and skills needed.  If you can suggest other preliminary work, I would also appreciate that.

(For your interest, we are not hiring for my problem direct's job.  This is a whole different department, but it cannot afford a similar mistake)

ashdenver's picture

I think the questions to ask would depend on the job itself.  You could, of course, ask the off-the-wall type of question just to see how they handle such a situation: "How many jelly beans (or gumballs) would it take to fill a 747?"  If they spout off a number, you'd have to probe to see how they came up with it.  If they pulled it out of the air, that's not a good sign.  If they respond by asking more questions - "Are there seats inside the airplane? Or people? What size are the jelly beans? Do we know what the interior volume of the aircraft is and/or how many jelly beans are required to fill a cubic foot?" then you've got yourself a critical thinker.

Otherwise, you could use the straightforward approach: "Tell me about a time when you used critical thinking on a project."

Or you could tailor it to the position for which you're interviewing: "This position relies on solid critical thinking especially when facing XYZ. Tell me about how you've leveraged your critical thinking skills in a similar situation."

Along those same lines, I'd probably also want to know how the candidate utilizes their resources.  They may not be required to have all the answers but a successful candidate in my book (especially in light of the critical thinking component) would also be able to demonstrate successful resource utilization.  If they can give examples where they were able to track down the right resource or best answer, weighing input along the way, boiling things down to key components, etc., I'd be more inclined to consider them as a final contender.

~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~
DiSC profile: 7-2-1-5

bffranklin's picture

One of my favorites to screen for this and a dozen other things is "what's on your bookshelf?"  In general, I'm looking for someone that reads about their field, but also reads things that are chewy and worthy of analysis.  It's not the only criteria, but it's one of those things like the receptionist identifying winning candidates by their shoes -- I've never had a winning candidate throw me pulp fiction and entry level domain books.  I've definitely never had a winning candidate not be "a big reader."

In this particular case, I'd probably also run with some variant of "tell me about an interesting and particularly thorny puzzle you've solved in your previous roles."  Asking about a puzzle should be more likely to produce an interesting story or accomplishment than asking about the amorphous concept of "critical thinking."  A winning candidate is going to take you through enumerating what they knew, enumerating what they didn't know, and the process they took to convert some of the unknowns to knowns in order to arrive at a solution.

 

Vinnie's picture

As background and by way of full disclosure - I did NOT think of asking this question (although I wish I had...). My wife was interviewing for a healthcare analytics firm and was in her final round of interviews with a senior director. He asked this question at the end of the interview. I now ask this question at the END of every one of my behavioral interviews because it is a genius way of gauging a person's willingness and ability to A) think on their feet B) think out of the box and C) not have any way of prepping. So here it goes, and it should be read exactly as I type it:

  • This is an analytical question: What is the circumference of the Earth?

Ok, ok...I'll wait for the jeering to die down.

But this question knocks people for such a loop that only the best, most analytical people who can think on their feet will give you a half way decent answer. And here's the best part - the actual answer DOES NOT MATTER. And to go further - if they know the exact answer then that might tell you something too! It's the thought process behind the answer that actually does matter. I often encourage relatively pensive, quite interviewers to "talk it out". It's amazing some of the answers you get.

BTW, my wife got the job with this simple answer: "Polar or Equitorial circumference?"

She's smart.

STEVENM's picture

I'd have to recommend not using the above question to anyone reading. 

Edit:  Or anything like it.

Vinnie's picture

...but if you are open minded and try this, just for 1 round of interviews, you will see the power of a simple analytical question.

Think it's silly - take a look at some of the questions asked in a Google interview:

  1. How many golf balls can fit in a school bus?
  2. How much should you charge to wash all the windows in Seattle?
  3. How many piano tuners are there in the entire world?
  4. How many times a day does a clock’s hands overlap?

www.businessinsider.com/15-google-interview-questions-that-will-make-you-feel-stupid-2009-11

People may think my post is a joke, but it's not. I get the same reaction from co-workers....until I let them sit in on my interviews. Manager tools interview questions book-ended by my "circumference" question has changed many opinions in my organization.

STEVENM's picture

It's not that I think it's silly (though be fair, it kind of is.  Intentionally so).  I'd never dismiss an idea for that.  It's that I think it's ineffective.  It's designed to see if someone will cope quickly enough to jump through the hoop, and will contort themselves mentally in a way you like.  You might learn who is more 'slick' but that's about it.  Which on its own would be harmless, but:  Even though you can tolerate letting good people slip through as long as you get another good person, you can't tolerate letting not good enough people pass the test.  And I think this question gives them an easy opportunity to do that.  This conversation backs that up in my mind, you seem quite attached to the one exception in your behavioral interview.  You are bound to place a little extra emphasis on it because it's your twist.

When the most honest answer, the "I don't have a clue and don't currently have the tools to find out.  The best I can do is guess.  I'll do that if you like, but it's a guess." answer, is actually a wrong answer then the question is bad.  While you might claim you wouldn't look down on someone who answered that way, I suspect you'll favor the person who contorts every time.  It's the opposite side of the bad interview coin ('clever' interviewing from the interviewee side).  'Clever' interviewing is ineffective no matter which side it's coming from as far as I can tell.

As for Google, and don't take this as an attack at all please, but you're probably not as smart as Google.  Nor are you likely to be working at a company that so consistently tries to break into new markets (the point being they have something very specific they are looking for, and lots of experience doing it).  Even if there is a way to apply this in a positive way, and a situation where it could be used positively, you're probably not equipped to do it and you're probably not in that situation.

And that's ok.  I'm not either.  Not by a long shot.

mattpalmer's picture

Having used the above sort of "analytical question" in interviews, and making really bad hires, I've kinda got to agree with Steven.  I don't think they're effective.

What I have found to work very well is simulation.  It's incredibly difficult to do, because you can't just fire off some stock questions to everyone who walks through the door.  You've got to consider the specific job involved, and then identify some work-related task that can be performed by someone not familiar with your firm's particular ideosyncracies, and then produce a full set of support materials.

For example, I'm hiring for support sysadmins at the moment.  We've put together a virtual machine that has been "broken" in a number of different and interesting ways, and we send the candidate a mocked up "support ticket" from the "customer" describing some vague symptoms.  The candidate needs to fix the machine.  It took several days of one of my top people to come up with two vaguely useful scenarios, and setup the machines.  It's *hard* to come up with good scenarios, because you want to have a variety of tasks of different skill levels, but they need to be orthogonal (so you don't have to fix problem A in order to be able to tackle problem B -- we want to see which areas the candidate is strong, and which they're weak).  They also need some improvement and rework to be even better, which will take some more time.

Despite all this, though, they've been an excellent tool so far.  I've had much more confidence in making "hire" calls on several people, and I've been able to filter out several people who looked good on paper based on abysmal performance on the simulator.  Finally, I think the predictive value of the simulator is solid -- we haven't put much weight on the simulator results in our hiring decisions, but so far we haven't had a candidate whose simulator results are at odds with our other sources of information.  Once we've got more data points and are *really* confident it's the anatidae's testicles, I'm considering making the simulator the first filter phase for sysadmin candidates, because it takes so little of our time to administer (relative to phone screens or in-persons).

In short: I think job simulation is not easy to do, but it is immensely valuable and well worth the additional up-front cost of doing well.

Vinnie's picture

I completely agree Matt. I have actually used that technique when hiring a Project Management mentor for the PMO I run. During the interview I had the candidates simulate a WBS session with myself and 3 other interviewers. Each candidate was presented with 3 different Business requirements. They were able to pick any of the 3 scenarios and were asked to facilitate definition of the first few levels of the WBS during the hour. It worked really well and we were able to pick the right candidate who still works for me and is up for promotion this year.

mattpalmer's picture

 Or, at least, I didn't get any "analytical" questions when I interviewed with Google (got two offers out of them, too, so I mustn't have done too badly at the questions I did get).

derosier's picture

 I used to use such stupid questions before I listened to MT and really learned to interview well.

Silly brain-teaser questions as stated above tell you one thing only: has the candidate heard the question and the requisite answer before. Period.

When I encounter a technical person or manager who asks me these questions I know they don't know how to hire well. Thus they don't get good people on the team you're working with. With the exception of the first time it happened to me, any time I'm asked these questions by someone who will be my direct manager I always turn down the position if I'm offered it. If asked by a potential co-worker I take it as a red flag and weigh it by how much I think I'll be dependent on that person.

Asking these sort of questions tell you nothing about "what has the person done and how well they've done it". Worse, if you ask it too early in the interview it can really tank the interview.

When I do interviews, I follow MT advice pretty strictly. MT doesn't really address technical interviews (ie, can they do skill X), so if doing a technical interview (programmers), I add analytical technical questions based on realistic scenarios they're likely to encounter in the job. Things they really are expected to know. I do include questions to ask them to "write code" on the whiteboard, but they're easy questions, relevant to the position and experience, and they can write it in any language or even pseudocode they like.

The key with these questions are they're formulated to tell me if they actually know the things they claim to on their resume. They're also intended to be easy for someone with the right experience to answer.  For example, I always ask about the "volatile" keyword in C, any embedded programmer that can't easily answer that can't do the work, and anybody who actually has been an embedded programmer for longer than a month can easily answer the question. If there's any embedded guys on this forum you know exactly what I'm saying.

I always try to have a broad set of these technical questions so that in case I hit upon something that the candidate just doesn't know, but doesn't eliminate them I can ask a different set.

Please don't give Google's use of this stuff as evidence that this practice is good interviewing. I've been through their hiring process. It makes no sense and is heavily skewed to hire a very particular demographic.

- Steve