Top 7 Ways to run great interviews
Whether you want to run structured or unstructured interviews, it's important to structure your goals, scoring, process, training and data collection methods.

Top 7 Ways to run great interviews

Do you or your organization want to run great interviews?  Do you want to talk to a candidate for 15 minutes to an hour and be able to predict how they will do in your company over the next 1-5 years?  You are not alone.

At first glance, interviews seem to be both easy and low cost, requiring that all you have to do is ‘show up’, and they are also often perceived to be ‘fair’ by the interviewee.

As a result, they remain the most widely used personnel selection method, and while they have been shown to often have specific interviewer biases (for example, scoring people most like the interviewer higher, and discriminating based on physical ‘attractiveness’ and certain demographic and personality traits such as ethnicity, age, gender and introversion), they still tend to produce less ‘adverse selection’, aka bias, than many other forms of hiring.

They also have a very low correlation to cognitive tests and personality assessments, making combinations of multiple methods likely more robust and effective. See, for example:

“Given the typically low correlation between these structured interviews and mental ability tests (Cortina, Goldstein, Payne, Davison, & Gilliland, 2000; Huffcutt et al., 1996), there is a distinct possibility of using both to enhance and maximize validity.” (Source 1)

In short, the effectiveness (and limits) of interviews has been studied across cultures and over decades, and it remains one of the best ways to decide who to hire — lagging behind only cognitive ability and integrity tests.

Interviews are a low-cost, easy way to evaluate talent — if they are done right, but most companies simply can’t run great interviews consistently.  As former Google SVP of People Operations, Laszlo Block famously found:

“Years ago, we did a study to determine whether anyone at Google is particularly good at hiring. We looked at tens of thousands of interviews, and everyone who had done the interviews and what they scored the candidate, and how that person ultimately performed in their job.  We found zero relationship. – It’s a complete random mess…”

So, what kind of interviews do work best?

For roughly 1/2 a century, it was nearly universally believed that ‘structured’ interviews were much more predictive than ‘unstructured’ interviews when it came to predicting job and training performance.

And, just what are structured interviews?

While early meta-analysis suggested up to 15 dimensions to structured interviews, more recent meta-analytic work has revealed that there appear to be just four major dimensions to an effective ‘structured interview.’

THE STRUCTURED INTERVIEW

CORE ELEMENTS OF AN EFFECTIVE STRUCTURED INTERVIEW (Source 26)

  • Ask identical questions in an identical order
  • Grade on a consistent scoring key (with, when possible, consistent scorers)
  • Ask questions of similar question sophistication, based on job critical traits determined by a job study or critical job requirements assessment
  • Minimize rapport building between the interviewer and interviewee

Structured interviews typically take the form of either a) a Situational Job Interview (which means asking a job applicant to imagine themselves in a set of specific work situations, and then give information of how they would respond) vs. b) a Behavioral Interview (asking a job applicant to give specific past examples of times they found themselves in a given situation, and talking about how they actually responded, and why).

While, initial research suggested that the Behavioral Interview format was better for a broader range of jobs; more recent research shows this is likely not the case — at least for low and medium skilled jobs. For example:

“…Across both low and medium complexity positions… it would appear that validity tends to be very robust in these situations, and not particularly dependent on the type of job involved or the structured interview format utilized.” (Source 1)

Structured interviews are effective, but less so, in predicting future performance in higher cognitive-complexity jobs. When they are, they likely take the form of a) a panel of interviewers, b) tougher scenarios with no clear cut ‘right or wrong’ answers, and c) at least one person with the ability to evaluate ‘technical skills’ if and when needed.

Most of the remaining comments will focus on interviewing for low and medium skilled positions, which are the bulk of all jobs in America.

In these cases, ‘structured’ interviews have been shown to be both significantly and highly predictive of both job and training performance. They have also been shown to have the ability to be valid measures (although construct validity has been a question).

As a result, there are endless web pages and papers published around best ‘question’ and ‘answer’ techniques. For example, the ‘STAR’ method is frequently described among ‘best practices’ for answering a Behavioral Interview. The core features of this method are a push for the respondent to use ‘specificity’, and concrete examples:

(S) Situation. Describe the situation in which the event took place.

(T) Task. Describe the task you were asked to complete. If there was a particular problem or issue you were trying to solve, describe that here.

(A) Action. Explain what action you took to complete the task or solve the problem.

(R) Results. Explain the result of your actions. For example, if your actions resulted in completing a task, resolving a conflict, improving your company’s sales record, etc., explain this. Try to focus on how your actions resulted in a success for the company. (Source 2)

Many Universities now hand out long lists of potential questions to graduates to help them prepare. There is also some evidence that allowing candidates access to questions, and explaining the reasons behind the process both increase applicant satisfaction with the less personal nature of structured interviews.

However, over the past 10 years, there have been a number of interesting issues emerging around the interview in the research literature.

For example, unstructured interviews were shown to be more closely related to a job candidates own self-report of conscientiousness, emotional stability and grade point average.

Structured interviews scores on these personality dimensions had almost no correlation to these ‘job performance critical’ constructs.

Then, in the last few years, new and better meta-analytic techniques, and the most up-to-date research in the field now show that the two forms of interviews are likely very similar in their ability to predict real job outcomes.

“Although the widely held belief is that structured, rather than unstructured, interviews are superior in validity, our results suggest that unstructured interviews may indeed possess greater validity than previously recognized. In other words, it may be the case that unstructured interviews are as valid as structured interviews.” (Source 26)

This is a MAJOR reversal of decades of work.

These new, more reliable approaches have found that the interview is, in fact, very robust and predictive — and a great compliment to cognitive and integrity testing — and also that both structured and unstructured interview techniques appear to be equally valid.

SYNTHESIS:  OUR TOP 7 LIST OF BEST PRACTICES

So, how does interviewing work best?

  1. Decide when and how the job interview will be used in the overall hiring process — at what stage in the talent funnel, and for what specific purpose(s).
  2. Decide on the amount of time and interview format (automated vs. in-person, structured vs. unstructured — single person vs. multiple people interviewing the candidate one at a time vs. group).  These should vary by job type — keep the interview format related to the type of job, i.e. shorter for lower skilled workers and more in-depth for more highly skilled, technically complex positions.
  3. Define the Job Competencies to assess. The broadest ‘classes’ of traits to discuss include mental abilities (problem solving and decision making examples and process and general communication ability), personality (integrity, reliability, conscientiousness, emotional stability, and extraversion), social skills (teamwork, collaboration), interest and preferences (i.e. ideal motivations, values, creativity, ideal jobs, etc), org. fit questions, knowledge and skills and physical attributes (if relevant; i.e. ability to lift X pounds). Interviews are likely least effective at assessing personality traits.
  4. Create the ‘scoring rubric’ — and define the scoring values and methodology clearly.
  5. Train all people who will be conducting interviews on the competencies and the scoring, and make them aware of potential cognitive traps and common interview ‘bias’ issues.  The simplest lesson here is focus on evaluating the specifics of the ‘job-competencies’ demonstrated, and not on the person.
  6. Allow each interviewer to independently score each applicant on every assessment item as soon as possible during or right after the interview, before talking to anyone else or being ‘biased’ by discussion in any way.
  7. Use technology to aid and standardize this process, and allow for systematic reviews and improvement. If possible, add process and outcome-based analytics.

At Talytica, we are committed to staying on the forefront of the latest research, and consistently giving you the tools to follow the best practices from the most recent research work. We recommend consistent training and a consistent process for anyone who will be making hiring and promotion decisions. We give you the tools to have multiple, independent scorers in a repeatable process and framework. We make it easy to collect and use unstructured or structured interviews as ‘input data’ in models.

We have standard interview tools for both structured and unstructured interviews for organizations of all sizes.

We even offered fully outsourced hiring, following best practices.

So, get in touch.

SOURCES

  1. The Impact of Job Complexity and Study Design on Situational and Behavior Description Interview Validity. Conway, J; Klehe, U and 4 others. 2004.
  2. Source 2. https://www.thebalance.com/behavioral-job-interview-questions-2059620
  3. Arvey, R., & Campion, J. (1982). The employment interview: a summary and review of current research. Personnel Psychology, 35, 281–322.
  4. Bayne, R. & Fletcher, C. (1983) Selecting the selectors. Personnel Management, 15, 42–44.
  5. Burnett, J. R., & Motowidlo, S. J. (1998). Relations between different sources of information in the structured selection interview. Personnel Psychology, 51, 963–983.
  6. Campion, M., Palmer, D., & Campion, J. (1997). A review of structure in the selection interview. Personnel Psychology, 50, 655–702.
  7. Conway, J. M., Jako, R. A., & Goodman, D. (1995). A meta- analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80, 565–579.
  8. Dayan, K., Fox, S., & Kasten, R. (2008). The Preliminary Employment Interview as a Predictor of Assessment Center Outcomes.Journal of Selection and Assessment,16, 102–111.
  9. Dipboye, R. L. (1997). Structured selection interviews: Why do they work? Why are they underutilized?. In N. Anderson & P. Herriot (Eds.), International handbook of selection and assessment (pp. 455–473). Chichester: John Wiley & Sons.
  10. Fletcher, C. (1992). Ethical issues in the selection interview. Journal of Business Ethics, 11, 361–367.
  11. Hausknecht, J.P., Day, D. V., & Thomas, S. C. (2004). Applicant reactions to selection procedures: An updated model and meta-analysis. Personnel Psychology, 57, 639–683.
  12. Herriot, P. (1993). Interviewing. In P. Warr (Ed.), Psychology at work (pp. 139–159). London: Penguin.
  13. Huffcut, A., & Roth, P. (1998). Racial group differences in employment interview evaluation. Journal of Applied Psychology, 83, 179–189.
  14. Huffcutt, A.J. & Woehr, D.J. (1998) Further analysis of employment interview validity: A quantitative evaluation of interview related structure methods. Journal of Organizational Behavior, 20, 549–560.
  15. McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599–616.
  16. Moscoso, S. (2000). Selection interviews: A review of validity evidence, adverse impact and applicant reactions. International Journal of Selection and Assessment, 8, 237–247.
  17. Phillips J. M. (1998). Effects of realistic job previews on multiple organizational outcomes: A meta-analysis. Academy of Management Journal. 41, 673–690.
  18. Pulakos, E., Schmitt, N., Whitney, D., & Smith, M. (1996). Individual differences in interviewer ratings: The impact of standardisation, consensus discussion and sampling error on the validity of the structured interview. Personnel Psychology, 49, 85–102.
  19. Pulakos, E., & Schmitt, N. (1995). Experience-based and situational interview questions: Studies of validity. Personnel Psychology, 48, 289–308.
  20. Rasmussen, K. G. (1984). Non-verbal behavior, verbal-behavior, resume credentials, and selection interview outcomes. Journal of Applied Psychology, 69, 551–556.
  21. Reilly, R. R. & Chao, G. T. (1982) Validity and fairness of some alternative employee selection procedures. Personnel Psychology, 35, 1–62.
  22. Salgado, J. F. (1999). Personnel selection methods. In C L. Cooper and I. T. Robertson (Eds.) International Review of Industrial and Organizational Psychology. New York: Wiley.
  23. Silvester, J. (1997). Spoken attributions and candidate success in graduate recruitment interviews. Journal of Occupational and Organizational Psychology, 70, 67–74.
  24. Stevens, C. K., & Kristoff, A. L. (1995). Making the right impression: A field study of applicant impression management during job interviews. Journal of Applied Psychology, 80, 587–606.
  25. Wiesener, W.H. & Cronshaw, S.F. (1988) A meta-analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61, 275–290.
  26. Rethinking the Validity of Interviews for Employment Decision Making: Implications of Recent Developments in Meta-analysis (2015). By In-Sue Oh, Bennett E. Postlethwaite, Frank L. Schmidt.
  27. Arvey, R. D., & Campion, J. E. (1982). The employment interview: A summary and
    review of recent research. Personnel Psychology, 35, 281–322.
  28. Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: What do we know and where do we go next? International Journal of Selection and Assessment, 9, 9–30.
  29. Berry, C. M., Sackett, P. R., & Landers, R. (2007). Revisiting interview-cognitive ability
    relationships: attending to specific range restriction mechanisms in meta-analysis. Personnel Psychology, 837–874.
  30. Borman, W. C., White, L. A., Pulakos, E. D., & Oppler, S. H. (1991). Models of supervisory job performance ratings. Journal of Applied Psychology, 76, 863–872. Campbell, J. P., McCloy, R. A. Oppler, S. H., & Sager, C. E. (1993).
  31. A theory of performance. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organization (pp. 71–98). San Francisco: Jossey-Bass.
  32. Structured vs. Unstructured Interviews, Campion, M. A., Palmer, D. K., & Campion, J. E. (1997). A review of structure in the
    employment interview. Personnel Psychology, 50, 655–702.
  33. Chapman, D. S., & Zweig, D. I. (2005). Developing a nomological network for interview
    structure: Antecedents and consequences of the structured selection interview.
    Personnel Psychology, 58, 673–702.
  34. Conway, J. M., Jako, R. A., & Goodman, D. F. (1995). A meta-analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80, 565–579.
  35. Cooper, H. (2003). Editorial. Psychological Bulletin, 129, 3–9. Cortina, J. M., Goldstein, N. B., Payne, S. C., Davison, H. K., Gilliland, S.W. (2000).
    The incremental validity of interview scores over and above cognitive ability and conscientiousness scores. Personnel Psychology, 53, 325–351.
  36. Dipboye, R. L. (1997). Structured selection interviews: Why do they work? Why are the underutilized? In N. Anderson, & P. Herriott (Eds), International Handbook of Selection and Assessment (pp. 455–473). New York: Wiley.
  37. Dipboye, R. L., & Gaugler, B. B. (1993). Cognitive and behavioral processes in the selection interview. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organization (pp. 135–170). San Francisco: Jossey-Bass. Structured vs. Unstructured Interviews.
  38. Dipboye, R. L., Gaugler, B. B., Hayes, T. L., & Parker, D. (2001). The validity of unstructured panel interviews: More than meets the eye? Journal of Business and Psychology, 16, 35–49. Dipboye, R. L., & Jackson, S. L. (1999).
  39. Huffcutt, A. I., & Arthur, W. Jr. (1994). Hunter and Hunter (1984) revisited: Interview validity for entry-level jobs. Journal of Applied Psychology, 79, 184–190.
  40. Maurer, T. J., & Solamon, J. M. (2006). The science and practice of a structured interview coaching program. Personnel Psychology, 59, 433–456.
  41. McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb, W. L. III (2007). Situational judgment tests, response instructions, and validity:
  42. McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 799, 599–616.
  43. Motowidlo, S. J., & Burnett, J. R. (1995). Aural and visual sources of validity in employment interviews. Organizational Behavior and Human Decision Processes, 61, 238–249.
  44. Palmer, D. K., Campion, M. A., & Green P. C. (1999). Interviewing training for both applicant and interviewer. In R. W. Eder & M. M. Harris (Eds.), The employment interview handbook (pp. 337–352), Thousand Oaks, CA: Sage.
  45. Implicit sources of bias in employment interview judgments and decisions. Organizational Behavior and Human Decision Processes, 101, 152–167. Raju, N. S., & Brand, P. A. (2003).
  46. Ryan, A. M., McFarland, L., Baron, H., & Page, R. (1999). An international look at selection practices: Nation and culture as explanations for variability in practice. Personnel Psychology, 52, 359–392.
  47. Schmidt, F. L., Hunter, J. E., McKenzie, R. C., & Muldrow, T. W. (1979). The impact of valid selection procedures on work-force productivity. Journal of Applied Psychology, 64, 609–626.
  48. Van der Zee, K. I., Bakker, A. B., & Bakker, P. (2002). Why are structured interviews so rarely used in personnel selection? Journal of Applied Psychology.