Structured Interviews, Assessment Centers and Hiring Decisions: Why the Evidence Favors the Formula Over the Committee's Final Discussion

"Structured interview" is often used as though it names a single technique, a fixed script read aloud to every candidate. Michael Campion, David Palmer and James Campion's (1997) review in Personnel Psychology, "A Review of Structure in the Selection Interview," took a more useful approach: rather than defining structure as one property an interview either has or lacks, they identified fifteen separate components that researchers had used across decades of studies, grouped into two families. One family concerns the content of the interview itself: whether questions are built directly from a job analysis, whether every candidate is asked the same questions in the same order, whether follow-up probing and prompting are limited or banned outright, and whether candidates are kept from asking their own questions until scoring is complete. The second family concerns how answers are evaluated afterward: whether each answer is rated on a defined scale with example anchors rather than a global impression, whether multiple traits are scored per answer, whether the same panel rates every candidate, and, a detail easy to miss and important later in this lesson, whether individual ratings are combined statistically into a final score rather than through open discussion among raters.

Gary Latham, Lise Saari, Elliott Pursell and Michael Campion's (1980) article in the Journal of Applied Psychology, "The Situational Interview," introduced a question format built on a specific logic: present candidates with a hypothetical job-relevant dilemma, drawn from a systematic analysis of the job's actual critical incidents, and ask what they would do. A situational question for a warehouse supervisor role might describe a scenario in which two team members are in an open dispute during a shift with a delivery deadline an hour away, then ask the candidate to describe their response. The scenario is not invented casually; it comes from goal-setting theory's premise that stating an intention is a reasonably good predictor of later behavior, and from a structured job analysis that identifies which incidents actually separate strong performers from weak ones in that specific role. Answers are then scored against a rating scale built from real responses, ranging from poor to excellent, collected during the same job analysis, so an interviewer is comparing a candidate's answer to documented examples rather than to personal taste.

Tom Janz's (1982) study in the same journal, "Initial Comparisons of Patterned Behavior Description Interviews Versus Unstructured Interviews," introduced a different format resting on a different premise, that specific past behavior predicts future behavior better than a hypothetical answer about intended behavior. Instead of asking what a candidate would do, the behavior description interview asks what the candidate actually did the last time they faced a comparable situation, commonly organized around the structure now popularized as STAR, the situation or task faced, the action taken, and the result that followed. Janz's original study interviewed fifteen teaching assistants four times each, twice using interviewers trained in the patterned behavior approach and twice using interviewers trained in conventional technique, and found the behavioral interviews correlated with later performance at .54 compared with .07, essentially no relationship at all, for the standard interviews. That gap, reported in a small study of one occupational group, should not be read as a universal validity figure for every behavioral interview ever run since; it is better read as the finding that got behavioral interviewing taken seriously enough to generate the much larger body of research summarized next.

Michael McDaniel, Deborah Whetzel, Frank Schmidt and Steven Maurer's (1994) meta-analysis in the Journal of Applied Psychology, "The Validity of Employment Interviews: A Comprehensive Review and Meta-Analysis," pooled results from 160 studies and found that interview validity depended heavily on both structure and content type. Structured interviews outperformed unstructured ones, with a mean corrected validity around .44 against job performance, and among structured formats, situational interviews showed the highest validity at roughly .50, ahead of job-related behavioral questions and well ahead of psychologically oriented questions probing traits or background indirectly. On the strength of that finding alone, a reasonable conclusion would be that situational questions should simply replace behavioral ones.

Arthur Huffcutt, Jeff Weekley, Willi Wiesner, Timothy DeGroot and Chris Jones's (2001) study in Personnel Psychology, "Comparison of Situational and Behavior Description Interview Questions for Higher-Level Positions," complicates that conclusion. Testing both formats with federal investigative agents, a senior, complex role, they found very little correspondence between situational and behavior-description questions written to measure the same underlying characteristic, and found behavior-description ratings linked to a measurable personality trait, extraversion, in a way situational ratings were not. For complex, higher-level jobs the two formats appear to tap something different, so the McDaniel et al. ranking, built mostly from a broader mix of job levels, should not be treated as settling which format to use for every role. A reasonable working rule: situational questions tend to perform well where candidates lack directly comparable past experience, common in frontline and entry-level hiring, while behavioral questions draw on a track record that becomes harder to substitute with a hypothetical as role complexity and candidate experience increase.

A question format, situational or behavioral, only delivers the validity the research promises if the scoring that follows it is itself structured. The Campion, Palmer and Campion (1997) components on the evaluation side of the ledger matter as much as the ones on the content side. A rating scale built with real example answers at each point, poor, adequate, good, excellent, tied to the specific incident the question describes, gives two different interviewers a shared reference for what a 4 looks like, in a way a bare numeric scale without anchors cannot. Scoring immediately after each answer, rather than reconstructing an overall impression at the end of a thirty-minute conversation, reduces the chance that one memorable answer, good or bad, colors the recall of everything that came before or after it, a basic finding from how human memory reconstructs sequences of events rather than storing them individually.

The order of operations in a panel interview matters just as much as the scale itself. When every interviewer records an independent score before the group discusses the candidate, the panel preserves genuinely separate pieces of evidence, several observers' honest reads of the same behavior. When the group discusses first and scores afterward, the discussion tends to collapse toward whichever view is expressed first or most confidently, a pattern familiar from research on group decision-making well beyond interviewing specifically. A panel that skips this sequencing step can call itself structured on every other dimension and still lose a meaningful share of the reliability gain structure is supposed to deliver.

A common assumption treats more interviewers, and more rounds of interviewing, as a straightforward way to reduce the risk of a bad hire, on the reasoning that more eyes catch more problems. The reasoning is not baseless; combining several independent, reasonably accurate judgments does tend to increase reliability, the same logic behind averaging several exam graders rather than relying on one. But the gain from an added rater follows diminishing returns, and depends on that rater contributing a genuinely independent, structured judgment rather than another unstructured impression shaped by the same résumé and the same small-talk rapport earlier interviewers already responded to. Campion, Palmer and Campion's (1997) review frames the structural components, not the raw count of interviews, as the lever that actually moves validity; a candidate who sits through five unstructured conversations with five managers asking whatever comes to mind has not necessarily been assessed more rigorously than one who sat through a single well-built structured interview, and has absorbed more scheduling cost and more chances to drop out from fatigue along the way.

An assessment center is not a single exercise but a method: candidates move through several simulations, commonly an in-basket exercise in which a candidate works through a pile of memos, emails and decisions a manager in the target role might face in one morning, a leaderless group discussion in which several candidates jointly work a business problem with no one assigned to lead, and sometimes a role-play negotiation or a short presentation, while multiple trained assessors independently rate each candidate against a defined set of dimensions, commonly planning and organizing, oral communication, decisiveness and interpersonal sensitivity. Barbara Gaugler, Douglas Rosenthal, George Thornton and Cynthia Bentson's (1987) meta-analysis in the Journal of Applied Psychology found an overall corrected validity of roughly .37 for the assessment center's final rating against later job performance, a genuinely useful level of prediction, and, notably for a theme this lesson returns to directly, found that mechanically combined composites built from the individual dimension ratings produced higher operational validity than the single holistic overall rating a staff psychologist typically produces by integrating everything in their head.

The assessment center's intended logic depends on an assumption worth stating plainly: that a candidate's score on, say, "decisiveness" should look reasonably similar whether it was observed in the in-basket exercise or the group discussion, because decisiveness is meant to be a stable characteristic the candidate carries from one situation to the next. Paul Sackett and George Dreher's (1982) study in the Journal of Applied Psychology, "Constructs and Assessment Center Dimensions: Some Troubling Empirical Findings," tested that assumption directly across three organizations and found the opposite pattern from what the dimension-based logic predicted. Ratings of the same dimension correlated weakly across different exercises, while ratings of different dimensions within the same exercise correlated strongly, and the underlying factor structure of the data lined up with which exercise a candidate was doing, not with which dimension was supposedly being measured. The finding became known as the exercise effect, and it is not a minor statistical footnote; it means that what an assessment center's dimension scores often capture is closer to "how this candidate performed in this particular exercise" than "how much decisiveness, as a trait, this candidate possesses."

Filip Lievens and James Conway's (2001) meta-analysis in the Journal of Applied Psychology, "Dimension and Exercise Variance in Assessment Center Scores," reanalyzed thirty-four earlier studies using a more complete statistical model, and under that model found dimension variance roughly equal to exercise variance rather than dramatically smaller, a more encouraging picture for the dimension-based rationale than Sackett and Dreher's original figures suggested alone. That the exact size of the exercise effect still shifts with the model applied to the same data is an honest finding worth stating rather than resolving artificially; the effect is real and well replicated, but how much of an assessment center's variance it accounts for remains an open methodological argument, not a single settled number.

Charles Lance's (2008) article in Industrial and Organizational Psychology, "Why Assessment Centers Do Not Work the Way They Are Supposed To," pressed the exercise-effect finding into a direct critique: if post-exercise dimension ratings mostly reflect which exercise produced them rather than the dimension they claim to measure, the standard practice of reporting a candidate's score on each named competency rests on a construct-validity foundation the data does not actually support. That critique was published alongside a direct reply in the same journal issue, and the honest reading is not that one side has been proven right and the other wrong. The overall predictive validity Gaugler and colleagues documented has held up across decades of use; what remains genuinely disputed is why assessment centers predict performance, whether it is the named dimensions doing the work, general performance effectiveness displayed differently across exercises, or some mix of both. An organization running an assessment center today can reasonably keep using the overall score as a predictor while being honest that dimension-by-dimension feedback, "you scored low on decisiveness," rests on weaker ground than the overall number does.

Paul Meehl's (1954) book Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence, published by the University of Minnesota Press, compared two ways of combining the same predictive information into a final judgment. Clinical, or holistic, combination lets a trained expert integrate the available data in their head, weighing cues as seems appropriate in the moment, informed by experience and intuition. Mechanical, or statistical, combination applies a fixed, predetermined rule, a regression equation, a simple weighted sum, or even an equal-weighted checklist, to the same inputs, removing case-by-case discretion from how the pieces get combined. Meehl reviewed roughly twenty studies, drawn mostly from clinical psychology and medicine rather than hiring specifically, comparing the two approaches on the same predictive tasks and found that the mechanical rule matched or outperformed the expert's holistic judgment in nearly every one, a result that directly challenged the professional assumption that trained judgment, applied flexibly, should outperform a fixed formula built from the same information.

The finding should state plainly what it does, and does not, say. It is not a claim that experts have nothing useful to contribute, and it is not a claim that formulas can succeed with bad or irrelevant input data; a mechanical rule applied to poor information still produces a poor prediction. It is specifically a claim about the combination step: given the same valid information, a fixed rule for weighing and adding it up tends to outperform a person's in-the-moment integration of it, because that weighting shifts from case to case in ways that are not necessarily better calibrated, often swayed by recency or vividness rather than by whatever actually improves accuracy.

William Grove, David Zald, Boyd Lebow, Beth Snitz and Chad Nelson's (2000) meta-analysis in Psychological Assessment, "Clinical Versus Mechanical Prediction: A Meta-Analysis," revisited Meehl's question across a much larger body of research, well over a hundred studies spanning medicine, mental health and other prediction tasks, and found mechanical prediction outperformed clinical judgment on average, by roughly ten percentage points in accuracy, with clinical judgment substantially better in only a small minority of comparisons and mechanical prediction substantially better in a much larger share. Nathan Kuncel, David Klieger, Brian Connelly and Deniz Ones's (2013) meta-analysis in the Journal of Applied Psychology, "Mechanical Versus Clinical Data Combination in Selection and Admissions Decisions," brought the question directly into the hiring and academic-admissions domain this lesson is concerned with, and reported essentially the same pattern Grove and colleagues had found more broadly, mechanical combination of the same predictive data outperforming expert holistic combination by a comparable margin.

Stated plainly, because the finding deserves to be stated plainly rather than softened: when an organization has already collected valid, job-related information through a structured interview, an assessment center, and perhaps a cognitive ability test, the research says the hiring committee's own discussion does not reliably improve on simply adding the scores together according to a rule set in advance. That conclusion runs against how selection decisions are made in most organizations most of the time, and against a deeply held intuition that experienced judgment, applied to a specific candidate in a specific room, must see something a formula cannot. Meehl's original finding and its later confirmations say that intuition is usually wrong about its own advantage, not that the people holding it are careless or unqualified.

Scott Highhouse's (2008) article in Industrial and Organizational Psychology, "Stubborn Reliance on Intuition and Subjectivity in Employee Selection," examined why the mechanical-combination finding, well established and widely taught, has so little practical traction in actual hiring processes. He identified several implicit beliefs that sustain the resistance rather than treating it as simple ignorance of the research. One is what he calls the myth of expertise, the belief that near-perfect, case-by-case precision in judging a candidate is genuinely achievable by a sufficiently experienced observer, which makes trusting a formula instead feel like a needless concession of accuracy rather than a documented improvement. Another is a resistance to viewing selection as fundamentally probabilistic, with error built into even the best process, because that framing sits uncomfortably against the confidence a hiring manager is expected to project when recommending a candidate to their own superiors.

There are also real organizational and political costs to adopting mechanical combination, and these are worth naming rather than dismissing as imaginary. A formula removes the discretion a senior manager may want to exercise for reasons that have nothing to do with measurement error, protecting a favored internal candidate, accommodating a business relationship, or simply retaining the authority that comes with the final say. A rigid rule applied without any override capacity can also fail visibly in an individual case, even while performing better on average, and a single well-publicized bad outcome from "the formula" tends to generate far more pressure to abandon it than the larger, less visible accumulation of weaker outcomes from years of holistic override ever does. None of this makes the underlying finding less true. It explains, honestly, why a practice this well supported by seventy years of research remains the exception rather than the rule in how hiring committees actually behave.

Lesson 38 established that structured interviews and assessment centers sit well up the validity hierarchy among selection methods. This lesson has argued that earning that validity in practice depends on specific, checkable design choices rather than on labels: situational or behavioral questions built from real job analysis, anchored scoring completed before group discussion, panels that add genuinely independent judgment rather than additional unstructured impressions, and assessment centers whose overall score is trusted more than their exercise-driven dimension breakdown. None of that design work, however careful, resolves the final step every selection process eventually reaches, what to do once several imperfect measures are sitting on the table together. The research on that final step is unusually clear for a field this contested: a predetermined mechanical combination of valid information tends to outperform a committee's informed, well-intentioned discussion of the same information, a finding that has survived seventy years of attempts, including some of its own authors' hopes, to find the conditions under which expert judgment would win instead.

The gap this lesson leaves open is not a gap in the evidence. It is the gap between what the evidence recommends and what a hiring committee, accountable to its own organization, is actually willing to do with a candidate sitting on the other side of a decision that feels like it should involve more than arithmetic. Highhouse's account of that resistance is honest rather than dismissive: the costs of handing over discretion are real, even if the accuracy case against holding onto it is stronger than most committees realize. A hiring process that takes this lesson seriously does not necessarily remove every human judgment from the room; it decides, in advance and on the record, which judgments are structured enough to earn a seat in the final formula and which are better captured as feedback for the candidate, rather than as grounds to overrule it after the fact. Lesson 40 picks up from here, once a hiring decision has actually been made, and turns to onboarding the selected candidate and measuring whether the recruitment and selection effort behind Lessons 36 through 39 is actually working.

Select a real or realistic mid-level managerial role you can describe in reasonable detail, drawn from your own organization (disguised if needed), a documented case, or a role you research directly. Using that role as your basis, prepare a report of 2,000 to 2,500 words that:

1. designs one situational interview question and one behavioral interview question for the role, each grounded in a stated job requirement, with a three-point anchored scoring scale for each, applying Latham, Saari, Pursell and Campion's (1980) and Janz's (1982) formats correctly;
2. identifies at least four of Campion, Palmer and Campion's (1997) structural components your proposed interview process would use, and explains specifically how each one is implemented, not merely named;
3. designs one assessment center exercise appropriate to the role (an in-basket or a leaderless group discussion) and explains, using Sackett and Dreher's (1982) exercise-effect finding, what caution your scoring and feedback approach would need to build in;
4. states a specific, predetermined mechanical rule for combining the interview score, the assessment center score, and any other selection measure you include into one final decision score, and justifies the weights you assign; and
5. argues explicitly, using Meehl (1954), Grove et al. (2000) or Kuncel et al. (2013), and Highhouse (2008), whether your own organization (or the one in your case) would actually follow the mechanical rule you designed when a strong committee impression disagreed with it, and what that answer reveals about the gap between the evidence and actual practice.

Use at least five academic sources, including at least four from this article's reference list.

Assessment rubric (100 marks)

1. Campion, M. A., Palmer, D. K., & Campion, J. E. (1997). A review of structure in the selection interview. Personnel Psychology, 50(3), 655–702. doi.org/10.1111/j.1744-6570.1997.tb00709.x
2. Gaugler, B. B., Rosenthal, D. B., Thornton, G. C., III, & Bentson, C. (1987). Meta-analysis of assessment center validity. Journal of Applied Psychology, 72(3), 493–511.
3. Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30. doi.org/10.1037/1040-3590.12.1.19
4. Highhouse, S. (2008). Stubborn reliance on intuition and subjectivity in employee selection. Industrial and Organizational Psychology, 1(3), 333–342. doi.org/10.1111/j.1754-9434.2008.00058.x
5. Huffcutt, A. I., Weekley, J. A., Wiesner, W. H., DeGroot, T. G., & Jones, C. (2001). Comparison of situational and behavior description interview questions for higher-level positions. Personnel Psychology, 54(3), 619–644. doi.org/10.1111/j.1744-6570.2001.tb00225.x
6. Janz, T. (1982). Initial comparisons of patterned behavior description interviews versus unstructured interviews. Journal of Applied Psychology, 67(5), 577–580.
7. Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus clinical data combination in selection and admissions decisions: A meta-analysis. Journal of Applied Psychology, 98(6), 1060–1072. doi.org/10.1037/a0034156
8. Lance, C. E. (2008). Why assessment centers do not work the way they are supposed to. Industrial and Organizational Psychology, 1(1), 84–97. doi.org/10.1111/j.1754-9434.2007.00017.x
9. Latham, G. P., Saari, L. M., Pursell, E. D., & Campion, M. A. (1980). The situational interview. Journal of Applied Psychology, 65(4), 422–427.
10. Lievens, F., & Conway, J. M. (2001). Dimension and exercise variance in assessment center scores: A large-scale evaluation of multitrait-multimethod studies. Journal of Applied Psychology, 86(6), 1202–1222.
11. McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79(4), 599–616.
12. Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. University of Minnesota Press.
13. Sackett, P. R., & Dreher, G. F. (1982). Constructs and assessment center dimensions: Some troubling empirical findings. Journal of Applied Psychology, 67(4), 401–410.

Verification notes: all thirteen references were checked via web search against publicly indexed bibliographic records, including Wiley Online Library, APA PsycNet-indexed journal pages, and academic citation databases, confirming author names, year, title, journal or publisher, volume and page numbers. The reported statistics (the .54 versus .07 validity figures from Janz, 1982; the .44 and .50 validity figures from McDaniel et al., 1994; the .37 corrected validity from Gaugler et al., 1987; the roughly ten-percentage-point mechanical-prediction advantage from Grove et al., 2000) are drawn from indexed summaries of each study rather than a full-text read of every article; no statistic or finding beyond what is reported in these sources is claimed. The Mallik Freight Brokers case, the tables, the decision scenario, the myth/fact items, and the synthesis connecting this lesson to Lessons 38 and 40 are original teaching material for this series.

Lesson 40 continues Module 08: once a hiring decision has been made using the methods in this lesson, the next step is onboarding the selected candidate and measuring whether the recruitment and selection effort behind Lessons 36 through 39 is actually working.