Psychometrics · 12 min read

Cognitive Ability Tests in Hiring: What Employers Need to Know

What a cognitive ability test measures, how reliable it is and how to use it fairly. With sample questions, norm groups, AI cheating and an FAQ.

Door Ingmar van Maurik · Founder & CEO, Making Moves

Short answer

A cognitive ability test measures how well someone reasons with words, numbers and abstract patterns. It is one of the most thoroughly researched selection tools: good cognitive tests are highly reliable (often 0.85 or higher) and mainly predict learning ability and performance in more complex roles. Never use it as the only criterion, compare scores with a suitable norm group and pay attention to fairness and cheating in online testing.

This article is part of our complete guide to assessments in hiring.

What does a cognitive ability test measure?

A cognitive ability test (also called an aptitude test or reasoning test) usually consists of several components:

ComponentWhat it measuresSample questionEspecially relevant for

|---|---|---|---|

Numerical reasoningWorking with numbers, tables and ratiosA team processes 120 applications a week; after automation 25% more. How many per week? (150)Finance, analysis, operations Verbal reasoningUnderstanding and drawing conclusions from textDoctor is to patient as lawyer is to ... (client)Legal, communications, policy Abstract reasoningRecognizing patterns without language or arithmetic2, 6, 12, 20, 30, ... (42)Learning ability in almost any role Spatial or mechanical reasoningRotating shapes, technical principlesWhich gear turns the other way?Engineering, logistics

Two formats are worth distinguishing. A speed test has many simple questions under time pressure; a power test has fewer, increasingly difficult questions. Modern online tests are often adaptive: the next question depends on the previous answer. That makes the test shorter and harder to share or copy.

What does a cognitive ability test predict?

According to the large meta-analysis by Sackett, Zhang, Berry and Lievens (2022), the average validity of cognitive tests for job performance is about 0.31. That is strong for personnel selection, though lower than the 0.51 reported by older studies. Its value lies mainly in:

  • Learning ability: how quickly someone picks up new tasks, systems and rules.
  • Complex roles: where a lot of information has to be processed and judged.
  • Training success: who will complete a graduate program or vocational training.

    • A cognitive test says nothing about motivation, integrity, cooperation or customer focus. That is why you combine it with a structured interview, a personality questionnaire or a situational judgment test. Together they predict better than each alone. Which combination works for which role is covered in the guide to assessments.

    How reliable is a cognitive ability test?

    Good cognitive tests are among the most reliable selection instruments, often with reliability between 0.85 and 0.95. Still, every score contains measurement error. With a reliability of 0.90 and a standard deviation of 15 points, the standard error of measurement is about 4.7 points. Two candidates scoring 108 and 103 are therefore not demonstrably different.

    So work with score bands instead of a hard single-point cut-off. How to judge reliability and validity, including a vendor checklist, is covered in what makes an assessment valid and reliable.

    Reading scores: norm group, percentile and stanine

    A raw score (the number of correct answers) says little. You compare it with a norm group:

  • Choose a suitable norm group. Cognitive tests are often normed by education level. A candidate for a graduate-level role is compared with graduates.
  • Percentile: a score at the 70th percentile means the candidate scores higher than 70% of the norm group.
  • Stanine: a scale from 1 to 9 with 5 as the average; useful for communicating broad levels.

    • Even better is a comparison with your own successful employees in the same role. Read how to build your own norm group.

    Fairness: group differences and language

    Cognitive tests show average group differences, related among other things to language and educational background. In the US, adverse impact is commonly checked with the four-fifths rule. Without measures, you unintentionally exclude good candidates. What helps:

  • Only use components the role requires. A heavy verbal test for a role where language matters little disadvantages non-native speakers without predicting anything.
  • Consider non-verbal components, such as abstract reasoning, if the role does not require a high language level.
  • Don't use the test as the only knock-out. Combine it with other methods and apply at most a minimum threshold if the role truly requires it.
  • Offer accommodations, such as extra time for dyslexia.
  • Monitor outcomes per group in your own candidate pool. More in [how AI can reduce bias in hiring](/en/articles/ai-reduce-hiring-bias).

    • Online testing in the age of AI

    Most cognitive tests are taken online and unproctored. That is convenient, but a new risk has emerged: language models can answer many verbal and numerical questions. Measures that work:

  • Tight time limits per question, so looking things up or retyping does not pay off.
  • Large, rotating item banks and adaptive tests, so questions cannot be shared.
  • More abstract and visual components, which are harder to paste into a chatbot.
  • A short proctored verification test for candidates you want to hire. If the score differs sharply, discuss it.
  • Transparency: tell candidates in advance that aids are not allowed and why.

    • Practice and retakes

    Candidates who take a test more than once score slightly higher on average: a meta-analysis by Hausknecht and colleagues (2007) found an average gain of about a quarter of a standard deviation. Two practical consequences:

  • Offer everyone practice questions. Then the advantage does not go only to those who have taken assessments before.
  • Set a retake policy, for example a fixed waiting period before someone may retake the test, and use different questions for a retake.

    • Where in the process?

  • Early at high volume: a short cognitive test as part of the pre-assessment, with at most a mild threshold, so you do not reject too many good people too early.
  • Later for specialist or complex roles: a more extensive test once the group is smaller.
  • Keep it short: twenty to thirty minutes is enough for most roles. Long tests increase drop-off.

    • Off-the-shelf or custom?

    Vendor tests are quick to deploy but work with generic norm groups, per-candidate fees and data held by the vendor. A custom cognitive test works with your own item bank, norms based on your own employees and anti-cheating measures that fit your process. Read why generic assessments often fall short or see how we develop custom assessments.

    Frequently asked questions

    What is a cognitive ability test?

    A standardized test that measures how well someone reasons with words, numbers and abstract patterns. Employers use it to compare learning ability and reasoning level with a norm group.

    How reliable is a cognitive ability test?

    Good cognitive tests usually have a reliability of 0.85 to 0.95, higher than most other selection instruments. Small score differences between candidates still fall within the measurement error.

    How long does a cognitive ability test take?

    Usually twenty to forty-five minutes, depending on the number of components. Adaptive tests are often shorter.

    What is a good score on a cognitive ability test?

    That depends on the norm group and the role. A score around the 50th percentile is average for the chosen norm group. What is "good enough" should be decided per role, preferably based on your own successful employees.

    Can you select on a cognitive ability test alone?

    Preferably not. A cognitive test says nothing about motivation or behavior and can disadvantage certain groups. Combine it with other methods and let a human make the decision.

    How do you prevent AI cheating in an online cognitive test?

    Use tight time limits, large rotating item banks or adaptive tests, more abstract components and a short verification test for candidates you want to hire.

    Summary

  • A cognitive ability test measures reasoning and mainly predicts learning ability and performance in complex roles.
  • Good tests are highly reliable, but read scores with their measurement error and work with score bands.
  • Compare with a suitable norm group, ideally your own successful employees.
  • Mind fairness: choose only relevant components, offer accommodations and monitor outcomes.
  • Take AI cheating seriously in online testing and offer everyone practice questions.
  • Always use it in combination with other methods.

    • Want a cognitive test that fits your roles, with your own norms and protection against cheating? See how we develop custom assessments or get in touch.

    Book intake call · View our AI Hiring System