How IQ tests are made, from blueprint to norms

0 The short answer

A professional IQ test is built in a fixed sequence of documented steps: a written blueprint, item writing and expert review, tryout, item analysis, a fairness review, a norm sample, reliability and validity studies, and a revision cycle that runs for years. The Stanford-Binet Fifth Edition came out of a 7-year revision project and was normed on 4,800 people aged 2 to 96. The WAIS-5 was normed on 2,020 people between February 2023 and January 2024, and the Woodcock-Johnson V collected 8,209 research cases across four studies. A quickly built test can copy the look of an IQ test, but it cannot show the paperwork that tells a reader what its score means.

1 What Does an IQ Test Developer Decide Before Writing a Single Item?

The first product of an IQ test is a written specification, and it fixes what the final score is allowed to mean. The Standards for Educational and Psychological Testing, published jointly by the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education in 2014, open their chapter on test design with a requirement that tests be designed and developed so as to support the validity of score interpretations. Standard 4.1 asks that the specifications describe the purpose of the test, the definition of the construct or domain measured, the intended examinee population and the interpretations for intended uses. The accompanying comment says the domain definition should be detailed enough to show which dimensions of knowledge, skill or cognitive process are included and which are excluded. Standard 4.2 adds the practical layer: content, proposed length, item formats, the desired psychometric properties of items and of the test, item ordering, time allowed, directions, administration and scoring and reporting procedures.

For intelligence tests the construct definition is usually a model of cognitive abilities. The Woodcock-Johnson V states in its technical abstract that its test design blueprint rests on the most current articulation of the Cattell-Horn-Carroll theories by Schneider and McGrew (2018), a blend of Carroll's 1993 and Horn's 1991 models. The same document shows how a blueprint changes between editions: the developers removed auditory processing (Ga) from the cognitive battery and moved those tests to a separate test library, citing Carroll's factor analytic research and user feedback that Ga tests deflated the general ability score for examinees with low auditory processing. The Stanford-Binet Fifth Edition is described in a Wiley chapter excerpt as covering five cognitive factors (Fluid Reasoning, Knowledge, Quantitative Reasoning, Visual-Spatial Processing and Working Memory) in both a verbal and a nonverbal domain, with five nonverbal and five verbal subtests. Pearson's WAIS-5 "By the numbers" sheet lists 20 subtests, 15 retained from the WAIS-IV and 5 new, and its overview brochure refers to a 7-subtest Full Scale IQ. The page on the Cattell-Horn-Carroll model explains the abilities these blueprints share, and the page on what IQ measures explains why a blueprint that names abilities is more useful than one that names a single trait.

Why does the blueprint come first? Because every later step is judged against it. An item is kept if it measures what the specification says the test measures. A norm sample is chosen to match the population the specification names. A validity study asks whether scores behave as the construct implies. Cronbach and Meehl's 1955 paper on construct validity in psychological tests is the classic treatment of that last idea, and the Standards comment on 4.0 says that initial specifications "may be modified or expanded as development proceeds" while both the initial and the final documentation give outside experts a basis for judging whether the intended uses are supported. A specification is therefore not a promise that nothing changes. It is the document that makes changes visible.

2 How Are IQ Test Items Written, and Who Checks Them?

Items are written to a specification by trained writers and then reviewed by other people before any examinee sees them, and the review process itself is something the Standards ask developers to document. Standard 4.7 states that the procedures used to develop, review and try out items and to select items from the item pool should be documented. Its comment adds that the qualifications of the people who develop and review items, and the processes used to train and guide them, are important parts of that documentation. Standard 4.8 asks that the review include empirical analyses, expert judges, or both, and that the qualifications, experience and demographic characteristics of any expert judges be recorded along with the instructions they received.

What do item writers work from? In the specification, each subtest has a format: a recall span of a given length, a matrix to complete, a word to define, a set of symbols to scan under a time limit. Writers rarely start from nothing. The chapter excerpt on the Stanford-Binet Fifth Edition lists, among that edition's features, the retention of a few classic Binet items alongside new scales. The Woodcock-Johnson V abstract describes routine content updates for every new edition: dropping out-of-date items, adding contemporary items to a test's item pool, revising artwork, changing administration procedures and clarifying instructions. In a specific case the document says out-of-date items were eliminated from the domain-specific science, social studies and humanities tests, the item pools were combined into two new tests, Academic Facts and Academic Vocabulary, and new items were written to cover twenty-first-century science, technology, engineering and math knowledge and vocabulary.

A good deal of the craft is plain rule following. Haladyna, Downing and Rodriguez reviewed the literature on multiple-choice item writing and validated a taxonomy of 31 guidelines against two kinds of evidence, the consensus of 27 educational testing textbooks and the results of 27 research studies and reviews published since 1990. That work was aimed at classroom assessment, and many intelligence test items are not multiple choice at all, so it should be read as an example of how item writing rules get tested against evidence rather than as the rulebook of any intelligence battery. The principle carries over. Rules about clear stems, plausible wrong options and one defensible right answer are meant to keep irrelevant features of wording from driving the score.

Items that are not multiple choice raise a second problem, which is scoring. A vocabulary item that accepts a spoken definition, or a similarities item that asks how two things are alike, needs a scoring rubric before it can be scored consistently. Standard 4.18 says that scoring procedures and criteria should be presented with enough detail and clarity to maximize the accuracy of scoring, and its comment says that in scoring complex responses developers must provide detailed rubrics and training in their use, with multiple examples of responses at each score level. The page on the types of subtests shows which tasks are scored by key and which by rubric, and the page on IQ test question formats describes the formats without reproducing items.

3 What Happens in a Tryout, and Why Does It Not Look Like the Final Test?

A tryout administers draft items to a sample of people so that the statistics can be computed, and its value depends on how closely the tryout sample and conditions resemble the intended population and the final test. Standard 4.9 asks that when item or form tryouts are conducted, the procedures used to select the sample and the resulting sample characteristics be documented, and that the sample be as representative as possible of the population for which the test is intended. Its comment warns that conditions in a tryout may differentially affect performance, and gives an example: test takers may be less motivated when they know their scores will not have an impact on them.

The comment to Standard 4.10 lists what can differ between a field test and the final form: the motivation of the test takers, item position, time limits, length of test, mode of testing (paper and pencil versus computer) and use of calculators or other tools. Each of these can change an item statistic. An item that sits fifth in a tryout booklet and thirtieth in the final test may be answered under different fatigue, and an item tried out without a time limit and then used with one may look easier than it plays. Developers are asked to consider such differences and to say how they might have affected item performance.

Modern projects run tryout, calibration, norming and validation studies side by side. The Woodcock-Johnson V technical abstract reports four separate studies conducted concurrently between February 2022 and January 2024, with 8,209 research cases in all: 5,837 norming cases, 1,430 calibration, alternate-form and test-retest cases, 639 concurrent validity cases and 369 clinical validity cases. The calibration study was designed with three purposes. It collected item data for Rasch calibration of the item pools for the second forms of the nonspeeded tests, it collected alternate-form reliability data for the second forms of the speeded and fluency tests, and it collected test-retest data for tests without a second form. Its sampling plan called for 70 examinees at each year of age for ages 3 and 4, 59 at each year for ages 5 to 19, and 59 in each 10-year group from ages 20 to 80 and older. The abstract adds that this study was not required to match census proportions for race and ethnicity, though the sample was monitored for diversity in race, ethnicity, region and education level. A tryout can therefore be smaller and less representative than a norm sample as long as it is large and varied enough to estimate the item statistics, which is the standard that 4.10 sets: a sample "of adequate size and diversity for the procedure".

For the WAIS-5, the public documents I could open do not describe the tryout. The Pearson overview brochure says the normative samples were collected in 2023 and 2024, and an independent paper by Winter, Trudel and Kaufman in the Journal of Intelligence does not describe item tryout or pilot steps in its account of the norms. The technical and interpretive manual that would contain that chapter is a commercial product, and this page does not quote it. That absence is worth stating plainly, because tryout is a step the Standards expect to be documented, and the reader of a summary document usually cannot see it.

4 How Does Item Analysis Decide Which Items Survive?

Item analysis turns tryout answers into two numbers per item, how hard it was and how well it separated stronger from weaker performers, and developers keep, revise or drop items on those numbers. Standard 4.10 names the screening data explicitly: item difficulty, item discrimination and differential item functioning for major examinee groups. It asks developers to document the model used to evaluate items, whether classical test theory, item response theory or another model, the sample used, and the process by which items were screened. Difficulty, in the classical sense, is the proportion of the tryout sample that answered correctly. Discrimination is how strongly success on the item goes with success on the rest of the test. An item that nearly everyone passes or fails tells the test little, and an item that high scorers miss more often than low scorers works against the construct and is usually rewritten or removed. Those two definitions are the ordinary textbook ones, stated here in our words.

Item analysis also decides item order, which matters more in individually administered intelligence tests than in a printed exam. Wechsler-style subtests use start points, reversal rules and discontinue rules, and a 2021 study of the Egyptian adaptation of the WAIS-IV by Abdelhamid, Bassiouni and Gómez-Benito notes that those rules assume items are ordered by difficulty. The authors applied item response theory to 250 adults aged 18 to 24 and reported that the item order in many subtests differed from the order in the manual: only Digit Span and Letter-Number Sequencing kept the original order, and, for example, one Arithmetic item was easier than three items placed before it. They also reported that about 70 percent of items discriminated well, that 25 items did not fit the preferred model, and that a few items, among them particular Matrix Reasoning and Vocabulary items, discriminated poorly. Two limits apply. The sample was 250 young adults in one country, tested on an adapted version, and several subtests were left out of the analysis, so the study says nothing about the US norming of the WAIS-IV. It also does not show that the published tests are unsound, and the authors conclude that the adapted test offers a highly valid assessment of intellectual abilities despite the need for some improvements. What it shows is that item order and item quality are empirical questions that can be rechecked years after publication, and that the WAIS-IV manual did not report the item response theory difficulty values a reader would need to recheck them.

Speeded tests are handled differently. On a task such as scanning for symbols under a time limit, the score reflects how many items are completed in the time allowed. The Woodcock-Johnson V abstract shows one solution: for its speeded and fluency tests the developers collected alternate-form reliability data on second forms, and for some tests without a second form they collected test-retest data. For rate-based naming tests the abstract reports only test-retest coefficients, because marginal reliability would be uninterpretable given the rate-based scoring. The page on why IQ tests are timed explains what the clock adds to a score, and the page on how IQ is calculated follows a raw score through the scoring chain once the items are fixed.

5 What Do Item Response Theory and Rasch Calibration Add?

Item response theory describes each item by its own parameters, which lets developers place items on a common scale, estimate precision at each ability level and build parallel forms, and the price is that its assumptions must be checked. Reise, Ainsworth and Haviland introduce three fundamentals of item response theory: item response functions, information functions and invariance, and they report that available evidence suggests the differences between item response theory and traditional psychometric methods are not trivial. Abdelhamid and colleagues explain the practical point in the context of intelligence testing: in classical test theory, item estimates depend on the sample and person estimates depend on the items, while item parameters estimated under item response theory are invariant across samples. That invariance is what lets a developer calibrate a pool of items on one sample and then use those items in another group, or in another form, without starting again.

Intelligence tests use these models in different ways. The Woodcock-Johnson V technical abstract describes collecting item data to Rasch calibrate the item pools for the second forms of its nonspeeded tests, and its norms are built on W scores, which the abstract describes as rising rapidly with age during childhood and stabilizing from middle adulthood on. The Stanford-Binet Fifth Edition uses the approach for what the excerpt calls Change-Sensitive Scores, which give criterion referenced interpretation of scores based on item response theory, and it begins with two routing subtests, one nonverbal and one verbal, that place each examinee at an appropriate level before the functional-level subtests. In the Egyptian WAIS-IV study the models were fitted afterward as analysis only: the two-parameter logistic model fit best for dichotomous subtests and the graded response model for polytomous ones, and, the authors state, they were not used for the manual's own scoring.

Item response theory is not a free upgrade, because its conclusions are only as good as its assumptions. The comment to Standard 4.10 says that when item response theory is the basis of test development, developers should document the adequacy of fit of the model by showing how far its assumptions, such as unidimensionality, local item independence and, for certain models, equality of slope parameters, are satisfied. A subtest that mixes two abilities strains unidimensionality, and a set of items that share one stimulus strains local independence. The concern is not abstract: in the Egyptian WAIS-IV analysis, 25 items did not fit the preferred model, 14 of them in polytomous subtests.

Computer adaptive tests lean most heavily on these models, since each examinee receives a different set of items and the score must still be comparable. Standard 4.3 asks that the rationale and supporting evidence be documented for the procedures used in selecting items, in determining the starting point and termination conditions, in scoring and in controlling item exposure. Standard 5.16 asks for documentation that scores have comparable meaning over alternate sets of items. The page on adaptive IQ tests and stop rules covers what a stop rule changes in practice.

6 How Do Developers Check That an Item Is Fair Across Groups?

Fairness review combines expert judgment with a statistical screen for differential item functioning, which flags items that behave differently for equally able members of different groups and sends them to a human decision. The comment to Standard 4.8 says that, when sample size permits, empirical analyses are needed to check whether items function similarly for different groups. The comment to Standard 4.10 asks that the statistics used for flagging be described, including the groups analyzed, the criteria for flagging and the procedures for reviewing and making final decisions about flagged items, and it asks that sample sizes for groups of concern be adequate for detecting meaningful differential item functioning.

Two details deserve attention. First, by definition the screen compares people matched on ability: it asks whether an item is harder for one group among examinees with the same overall standing, which is a different question from whether the groups have different average scores. A difference in group averages is not by itself evidence of an item problem, and an item with no flagged difference does not prove that the test as a whole is free of bias. Second, flagging is a prompt for review. An item can be flagged and kept, because the difference may reflect the construct itself, and an item can be dropped for content reasons even when its statistics look clean.

The methods have a long literature. Swaminathan and Rogers presented a logistic regression model for characterizing differential item functioning between two groups, distinguishing uniform from nonuniform differences, and showed in simulations that it is more powerful than the Mantel-Haenszel procedure for detecting nonuniform differential item functioning and as powerful for uniform differences. Choices of this kind are what developers document: which statistic, which groups, which threshold. The Woodcock-Johnson V abstract states that special efforts were made to provide fair, unbiased measures and refers readers to the full technical manual, but the abstract itself does not list flagged items or thresholds. For the WAIS-5 and the Stanford-Binet Fifth Edition, the summary documents I could open say little about this step. That is not evidence the analysis was not done, since the full technical manuals are commercial products. It is a reminder that what a reader can verify depends on what a publisher releases.

A reader does not need the statistics to use the principle. Ask whether the publisher names the groups it checked, the sample size available for each, and what happened to the flagged items. The page on whether IQ tests are biased takes up measurement invariance and the evidence on group differences in depth, so this page stops at the construction step.

7 How Is a Standardization Sample Drawn?

A standardization sample is recruited to a plan that targets the population by age and by a short list of demographic variables, and the plan, the participation and the dates of testing are what a reader needs in order to judge the norms. Standard 5.9 states that reports of norming studies should include precise specification of the population sampled, sampling procedures and participation rates, any weighting of the sample, the dates of testing and descriptive statistics, and that technical documentation should indicate the precision of the norms themselves. Standard 5.8 asks that norms refer to clearly described populations that include the groups with whom test users will ordinarily want to compare their examinees. The Woodcock-Johnson V abstract quotes the Standards on this point: the validity of norm-referenced interpretations depends in part on the appropriateness of the reference group to which scores are compared.

Table 1 sets out what the sources I opened say about three current batteries.

The WAIS-5 figures come from Winter, Trudel and Kaufman, who report 180 people per age band for ages 16 to 69 and 100 per band for ages 70 to 90. The Stanford-Binet figures come from the Wiley chapter excerpt, which says the highest age grouping in the norm tables was 85 and older and that the composition of the sample closely approximated the stratification percentages reported by the US Census Bureau (2001). The Woodcock-Johnson V figures come from its technical abstract. The three sample sizes are not a ranking of quality: each instrument spans a different range of ages and divides its people into a different number of age groups.

Age grouping is where the arithmetic bites. The Woodcock-Johnson V sampling plan called for a total of 6,000 examinees across 24 sampling age groups, with a target of 250 per group: one-year groups from ages 3 through 19, 10-year groups from 20 through 79 and a single group for 80 and older. The abstract reports that the average obtained number per group from ages 3 through 79 was 245, and that the 80 and older group reached only 190, which it attributes to the difficulty of recruiting older adults. Sampling was concentrated at younger ages because the abilities measured change fastest there.

The Woodcock-Johnson V design also shows how large batteries are normed. With 60 tests, it was impractical to give every norming participant every test, so the developers used a multiple matrix sampling design: each examinee took about one third of the tests, for roughly 3 hours, and plausible scores for the untaken tests were imputed from the scores obtained. Individual weights were then applied so that the sample proportions matched the US population, each weight being the product of partial weights for demographic variables, and bootstrap resampling was used to construct the norms. According to the abstract, 251 resampled data sets were created, each sorted by age in months and divided into overlapping blocks of 250 examinees offset by 50, and each block yielded a median age, a median score and two spread estimates, which is how the age curves behind the standard scores were built. The abstract says the bootstrap approach allows for estimates of uncertainty and potential bias in the sample data. Whether one prefers that method to alternatives is a question for the page on how IQ scores are normed, which also covers the conversion of a raw score to a standard score with a mean of 100 and a standard deviation of 15. The point here is narrower. Every one of these choices is documented, dated and open to criticism, and that is what makes a norm a norm and not just a number.

8 How Is the Reliability of an IQ Test Established?

Reliability is documented with several different studies, because no single coefficient shows how consistent a score is across items, forms, occasions and scorers. The Woodcock-Johnson V abstract says that, in accordance with the Standards, developers should present reliability indices to support all the proposed uses and interpretations of the test scores, and that it computed several types for tests and clusters: marginal reliability from the norming sample, alternate-form reliability, and test-retest stability. Marginal reliability is computed from the standard error of measurement and the observed variance of scores, and the abstract reports it test by test in six age groups. The result is not a single number per test. The Matrices test is reported at 0.82 for ages 6 to 9, 0.85 for ages 10 to 14 and 0.87 for ages 50 to 80 and older, with a median of 0.85. Phonemic Word Retrieval is reported at 0.68 for ages 4 to 5 and at 0.94 for ages 20 to 49 and for ages 50 to 80 and older. The same task, in other words, measures some age groups far more precisely than others, and a reliability figure quoted without the age group it belongs to is incomplete. The page on reliability and validity explains the coefficients themselves, and Cronbach's 1951 paper on coefficient alpha and the internal structure of tests introduced coefficient alpha, one of the internal consistency estimates.

The Wiley chapter excerpt on the Stanford-Binet Fifth Edition shows the other usual pattern. It reports split-half reliabilities, with internal consistency of .95 to .98 for the IQ scores, .90 to .92 for the five Factor Index scores and average subtest reliabilities of .84 to .89. For the Full Scale IQ it lists a reliability of .98 and a standard error of measurement of 2.30 points. A composite built from many subtests is more reliable than any one of them, which is why the Full Scale IQ is the most precise number in a report and a single subtest score the least. Using the standard error, a 95 percent interval around a Full Scale IQ of that kind would reach roughly 4.5 points either side of the obtained score (our arithmetic: 1.96 times 2.30), treating the interval as centered on the obtained score. For the WAIS-5, Winter, Trudel and Kaufman report an average Full Scale IQ reliability of 0.97 and a stability coefficient of 0.93 for 201 people retested after an average of 29 days. Pearson's "By the numbers" sheet also lists interscorer agreement for the Verbal Comprehension subtests, which is the check that two examiners scoring the same spoken answers reach the same score.

9 What Kinds of Validity Evidence Do Developers Collect?

Validity is not a single study. It is the accumulated evidence that scores mean what the blueprint says, and the Standards call it a unitary concept rather than a list of separate types. Their chapter on validity says that sources of evidence "do not represent distinct types of validity" and describes validity as the degree to which all the accumulated evidence supports the intended interpretation of test scores for the proposed use. The sources it lists are evidence based on test content, on response processes, on internal structure and on relations to other variables, together with evidence about the consequences of testing. Cronbach and Meehl's construct validity paper is the older statement of the same idea.

Publishers summarize their evidence in lists. Pearson's WAIS-5 "By the numbers" sheet names validity studies against the WAIS-IV, the WISC-V, the DAS-II NU School-Age, the KBIT-2 Revised, the WIAT-4, the WRAML3 and the PPVT-5 and EVT-3, and special group studies for 11 groups: intellectual giftedness, mild and moderate intellectual disability, specific learning disorder in reading and in mathematics, ADHD, autism spectrum disorder, moderate or severe traumatic brain injury, mild cognitive impairment, probable dementia of the Alzheimer's type of mild severity, personal protective equipment, and English language learners. The Woodcock-Johnson V abstract reports a concurrent validity sample of 639 people, 66 of whom overlapped with the norming study, and a clinical validity sample of 369. After norming data collection was complete, it also applied a three-stage structural validity framework to the 5,837 norming cases, with cross-validation samples, to settle the final organization of the tests and clusters. These lists tell a reader which questions the developers asked. They do not say how the answers came out, which is why the full manuals matter.

The most informative validity evidence is the kind an outside team can recheck. Canivez, Watkins, McGill and Dombrowski published an independent structural validity analysis of the WAIS-5 in the journal Assessment in 2026, using the correlation matrix and descriptive statistics of the 2,020 person standardization sample as printed in the manual. They report that the manual did not include exploratory factor analyses, confirmatory models with fewer than five group factors, rival bifactor models, or model-based reliability and dimensionality estimates, and that their own analyses did not support five factors with separate Visual Spatial and Fluid Reasoning factors. A four-factor model with those two merged into the former Perceptual Reasoning factor, with measurement dominated by a general factor, was supported in their analysis, as it was for the WAIS-IV. They also note that they could not test whether the structure holds across sex, race and ethnicity or socioeconomic groups, because the publisher has not shared the standardization data.

This page does not adjudicate the disagreement: the paper is one team's analysis of published summary statistics, and the publisher's score structure stands as the one it reports. What the episode shows is how validity evidence works. A claim made in a manual becomes testable when the numbers behind it are published, and a test whose structure has been analyzed by people with no stake in it carries more information than one that has not. The page on index score differences shows what that structure means for reading a profile, and the page on the WAIS-5 describes the battery itself.

10 How Long Do Norms Last, and Why Are Tests Revised?

Norms age, so a test is a dated product, and the revision cycle that replaces it takes years of work. The Wiley chapter on the Stanford-Binet Fifth Edition says the test was published after a 7-year revision project, and it notes that the 1937 revision also took 7 years to create. The 1937 revision was standardized on 3,200 examinees, and the 1905 Binet-Simon scale, from which the line descends, consisted of 30 items. The page on the history of IQ testing covers that lineage. In the other direction, Winter, Trudel and Kaufman say the WAIS-IV was normed in 2007 and 2008 and the WAIS-5 between February 2023 and January 2024, an interval of about 15 years (our arithmetic).

Why not leave the norms alone? Standard 5.11 says that if a publisher provides norms, then as long as the test remains in print it is the publisher's responsibility to renorm it with sufficient frequency to permit continued accurate and appropriate score interpretations. The Standards comment adds that publishers should ensure up-to-date norms are available or provide evidence that older norms remain appropriate. The American Psychological Association's Guidelines for Psychological Assessment and Evaluation, approved in March 2020, make the same point from the user's side: Guideline 8 calls the agedness of the norms and the continued relevance of the constructs being measured important considerations in test selection.

The reason is that performance on intelligence tests has drifted. Pietschnig and Voracek's meta-analysis of one century of global IQ gains covers 271 independent samples, almost 4 million participants and 31 countries from 1909 to 2013. It estimates annual gains of 0.41, 0.30, 0.28 and 0.21 IQ points for fluid, spatial, full-scale and crystallized test performance, and reports that gains have decreased in more recent decades. At 0.28 a year, the Full Scale figure would be about 2.8 points per decade (our arithmetic). A direct comparison of the two latest Wechsler editions can show how large the drift is in a specific case. Winter and colleagues report a counterbalanced study of 186 nonclinical people in which the mean Full Scale IQ was 101.6 on the WAIS-IV and 99.7 on the WAIS-5, a difference of 1.9 points, and they describe the pattern as a Flynn effect of 1.2 IQ points per decade. A difference of that size is small beside the interval of about 4.5 points either side computed above for another battery, and it can still move a person across a cutoff when scores are used for decisions. The page on the Flynn effect and the page on the WAIS-IV to WAIS-5 change cover the evidence and its interpretation.

11 What Does a Test Built in a Weekend Leave Out?

A quickly assembled test can match a professional test on format and still lack the documents that let a reader judge what its score means, and those documents are the product of the steps above. Table 2 turns the steps into questions that any reader can put to any test, using the standards each step is held to.

A fair comparison needs a counterexample, and the International Cognitive Ability Resource is a useful one. Condon and Revelle describe the development and initial validation of this public-domain measure in the journal Intelligence in 2014. Their abstract says the first four item types were evaluated with both an offline university sample and a large online sample, and that concurrent and discriminative validity analyses suggest the public-domain status of these item types did not compromise their validity despite administration to 97,000 participants. A free or inexpensive instrument can therefore be documented and studied in the open, and the page on the ICAR test describes what it measures. Cost and price are not the dividing line. The dividing line is whether the development steps are written down and can be checked.

What a weekend can produce is an item set and a score. What it usually cannot produce is a tryout sample, a norm sample collected over many months, reliability tested after weeks, or outside replication. The Woodcock-Johnson V took four concurrent studies and 8,209 cases over two years. That is not an argument that only large publishers can build good tests, since smaller projects can document each step at smaller scale. It is an argument that the absence of documentation is itself information. The page on whether online IQ tests are accurate and the page on free versus validated IQ tests apply these questions to products. ACIS documents its own development in its technical manual, and this page does not summarize that document, so a reader can apply the same table to it.

12 How Should a Reader Use the Documentation Behind a Score?

The Standards for Educational and Psychological Testing and the American Psychological Association's assessment guidelines ask users to read a score against its documentation, which means checking the norm group and its date, the reliability and interval, the conditions of administration and the stated purpose before relying on a number. The Standards for Educational and Psychological Testing (AERA, APA and NCME, 2014) put the burden on both sides. Standard 4.0 asks developers to document design and development steps so that fairness, reliability and validity can be judged for the intended examinee population, Standard 5.8 asks that norms refer to clearly described populations, and Standard 5.9 asks for the sampling, weighting and dates. The APA Guidelines, approved in March 2020, ask psychologists in Guideline 6 to select assessment tools that demonstrate sufficient validity evidence for their uses, sufficient score reliability and sound psychometric properties, and measures that are fair and appropriate for the purpose, population, setting and context. Guideline 8 asks them to be aware of the need for the appropriate normative comparison.

Five checks follow from that, and each corresponds to a step on this page. First, find the norm group and the year of its data collection, and be wary of a norm with no date. Second, find the reliability for the score you are reading and the age group you belong to, and look for an interval rather than a single point. Third, find the purpose the evidence supports and use the score only for that purpose. Fourth, look at the conditions: a score obtained at home, unsupervised, is not the same event as a score from a psychologist's office, and the page on professional versus online IQ tests sets out the difference. Fifth, ask who may administer the instrument; the page on who can administer an IQ test explains the qualification levels publishers set.

We sell a paid assessment, so treat this paragraph as a disclosure and check it against the rest of the page. ACIS is an online, self administered test with 20 subtests in six domains. It reports a Full Scale IQ and six primary indices on the standard scale with a mean of 100 and a standard deviation of 15, with scaled subtest scores from 1 to 19, a percentile and a 95 percent confidence interval, against adult norms for ages 16 to 90. The prices read on October 13, 2026 are 15 dollars for the Quick form (6 subtests in 3 domains, about 45 minutes), 30 dollars for the Optimized form (13 subtests in 5 domains, about 110 minutes) and 50 dollars for the Full Scale form (all 20 subtests, about 175 minutes), each a one time payment with no subscription, a free trial with no card, a 5 day quality guarantee and 30 days to complete. It is unsupervised, it is not a clinical or diagnostic instrument, it is not for hiring, school accommodations or admission to high IQ societies, and it is available in English only. A reader who wants to apply the questions in Table 2 to ACIS can start from the technical manual.

13 Sources Behind This Page

Every figure on this page traces to one of the sources below, or is labeled as our own arithmetic. The arithmetic is limited to the interval half width of about 4.5 points computed from a published standard error, the 2.8 points per decade restatement of an annual estimate, and the interval of about 15 years between two norming periods. Documents on publisher sites were opened on October 13, 2026. The full technical manuals of the WAIS-5 and the Stanford-Binet Fifth Edition are commercial products and were not opened, so figures about them come from the secondary documents named below.

• American Educational Research Association, American Psychological Association and National Council on Measurement in Education. Standards for Educational and Psychological Testing. AERA, 2014, Standards 4.0 to 4.3, 4.7 to 4.10, 4.15, 4.18, 5.8, 5.9, 5.11 and 5.16, and the chapter on validity.
• American Psychological Association, APA Task Force on Psychological Assessment and Evaluation Guidelines. APA Guidelines for Psychological Assessment and Evaluation, approved by the APA Council of Representatives in March 2020, Guidelines 6 and 8.
• LaForte E M, Dailey D and McGrew K S. WJ V Technical Abstract. Riverside Assessments, 2025.
• Pearson. WAIS-5 "By the numbers", pearsonassessments.com, and the WAIS-5 overview brochure, pearsonassessments.com, 2024 to 2025.
• Winter E L, Trudel S M and Kaufman A S. Wait, where's the Flynn effect on the WAIS-5? Journal of Intelligence, 2024, volume 12, issue 11, article 118.
• Canivez G L, Watkins M W, McGill R J and Dombrowski S C. Construct validity of the WAIS-5: Complementary exploratory and confirmatory factor analyses of the 20 primary and secondary subtests. Assessment, 2026, online first.
• Essentials of Stanford-Binet Intelligence Scales (SB5) Assessment, chapter 1, chapter excerpt, John Wiley and Sons, reproducing figures from Roid G H, Stanford-Binet Intelligence Scales, Fifth Edition, Technical Manual, Riverside Publishing, 2003.
• Abdelhamid G S M, Bassiouni M G A and Gómez-Benito J. Assessing cognitive abilities using the WAIS-IV: An item response theory approach. International Journal of Environmental Research and Public Health, 2021, volume 18, issue 13, article 6835.
• Haladyna T M, Downing S M and Rodriguez M C. A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 2002, volume 15, issue 3, pages 309 to 333.
• Reise S P, Ainsworth A T and Haviland M G. Item response theory. Current Directions in Psychological Science, 2005, volume 14, issue 2, pages 95 to 101.
• Swaminathan H and Rogers H J. Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 1990, volume 27, issue 4, pages 361 to 370.
• Cronbach L J. Coefficient alpha and the internal structure of tests. Psychometrika, 1951, volume 16, issue 3, pages 297 to 334.
• Cronbach L J and Meehl P E. Construct validity in psychological tests. Psychological Bulletin, 1955, volume 52, issue 4, pages 281 to 302.
• Pietschnig J and Voracek M. One century of global IQ gains. Perspectives on Psychological Science, 2015, volume 10, issue 3, pages 282 to 306.
• Condon D M and Revelle W. The international cognitive ability resource: Development and initial validation of a public-domain measure. Intelligence, 2014, volume 43, pages 52 to 64.

14 Frequently Asked Questions