Research desk · Questions and evidence
Start with a question.
14 answers about buying hiring software and interpreting AI's effects on work. Each keeps its source and limitations attached.
AI hiring procurement evidence
Does an AI assessment still need job-related validation?
SHL's AI principles require reliability, validity and job relevance for AI assessments, as for other assessments. The document also treats explainability as a design requirement.
Limit: Principles alone do not establish predictive validity, accessibility or subgroup performance for your assessment, language and applicant population.
Next check: Request the technical manual, validation cohort, norm group, accommodation process and monitoring plan for the exact instrument and scoring version.
Open the evidence recordWhich input and output does the assessment actually use?
HireVue publishes an AI explainability statement describing its assessment approach. Treat that statement as a starting point for tracing candidate inputs, scoring and human decisions.
Limit: A public explanation does not verify the current model, enabled modalities, score use or retention settings in a prospective deployment.
Next check: Ask for the current statement, exact input modalities, model version, candidate notice, deletion controls and how an assessor can contest or override an output.
Open the evidence recordWho defines the criteria used to review applications?
Ashby documents job-specific resume criteria for AI-assisted application review, with organization-admin enablement and role-level permissions for editing criteria.
Limit: A resume can omit a skill the candidate has. Criteria matching is not a validated measure of future job performance.
Next check: Test ambiguous and incomplete resumes; identify who can change criteria, how changes are logged and whether a reviewer sees the underlying evidence before rejecting anyone.
Open the evidence recordDoes the audit cover the feature you plan to enable?
Greenhouse's AI principles page says Talent Matching and AI Interviewer undergo independent monthly bias audits by Warden AI. This record captures the vendor's claim, not a review of those audit reports.
Limit: An audit cadence does not specify the dataset, tested model version, selection thresholds, exceptions or applicability to a customer.
Next check: Obtain the latest feature-specific report, auditor identity, test dates, sample sizes, excluded groups and documented limitations. Compare them with the deployment you are buying.
Open the evidence recordCan one employer's audit stand in for yours?
Workday publishes a Secretariat analysis of its own Spotlight deployment. Testing completed March 20, 2026; the stated applicant period is September 1, 2025 through February 28, 2026.
Limit: Workday explicitly limits the analysis to its implementation and says the page does not satisfy a customer-specific legal obligation. It is not a universal fairness certificate.
Next check: Compare your job mix, thresholds, candidate pool and human review with the reported scope; request evidence for the currently deployed model and your own use case.
Open the evidence recordWhich contracted module performs the promised workflow?
UKG publishes separate service descriptions for Pro HCM, Pro Workforce Management, Ready and AI-related services. A product-family name is not a complete statement of the purchased scope.
Limit: A feature appearing in the portfolio does not prove it is included, configured, supported in a country or connected to the employer's ATS.
Next check: Map each promised action to a module, order-form line, implementation owner, data integration, country scope and acceptance test.
Open the evidence recordAI and employment evidence
Can AI assistance improve measured output at work?
The published study reports a 15% average increase in issues resolved per hour after AI assistance was introduced, with different effects across worker experience levels.
Limit: Issues resolved per hour is a task-productivity outcome in one setting, not an economy-wide employment, wage or headcount estimate.
Next check: Compare the work and experience mix with your team; count quality and escalations alongside throughput.
Open the evidence recordDid early-2025 coding tools always save time?
METR's randomized study found that permitting AI tools increased task-completion time by 19% in its sampled experienced open-source developers.
Limit: This result is not a ranking of current coding products and does not represent all developers, unfamiliar repositories or later models.
Next check: Read the 2026 follow-up before citing this as evidence about current tools; measure accepted changes and review time locally.
Open the evidence recordCan the follow-up settle the current speedup?
METR says its follow-up provides an unreliable estimate of current productivity effects because of participant and task selection, plus measurement difficulties with concurrent agents.
Limit: Raw follow-up estimates should not be advertised as a clean causal estimate or a definitive reversal of the earlier result.
Next check: Look for revised study designs and report who declined AI-disallowed tasks. Count concurrent-agent time consistently.
Open the evidence recordDoes potential exposure mean jobs disappear?
ILO and NASK estimate that about one in four jobs worldwide is in occupations potentially exposed to generative AI; the source distinguishes transformation from replacement.
Limit: Potential technical exposure is not observed adoption, feasible whole-job automation or a forecast of layoffs.
Next check: Pair exposure with actual adoption, wages, employment and transitions in the geography being discussed.
Open the evidence recordWhat can an exposure indicator predict on its own?
ILO's 2026 methodological brief warns against reading exposure measures alone as predictions of job losses or labor-market outcomes.
Limit: The warning does not imply no displacement; it specifies what this type of indicator cannot establish.
Next check: Identify which result is an exposure score and which uses actual labor-market outcomes before combining studies.
Open the evidence recordAre employment changes concentrated in particular groups?
Stanford's August 2026 revision reports a 19% employment gap for young workers in highly AI-exposed occupations relative to less-exposed peers, while reporting no widespread economy-wide displacement.
Limit: The authors describe these as non-causal patterns, with education, pre-trends and representativeness caveats. The gap is not AI-caused layoffs, an unemployment rate or 19% of all young workers losing jobs.
Next check: Check the paper revision, cohort definition, comparison group and robustness checks before making a causal or national-population claim.
Open the evidence recordDo reported task benefits show up in average labor outcomes?
The March 2026 NBER revision estimates null effects on earnings and recorded hours at worker and workplace levels, ruling out effects larger than 2% in its study horizon.
Limit: Null average earnings and hours effects do not mean no task change, no benefit for individual adopters or no future employment effects elsewhere.
Next check: Use the revised title and estimates; distinguish the measured horizon from the publication date and the earlier version's estimates.
Open the evidence recordWhat does usage telemetry say about employment?
Anthropic's June 2026 report changes its sampling and classification pipeline to study hourly use and outputs, with separate reporting for Claude conversations and first-party API traffic.
Limit: Claude usage is not a representative sample of all workers, a verified task-success census or a causal measure of jobs created or lost.
Next check: Read the methodology appendix before comparing report editions; keep platform use, self-reported benefits and administrative employment outcomes separate.
Open the evidence record