As machine learning models grow progressively complex – with modern large language models containing billions of parameters – traditional analytical approaches to explaining their decision-making processes have become increasingly complicated. This has driven a shift towards studying these systems through their observable behaviour, rather than their internal mechanisms. Grounded in the formal statistical modelling of unobservable psychological constructs (e.g., mental processes, decision-making, mental illness, bias), psychometric methods offer a rich tradition of principled methodological frameworks for the behavioural approach of explaining AI systems. By modelling the systems as “black box minds” with measurable traits, this approach treats AI decision-making not just as output to be interpreted post-hoc, but as a result of latent model mechanisms and processes that affect behaviour in downstream tasks, such as the unintended perpetuation of social biases or subtle misaligned behaviours. The field of psychometrics provides established techniques for quantifying and measuring these latent traits in AI systems, which can enable the development of diagnostic XAI methods to identify specific traits (e.g., high-risk-seeking behaviour) and corrective techniques that utilise these diagnostics to guide model refinement, such as targeted debiasing or alignment-tuning interventions.
Beyond explaining AI systems themselves through psychological frameworks, psychometric methods can also be used to evaluate the success of explanations across a range of domains. While a large number of XAI techniques have been successfully implemented, the interaction between explanations and human end-users remains an open measurement challenge. Given that explanatory processes function as an exchange between the explainer and the explainee, a successful explanation must be targeted towards and evaluated by the user. As such, explanations need not only be faithful to the system but also provide measurable utility to the explainee. To capture the interaction between the XAI system and the user, psychometric measurement theory provides the necessary foundation, requiring formal operationalisation of constructs through validated instruments and measurement models, rather than ad hoc usability checks. This measurement-theoretic approach is particularly appropriate for this problem, providing opportunities not only to formally model the success of explanations, but also to tailor explanations to the knowledge base of the target group, thereby improving the explanatory process (e.g., through adaptive explanation selection).
As such, psychometric methods offer a novel approach to both the creation and evaluation of machine learning explanations, opening new pathways to increase transparency and auditability.
Keywords: psychometrics, model behaviour, latent traits, algorithmic bias, model alignment, model refinement, human-in-the-loop evaluation, user understanding, adaptive explanations, user-tailored XAI, explanation utility
Topics
- Applying psychometric techniques to evaluate machine learning explanations.
- Measuring and integrating user understanding into the ML workflow.
- Developing and validating scales to assess the quality of explanations.
- Analysing and refining XAI human-computer interaction through the integration of quantitative, validated user feedback.
- Explain AI systems using formal models from psychology.
- Integrate user feedback into the XAI workflow through psychometric methods.
- Evaluate and compare existing explanation techniques through user feedback.
- Human-in-the-loop evaluation practices to refine explanation quality.
- Acquiring user feedback to promote the generation of understandable explanations.
- Identification of latent model mechanisms that affect downstream behaviour.
- Auditing training datasets for latent biases.
- Validation of psychometric measurement frameworks for XAI.
- Validation of psychometric materials for XAI.
- Robustness of psychometric measurement frameworks for XAI.
- Robustness of psychometric materials for XAI.
- Psychometrics-guided refinement of AI system behaviour.
- Psychometric analysis of embeddings.
- Investigation of models’ latent space representations.
- Psychometric approaches for emerging capabilities and/or traits in generative AI systems.
