A guide to measuring expert performance in forensic pattern matching

Samuel G Robson; Rachel A Searston; Matthew B Thompson; Jason M Tangen

doi:10.3758/s13428-024-02354-y

Back

A guide to measuring expert performance in forensic pattern matching

Journal article

Open access

Peer reviewed

A guide to measuring expert performance in forensic pattern matching

Samuel G Robson, Rachel A Searston, Matthew B Thompson and Jason M Tangen

Behavior research methods, Vol.56(6), pp.6223-6247

2024

DOI: https://doi.org/10.3758/s13428-024-02354-y

PMID: 38485882

Files and links (1)

pdf

Published2.80 MBDownload View

CC BY V4.0, Open Access

Abstract

Decision-making

Fingerprints

Forensic science

Proficiency tests

Forensic pattern matching

Expertise

Signal detection

Decisions in forensic science are often binary. A firearms expert must decide whether a bullet was fired from a particular gun or not. A face comparison expert must decide whether a photograph matches a suspect or not. A fingerprint examiner must decide whether a crime scene fingerprint belongs to a suspect or not. Researchers who study these decisions have therefore quantified expert performance using measurement models derived largely from signal detection theory. Here we demonstrate that the design and measurement choices researchers make can have a dramatic effect on the conclusions drawn about the performance of forensic examiners. We introduce several performance models – proportion correct, diagnosticity ratio, and parametric and non-parametric signal detection measures – and apply them to forensic decisions. We use data from expert and novice fingerprint comparison decisions along with a resampling method to demonstrate how experimental results can change as a function of the task, case materials, and measurement model chosen. We also graphically show how response bias, prevalence, inconclusive responses, floor and ceiling effects, case sampling, and number of trials might affect one’s interpretation of expert performance in forensics. Finally, we discuss several considerations for experimental and diagnostic accuracy studies: (1) include an equal number of same-source and different-source trials; (2) record inconclusive responses separately from forced choices; (3) include a control comparison group; (4) counterbalance or randomly sample trials for each participant; and (5) present as many trials to participants as is practical.

Details

Title: A guide to measuring expert performance in forensic pattern matching
Authors/Creators: Samuel G Robson - School of Psychology, The University of New South Wales, Kensington, NSW, Australia. sam.robson@unsw.edu.au
Rachel A Searston - University of Adelaide
Matthew B Thompson - Murdoch University
Jason M Tangen - The University of Queensland
Publication Details: Behavior research methods, Vol.56(6), pp.6223-6247
Publisher: Springer Nature
Identifiers: 991005650407307891
Murdoch Affiliation: Centre for Biosecurity and One Health
Language: English
Resource Type: Journal article

UN Sustainable Development Goals (SDGs)

This output has contributed to the advancement of the following goals:

#16 Peace, Justice and Strong Institutions

Metrics

27 File views/ downloads

99 Record Views

2 Times Cited - Web of Science

InCites Highlights

These are selected metrics from InCites Benchmarking & Analytics tool, related to this output

Collaboration types: Domestic collaboration
Citation topics: 2 Chemistry; 2.244 Chemometrics; 2.244.1784 Forensic Spectroscopy
Web Of Science research areas: Psychology, Experimental; Psychology, Mathematical
ESI research areas: Psychiatry/Psychology