Classifier + EEG
Contributed by:
YangLu
Characteristics:
Trust Type: Competency-based; Interaction: Influence; Stage: Mis-calibrated; Risk: Task Failure; System: Embedded; Test Environment: ITL-Immersive; Measurement: Physiological; Self-Reported; Application Domain: UC needs special equipment; Pattern: Reliability calibration;
Description
Participants were trained on the system monitoring subtask of the Air Force Multi-Attribute Task Battery (AF-MATB) - a simulator used to evaluate operator performance and mental workload. Participants were told that its algorithms had been developed by 'expert' or 'novice' teams. Wearing an electroencephalogram (EEG) cap, they then monitored 4 fluctuating dials all showing their acceptable range of operation and counted the number of automation failures they observed, or intervened by pressing a button. Data from the EEG was fed to a classifier which estimated their level of trust, which could then be compared with their self-reported trust ratings.
Commentary
Usecase evaluates a convolutional neural network (CNN)-based classifier to estimate likely trust calibration during human-automation interaction. MATB is a benchmark for flight performance checking for pilots but subjects don't need to be pilots for this usecase.
Original purpose
To establish a quantitative method of predicting calibration of user trust, varying reliability (High/Low) and credibility (High/Low).
RRI issues
Potential bias and generalisation of the finding - the applicability of findings to real-world scenarios, such as actual Air Force operations, depends on a representative sample that includes a range of proficiencies and demographic backgrounds.
Source
Choo, S., & Nam, C. S. (2022). Detecting human trust calibration in automation: a convolutional neural network approach. IEEE Transactions on Human-Machine Systems, 52(4), 774-783.