Classify Photos
Contributed by:
petam
Characteristics:
Trust Type: Combination; Interaction: Influence; Stage: Changing; Risk: Task Failure; System: Embedded; Test Environment: ITL-Online; Measurement: Behavioural; Self-Reported; Application Domain: Aspects are task-specific; Pattern: None;
Description
The experiment was conducted asynchronously via a mobile app. Participants were told to imagine that they had been hired by an organization focused on creating a database of species seen all over the Philippines. They were to be provided with a recognition AI to help classify any species they encountered. They were given three days to classify species sent to their system. The AI would provide its classification but the participants could input their own if they wished. Each day, the participant was provided 25 random photos via the app. On each trial they provided an evaluation of the AI's usefulness, emotions they felt, and their degree of trust.
Commentary
The experiment is designed to evaluate the effectiveness of XAI over time. Essentially, this is a standard image recognition usecase in which the AI system provides a classification and the participant must decide whether or not to trust its decision. Use of the mobile app to enable asynchronicity is the standout characteristic and the app itself facilitates data collection and temporality. Although in this case it's an online test environment (the mobile is used no differently from a computer), it's possible to imagine alternative implementations using the mobile so that experiments can effectively be performed 'in-the-wild'.
Original purpose
Passage of time (1st, 2nd, 3rd day)
RRI issues
None.
Source
Bernardo, E., Seva, R. (2023). Evaluating the Effect of Time on Trust Calibration of Explainable Artificial Intelligence. In: Tareq Ahram, Jay Kalra and Waldemar Karwowski (eds) Artificial Intelligence and Social Computing. AHFE (2023) International Conference. AHFE Open Access, vol 72. AHFE International, USA.