Using machine learning to infer reasoning provenance from user interaction log data: based on the data/frame theory of sensemaking

Kodagoda, Neesha and Pontis, Sheila and Simmie, Donal and Attfield, Simon and Wong, B. L. William and Blandford, Ann and Hankin, Chris (2017) Using machine learning to infer reasoning provenance from user interaction log data: based on the data/frame theory of sensemaking. Journal of Cognitive Engineering and Decision Making, 11 (1). pp. 23-41. ISSN 1555-3434

[img]
Preview
PDF - Final accepted version (with author's formatting)
Download (980kB) | Preview

Abstract

The reconstruction of analysts’ reasoning processes (reasoning provenance) during complex sensemaking tasks can support reflection and decision making. One potential approach to such reconstruction is to automatically infer reasoning from low-level user interaction logs. We explore a novel method for doing this using machine learning. Two user studies were conducted in which participants performed similar intelligence analysis tasks. In one study, participants used a standard web browser and word processor; in the other, they used a system called INVISQUE (Interactive Visual Search and Query Environment). Interaction logs were manually coded for cognitive actions based on captured think-aloud protocol and posttask interviews based on Klein, Phillips, Rall, and Pelusos’s data/frame model of sensemaking as a conceptual framework. This analysis was then used to train an interaction frame mapper, which employed multiple machine learning models to learn relationships between the interaction logs and the codings. Our results show that, for one study at least, classification accuracy was significantly better than chance and compared reasonably to a reported manual provenance reconstruction method. We discuss our results in terms of variations in feature sets from the two studies and what this means for the development of the method for provenance capture and the evaluation of sensemaking systems.

Item Type: Article
Research Areas: A. > School of Science and Technology > Computer Science
Item ID: 22104
Notes on copyright: Neesha Kodagoda, Sheila Pontis, Donal Simmie, Simon Attfield, B. L. William Wong, Ann Blandford, Chris Hankin, Using Machine Learning to Infer Reasoning Provenance From User Interaction Log Data: Based on the Data/Frame Theory of Sensemaking, Journal of Cognitive Engineering and Decision Making, Vol 11, Issue 1, pp. 23 - 41. Copyright © 2016 (Human Factors and Ergonomics Society). Reprinted by permission of SAGE Publications.
Useful Links:
Depositing User: Simon Attfield
Date Deposited: 07 Mar 2018 18:02
Last Modified: 14 Sep 2018 21:10
URI: http://eprints.mdx.ac.uk/id/eprint/22104

Actions (login required)

Edit Item Edit Item

Full text downloads (NB count will be zero if no full text documents are attached to the record)

Downloads per month over the past year