PDFs are provided to ensure timely dissemination of academic work. They can be downloaded for noncommercial, information purposes only, and may not be reposted without permission from the respective copyright holders.
ARTICLES peer-reviewed, archival proceedings
2018
DIAGRAMS
Read It This Way: Scaffolding Comprehension for Unconventional Statistical Graphs
Amy Rae Fox and James Hollan.
In Diagrammatic Representation and Inference. Springer International Publishing. 2018
How do you make sense of a graph that you have never seen before? In this work, we outline the types of prior knowledge relevant when making sense of an unconventional statistical graph. After observing students reading a deceptively simple graph for time intervals, we designed four instructional scaffolds for evaluation. In a laboratory study, we found that only one scaffold (an interactive image) supported accurate interpretation for most students. Subsequent analysis of differences between two sets of materials revealed that task structure–specifically the extent to which a problem poses a mental impasse–may function as a powerful aid for comprehension. We find that prior knowledge of conventional graph types is extraordinarily difficult to overcome.
2016
DIAGRAMS
Exploring Representations of Student Time-Use
Amy Rae Fox, Erica Vries, Laurent Lima.
In Diagrammatic Representation and Inference. Springer International Publishing. 2016
How does one visually represent the use of time? We explored students’ use of graphical metaphors by asking undergraduates at a public French university to generate representations of their personal time-use including: activities, sequence, duration, timing, and frequency. The resulting use of space and form was analyzed by way of an iteratively developed coding scheme. We discuss how the analyses of the spontaneous productions support previous research on spatial representations of time, and the implications for the design of time management tools for students.
Graph comprehension is the act of deriving meaning from graphs, an activity grounded in visuospatial reasoning that develops through a combination of instruction and practice. What we know about the mechanisms of graph comprehension stems from interleaving lines of inquiry in statistics, computer science, education, and psychology dating back to the 1980s. In this integrative review, I describe how models of graph comprehension evolved in response to developments in cognitive theory, offering a critical commentary on how foundational theories build upon each other, extending rather than replacing theoretical claims at different levels of analysis. I illuminate the landscape of contemporary research, before concluding with an argument for the role of visualization psychology in supporting theoretical integration across disciplinary boundaries.
Visualization Psychology: Foundations for an Interdisciplinary Research Programme
What might a discipline of Visualization Psychology look like? If research on the psychological aspects of visualization were to coalesce, in the sense of a Lakatosian research program, what refutation-resistant theoretical commitments would magnetize its “hard core”? In this chapter, we argue that any interdisciplinary inquiry concerned with psychological aspects of visualization should situate its phenomena in the broader context of external representation, as a (triadic) semiotic activity achieved via information processing in a distributed cognitive system.
PROCEEDINGS short papers, workshops, extended-abstracts, non-archival proceedings
2020
PROGRAMMING
Towards a dynamic multiscale personal information space: beyond application and document centered views of information
Amy Rae Fox, Philip Guo, Clemens Nylandsted Klokmose.
In Conference Companion of the 4th International Conference on Art, Science, and Engineering of Programming. Association for Computing Machinery. 2020
The historical moment when a person worked in front of a single computer has passed. Computers are now ubiquitous and embedded in virtually every new device and system, connecting our personal and professional activities to ever-expanding information resources with previously unimaginable computational power. Yet with all the increases in capacity, speed, and connectivity, our experiences too often remain difficult, awkward, and frustrating. Even after six decades of design evolution there is little of the naturalness and contextual sensitivity required for convivial interaction with computer-mediated information. We envision a future in which the existing world of documents and applications is linked to a multiscale personalized information space in which dynamic visual entities behave in accordance with cognitively motivated rules sensitive to tasks, personal and group interaction histories, and context. The heart of the project is to rethink the nature of computer-mediated information as a basis to begin to fully realize the potential of computers to assist information-based activities. This requires challenging fundamental presuppositions that have led to today’s walled gardens and information silos. Our goal is to catalyze an international research community to rethink the nature of information as a basis for radically advancing the human-centered design of information-based work and helping to ensure the future is one of convivial, effective, and humane systems. In this paper, we propose a new view of information systems, discuss cognitive requirements for a human-centered information space, and sketch a research agenda and approach.
WORKSHOP
A Psychology of Visualization or (External) Representation?
Amy Rae Fox
In Proceedings of the Workshop on Visualization Psychology @ IEEE VIS. . 2020
What is a visualization? There is limited utility in trifling with definitions, except insofar as one serves as a tool for communicating and conceptualizing our subject matter; a statement of identity for a community. To establish Visualization Psychology as a viable inter-disciplinary research programme, we must first define the object(s) of our collective inquiry. I propose that while we might refer to the study of "visualization" for the term’s colloquial accessibility and pragmatic alignment with other fields, we should consider for exploration a class of artifacts and corresponding processes more expansive and profound: external representations. What follows is an argument for the study of external representation as the foundation for a new interdisciplinary endeavor, and approach to mapping the corresponding problem space.
WORKSHOP
Surfacing Misconceptions Through Visualization Critique
Amy Rae Fox and Taylor Jackson Scott.
In Proceedings of the Workshop on Visualization Activities @ IEEE VIS. . 2020
Students of visualization come to formal education with an abundance of personal experience. However, one’s exposure to graphics through media and education may not be sufficiently diverse to appreciate the nuance and complexity required to design and evaluate effective representations. While many introductory courses in visualization address best practices for visual encoding of data based on perceptual characteristics, as cognitive scientists, we place equal value on representational decisions based on communicative context: how the representation is intended to be used. In this pedagogical activity, we aim to surface learners’ preconceived notions about what makes a visualization effective. Here we describe the structure and context of an introductory-level visualization activity, how it might be conducted in individual or group settings, our experience with the common misconceptions the activity can reveal, and conclude with recommendations on how they might be addressed.
2019
COGSCI
When Graph Comprehension Is An Insight Problem
Amy Rae Fox, James D Hollan, and Caren M Walker.
In Proceedings of the Annual Conference of the Cognitive Science Society. Cognitive Science Society. 2019
How do you make sense of an unconventional graph? Building on research demonstrating that prior knowledge of graphical conventions is difficult to overcome, we reconstrue graph reading as an insight problem. We hypothesize that imposing a mental impasse during a particular type of graph reading task will improve comprehension by inducing a sense of puzzlement, prompting learners to reconsider their interpretation. We find support for this proposal in a betweensubjects experiment in which participants presented with an impasse-formulated version of graph reading questions are significantly more likely to correctly interpret a graph featuring an unconventional coordinate system. We characterize the differential patterns of mouse movements for learners between conditions and discuss implications for the use of novel graphical forms in science communication.
ARTWORK
The Burden of Selfhood
Amy Rae Fox, Stefani Byrd, Sarah Ciston.
In Proceedings of the 2019 Conference on Creativity and Cognition. Association for Computing Machinery. 2019
The Burden of Selfhood is an interdisciplinary performance artwork exploring the intersection of feminism, identity and technology. By connecting methods from cognitive science, music, poetry, video and performance art, we investigate the experience of viewing and being viewed as a gendered body. Technology has accelerated the recursive gaze to the point that we continually perform and project back onto each other our internalized expectations for unattainable perfection. This poly-vocal performance uses large-scale data visualizations and live performers to make explicit both the collective gaze and the implicit impact of being seen. Select portions of the performance can be found at: https://www.youtube.com/watch?v=UPk2JSt-e9Q https://www.youtube.com/watch?v=j1DXmiBU_3w
2018
EARLI
Graphical Insight: How to Read an Unconventional Graph
Amy Rae Fox, Caren M Walker, and James Hollan.
In Proceedings of the European Association of Research on Learning & Instruction — SIG2 Comprehension of Text and Graphics. . 2018
How do you make sense of a graph that you have never seen before? Building on recent work demonstrating that prior knowledge of conventional graph types is extraordinarily difficult to overcome, we explore the use of implicit scaffolding to reconstruct graph reading as an insight problem. We hypothesize that constructing a mental impasse will improve learner performance by increasing the probability learners will reconsider their default interpretation strategies and recognize alternative interpretations of novel graphical forms. In a between-subjects laboratory experiment we find support for this hypothesis. Analysis of qualitative data suggests promising directions for understanding graphical intuitions, and we conclude with suggestions for future work that address the timing of mental model formation for unconventional graphic forms.
2016
COGSCI
Representing Sequence: The Influence of Timeline Axis and Direction on Causal Reasoning in Litigation Law
Amy Rae Fox and Martin Van Den Berg.
In Proceedings of the Annual Meeting of the Cognitive Science Society. Cognitive Science Society. 2016
Can the representation of event sequence influence how jurors remember and reason in a legal case? We addressed this ques-tion by examining the interaction between an individual’s pre-ferred spatial construal of time (SCT) for an external (visual-spatial) representation and the SCT of a courtroom graphic. One hundred fifty three undergraduates played the role of ju-rors in a fictitious civil trial. The details of a case were re-counted in a multimedia presentation featuring timelines an-imated in one of four orientations: left-right, right-left, top-bottom, and bottom-top. Participants were assessed on mea-sures of comprehension and causal reasoning. Results indi-cated effects of timeline orientation and SCT choice behav-ior on comprehension and reasoning. We discuss these results in terms of the role of attention in temporal-causal reasoning, and implications for the design of multimedia materials for the courtroom.
COLLOQUIA colloquia, symposia
2018
SYMPOSIUM
Overcoming Graphical Fixedness: Scaffolding Comprehension for Unconventional Graphs
How do we make sense of a graph we’ve never seen before? Kosslyn [1] suggests we instantiate a hierarchically-organized graph schema. But what schema is triggered for novel representations? Pinker [2] speculates readers instantiate a “general graph schema” likely based on the coordinate system and “predominate graphical forms”. But what information does this schema contain, and how does it interact with prior knowledge of other graphical forms? Here we investigate the graph schema by exploring how learners read an unconventional representation. We ask: (1) What strategies do learners employ to make sense of an unconventional graph? (2) What explicit information might scaffold (self-directed) comprehension? (3) Can structuring graph-reading tasks as insight problems help overcome contradictory prior knowledge? (4) What is the time-course of mental model formation? (5) What inscriptions serve as the graphical framework, triggering a particular graph schema?
SYMPOSIUM
Stepping Closer To A Science of Interaction: A Paradigm for Studying the Cognitive Mechanisms of Interaction
Interaction is crucial for effective visualization. Yet, most of our research centers on representations of information, and the extent to which they support human operators performing well-specified tasks. Despite impressive progress developing tools for interaction, our understanding of how to employ these techniques is based largely on intuition and ad-hoc evaluation from disparate research communities. What happens—in the mind and body—when a user “interacts” with a visualization? In this dissertation I aim to develop a paradigm for studying interaction with representations in the same principled way we have treated visual encodings. Such a paradigm necessarily involves multiple levels of analysis, and seeks to bridge a functional gap between the study of cognition and the study visualization. Only once we understand how and why different interaction techniques are effective can we offer predictive guidelines for the design of human-information systems
In cognitive science, diagrams are often treated as tools for thinking. They may be both reflective of, and active in, our cognitive activities. Unfortunately, our tools for analyzing diagram construction are limited and often difficult to scale. In this brief I will introduce a potential line of research utilizing digital pen systems to explore the production of diagrams. I will discuss the affordances of digital pen technologies and how these might be utilized in diagram research. Finally, I will discuss potential research questions and directions for future work.
POSTER PRESENTATIONS short papers and abstracts presented as posters
2023
POSTER
Can Diamond Plots Mitigate Causal Inferences from Correlational Data?
Amy Rae Fox
In Proceedings of the Annual Meeting of the Psychonomics Society. . 2023
Empirical evidence in graph comprehension indicates that humans often draw causal conclusions from common data visualizations like bar charts and scatterplots, even when labels indicate such inferences are unwarranted due to the research design. In 2018, a widely-circulated paper shared on arXiv proposed that a new graphical formalism: Diamond Plots (scatterplots rotated 45 degrees CCW) may ameliorate this problem by challenging a convention ostensibly implied by the Cartesian coordinate system that variables depicted on the x-axis cause changes in variables depicted on the y-axis. In a series of experiments, we test this hypothesis by comparing how scatterplots and diamond plots support appropriate correlational vs. causal inferences and estimation of correlation magnitude and direction. We find that although this new visualization technique may yield a small improvement in causal inference, it also yields a detrimental effect on confidence and accuracy of correlation judgements. Our results suggest that improving reasoning with data visualizations requires more than innovation in graphical formalisms but also an understanding of what individual and situational factors cause unwarranted inferences in the first place.
POSTER
Diamond Plots — An Evaluation
Amy Rae Fox
In Presented at the Gordon Research Conference on Visualization in Science and Education. . 2023
Empirical evidence in graph comprehension indicates that humans often draw causal conclusions from common data visualizations like bar charts and scatterplots, even when labels indicate such inferences are unwarranted due to the research design. In 2018, a widely-circulated paper shared on arXiv proposed that a new graphical formalism: Diamond Plots (scatterplots rotated 45 degrees CCW) may ameliorate this problem by challenging a convention ostensibly implied by the Cartesian coordinate system that variables depicted on the x-axis cause changes in variables depicted on the y-axis. In a series of experiments, we test this hypothesis by comparing how scatterplots and diamond plots support appropriate correlational vs. causal inferences and estimation of correlation magnitude and direction. We find that although this new visualization technique may yield a small improvement in causal inference, it also yields a detrimental effect on confidence and accuracy of correlation judgements. Our results suggest that improving reasoning with data visualizations requires more than innovation in graphical formalisms but also an understanding of what individual and situational factors cause unwarranted inferences in the first place.
POSTER
Empirical Construct Diagrams: Diagramming the Structure of Variables in Empirical Research Studies
Amy Rae Fox
In Presented at PsychTERMS — Teaching to Enhance Research Methods & Statistics in Psychology . . 2023
Data visualizations facilitate knowledge construction about data, both in the classroom and research practice. But we lack similarly powerful tools for representing the conceptual structure of the studies that generate these data. In this poster I present my adaptation of a diagrammatic notation first developed for software systems [IDEF0]. Through examples, I demonstrate how this new diagram makes explicit an empirical study’s: central theoretical construct, variables that are systematically varied, structurally or statistically controlled, left uncontrolled, or measured, and how each variable is operationalized. I further demonstrate how the diagram can be used as an instructional aid or formative assessment in methods and content courses when teaching students how to read and interpret experimental research.
2022
POSTER
Graphical Discovery: Transitions in Understanding a Novel Coordinate System
Amy Rae Fox, James D Hollan, and Caren M Walker.
In Proceedings of the Annual Meeting of the Psychonomics Society. . 2022
— How do you make sense of an unfamiliar coordinate system? Building on recent work demonstrating that prior knowledge of conventional data visualizations is extraordinarily difficult to overcome, we explore the use of implicit scaffolding to reconstruct graph reading as an insight problem. We hypothesize that constructing a mental impasse will improve learner performance on an unconventional graph reading task by increasing the probability learners will reconsider their default strategy and recognize alternative interpretations of novel graphical forms. We find support for this hypothesis in a between-subjects laboratory experiment testing comprehension of a statistical graph with an unconventional coordinate system. Subsequent analyses of mouse-tracking data suggest promising directions for understanding primitives of graphical intuitions.
2017
POSTER
How Can we Scaffold Comprehension for Unconventional Graphs?
Amy Rae Fox and James D Hollan.
In Presented at the Gordon Research Conference on Visualization in Science and Education. . 2017
Previous work on scaffolding graph comprehension has focused on connecting the features of a graph to its domain content (Mautone & Mayer, 2007). But what happens when we encouter a graph that is completely unfamiliar? The two graphs below represent the same information about a schedule of events. They are informationally equivalent: all data (start, end and duration) that can be read from one graph can be read from the other. But the graph on the left uses an unconventional coordinate system where a single point has two intersections with the x-axis —neither of which are orthogonal, as we expect in a Cartesian coordinate system. The Triangular Model of Interval Relations was introduced by a team of GIS researchers (Qiang, Vaicke, De Maeuer, Van de Weghe, 2014) looking for a more efficient way to represent large data sets of intervals. It is an alternative to the conventional representation for intervals in this domain —the linear model.
2015
EARLI
Dynamic vs. Static Visualizations for Learning Procedural and Declarative Information
Savannah Loker, Erica Vries, and Amy Rae Fox
In Conference of the European Association for Research on Learning and Instruction (EARLI). . 2015
The present investigation addresses the use of static vs. dynamic visualizations for learning declarative and procedural information under the topic domain of computer networking. One hundred and ten undergraduates were exposed to both procedural and declarative instructional material either in the form of static visualizations or dynamic visualizations. Participants were tested twice on both declarative and procedural information, once before exposure to the instructional material to measure baseline knowledge and once after exposure to measure information gain. The format of each test matched the nature of the information presented. The declarative test took the form of open-ended questions; the procedural test was a performance test. The results of this study indicated effects of visualization type for learning a procedure. Results are discussed in terms of multimedia learning, and applied to the design of multimedia materials
AERA
Constructing knowledge from interactive visualizations: How conceptual understanding influences representational choice
Ulrich Ludewig, Neil Schwartz, Neil Jacobson.
In AERA — Annual Meeting of the American Educational Research Association. . 2015
Users often prefer representations containing task-irrelevant information (Smallman & Cook, 2011) even when they are experts (Hegarty, Smallman, Stull, & Canham, 2009). Representational choice, a component of meta-representational competence (diSessa, 2004), may also be influenced by conceptual understanding of what the display represents. Canham and Hegarty (2010) stated that an individual’s conceptual understanding should enhance their focus on task-relevant information. We argue that conceptual understanding enhances a focus on task-relevant information and accordingly requires less deselection of task-irrelevant information. Therefore, this study aimed to investigate the influence of an individual’s conceptual understanding on representational choice. Two factors, Essay Task (content, surface) and Initial Display (overview, blank) were crossed yielding a 2 x 2 between-subjects design with two dependent variables: Representational choice and Performance. Participants viewed an Interactive Data Visualization (IDV) depicting indicators of human development and completed an essay task by conceptually interpreting the data (content) or by only describing the display’s functions (surface). To use the IDV, users had to either (1) deselect task-irrelevant information from an overview initial-display or (2) to select task-relevant information from a blank initial-display. Representational choice was measured as the amount of task-irrelevant data chosen to answer graph-reading questions. Performance was measured as time spent on correctly answered graph-reading questions. Ninety-five undergraduates participated in the study. Participants began by developing domain knowledge by reading a passage about historical events and were assigned to one of two essay tasks (content, surface). Finally, they completed 12 graph-reading questions consisting of multiple-choice questions, using the initial-display of their experimental condition. We hypothesized that participants assigned to the content essay task, by building conceptual understanding, would deselect less task-irrelevant information from the overview initial-display, but perform as well as users in the surface essay task. Representational choice. Results indicated a significant main effect for Essay Task, F(1, 91) = 5.888, η² = .06, p = .017, however, this finding is qualified by the significant interaction with Initial Display, F(1, 91) = 5.027, η² = .05, p =.025. In the blank condition, both task groups showed low task-irrelevant data. In the overview condition, the content group chose representations with more task-irrelevant data as compared to the surface group, d(94) = .67, p < .01. Performance. Results indicated a significant interaction between Essay Task and Initial Display, F(1, 91) = 5.723, η² =.06, p = .019. The fastest performance was accomplished by the surface group in the blank condition. In the overview condition, the content group, even in the presence of more irrelevant information, attained the same performance as the surface group. The representational choice of the content group in the overview condition contained the more irrelevant data. However, their task performance did not suffer in terms of the speed for correct answers. Therefore, they seemed to be able to focus attention on the task-relevant data in the presence of irrelevant noise. In sum, our results can be explained by Canham and Hegarty’s (2010) finding about the effect of conceptual understanding on attention allocation.
EARLI
Impact of need for cognition and design of infographics on attitude change: a study on homophobia.
Sabine Preuss, C Nurra, Erica Vries.
In Conference of the European Association for Research on Learning and Instruction (EARLI). . 2015
The present investigation aimed to determine whether implicit and explicit attitudes toward homosexuals can be influenced by the information contained in an infographic, moderated by peoples’ Need for Cognition (NFC). We were also interested in the degree to which learning from the infographic, and participants’ perceived personal involvement with homosexuality, were related to attitudes toward lesbians and gay men. Three infographics were designed that displayed different amounts of information about homosexuality (high vs. low vs. none). Results revealed that an infographic with a high amount of information influences implicit attitudes only toward lesbians in a positive way, moderated by people’s NFC. For the explicit attitude measures, we found a main effect for NFC. Perceived personal involvement was the only predictor of attitudes toward homosexuals under certain conditions. Especially the attitudes of people high in NFC toward homosexuals can be influenced by an infographic; the high information infographic is most effective. Since involvement might play a role in attitude change, the relationship between involvement, attitude, and NFC requires further investigation.
THESES
2022
DISSERTATION
Tales of Graphical Discovery: A Case Study at the Intersection of Graph Comprehension and Visualization Design
“A picture is worth 1000 words,” the adage goes, but only—I argue—if you know how to read it. The same is true of graphs, charts, and diagrams. As powerful as these visuospatial tools may be in their communicative efficiency, they needn’t be immediately easy to understand. In fact, there are often trade-offs between a graph’s discoverability and efficiency. Even for informationally equivalent forms, the computational efficiency of a less conventional representation may outweigh concerns over ease of use for the untrained reader. It is this fact that underlies much innovation in Information Visualization, and the development of sophisticated interfaces for highly skilled workers performing specialized tasks. Sometimes this work results in novel, unconventional representations computationally suited to particular complex tasks, but that would present a challenge to the lay reader. Meanwhile, most work in remediating errors in graph comprehension has focused on “second order” readings: characterizing the trends or relationships between data represented in a graph. The ability to make these readings allows us to use graphs as vehicles for learning concepts—especially in science. We tend to accept a priori that well-designed graphs readily afford first-order readings: operations for extracting data from a graph. Accordingly, we know more about learning with representations, than we do about the learning of representations.In this dissertation, I use simple graphs with an unconventional coordinate system to explore how readers extract information from a graph when they lack knowledge of its graphical formalism. I address what the systematic errors readers make can tell us about our graphical intuitions, and the interaction of perceptual and conceptual processing that underlies graph comprehension.
2015
THESIS
Visualizing Time: The Influence of Timeline Axis and Direction on Causal Reasoning in Litigation Law
Can the visual-spatial representation of a sequence of events influence comprehension, causal reasoning and decision-making in litigation law? The present investigation addresses this question by examining the interaction between an individual’s preferred spatial construal of time (SCT) for a representational task and the SCT of a stimulus. One hundred fifty three undergraduates played the role of jurors in a fictitious civil litigation. The details of a case were recounted in a multimedia presentation of witness testimony, featuring an animated timeline in one of four orientations (Left-to-Right, Right-to-Left, Top-to-Bottom, and Bottom-to-Top). Participants were assessed on measures of comprehension, causal reasoning and decision-making. Results indicated effects of timeline orientation and SCT choice behavior on comprehension and reasoning. Results are discussed in terms of spatial and temporal cognition, and applied to the design of multimedia materials for the courtroom.