The appeal of Dashboard-Zero is its simplicity.
User-testing is employed across almost every industry, most often with surveys given to users after they experience a product. However, memory is fallible鈥揻eedback captured in the moment, as users experience a product, yields much more reliable information. But how can you do that without creating a distraction?
Elsy Meis, who graduated with a BS in Creative Technology and Design this spring, proposes an approach in a paper she will present later this month at the Human Computer Interaction International Conference, held online June 26鈥擩uly 1. Titled 鈥淗CI Strategies for Informing the Design of a Teacher Dashboard: How Might Real-Time Situational Data Determine the Potential for Technological Support in the Classroom?,鈥� she discusses user testing of a teacher dashboard developed by a 抖阴旅行射 Boulder team from the Institute of Cognitive Sciences.
Teacher dashboards provide teachers with information about real-time student learning. In this case, the system gauges student learning by using natural language processing to monitor conversations among small groups of students working on specific problems. Based on the content of the conversations, the system provides the teacher with a real-time assessment of student learning group by group.听
However, given the existing stresses of the classroom environment, the user experience for such technology must be finely tuned for teachers to adopt it. In her paper, Meis critiques the current retrospective user-survey approach to testing, pointing out the weakness of relying on memory. Instead, she proposes an approach to gathering data during the lesson that involves a single red button on an iPad:
鈥淲hen they press the button, they are answering yes to the question: 鈥楢re you feeling overwhelmed?鈥欌�� says Meis. 鈥淭he testing sessions are recorded from multiple angles, so have all the context and it鈥檚 not hard to figure out why they were feeling stressed.鈥澨�
鈥淯ntil now, the research group has interviewed teachers after their class ended to determine how they felt at different points during the class, or they would interview teachers based on hypothetical situations. Using this approach, we get more information without having to ask them,鈥� says Meis.听
The HCI International selection committee for late-breaking work clearly agreed, admitting her paper to the conference. 鈥淭he paper deals with a very specific application of this idea,鈥� says Meis, 鈥渂ut the hope is that it is seen as applying to a wide range of user-testing scenarios.鈥�
This research was supported by the NSF National AI Institute for Student-AI Teaming (iSAT) (DRL ).听听
听
听
Publication
Elsy Meis, Samuel Pugh, Rachel Dickler, Mike Tissenbaum and Leanne Hirshfield, 2022, 鈥淗CI Strategies for Informing the Design of a Teacher Dashboard: How Might Real-Time Situational Data Determine the Potential for Technological Support in the Classroom?鈥� In Proceedings of the 2022 HCI听International, pdf. (Virtual鈥擩une 26-July 1, 2022).
听
As an undergraduate student, Elsy Meis does not fit the typical profile of a first author on an academic paper admitted to an international conference. It鈥檚 rare for undergraduates to be listed as authors; even moreso as first authors.
However, Meis jumped into the world of research with both feet. Wanting to get involved in research since the beginning of her junior year, when she saw听a position with 抖阴旅行射 Boulder鈥檚 Institute of Cognitive Science, she immediately applied. Initially, she was assigned graphic design work, with an emphasis on user experience.听
Then, since her design work impacted the teams鈥� overarching research goals, Meis asked her supervisor, Associate Research Professor Leanne Hirshfield, to join their team meetings. 鈥淎t first it was so intimidating, but the research team has been super-encouraging and welcoming,鈥� says Meis.
Hirshfield explains,听"Although I initially hired Elsy with a specific set of tasks in mind, to say that she hit the ground running is an understatement; she has fully immersed herself in our work听and is now听a lead researcher, involved with many efforts throughout the institute."
It was during a meeting when they were discussing the shortcomings of surveys that Meis suggested the red button idea. 鈥淚 had this picture of the Staples 鈥楨asy鈥� button in my mind. After talking it through, Leanne and the team encouraged me to take the idea further, so I did,鈥� says Meis.