����������

While the idea of using technology as a teaching tool in classrooms holds allure, why use technology if it does not have a positive effect on learning? During his Teaching with Technology presentation, Dr. Tim Weston spoke on the importance of assessment in determining the efficacy of teaching with technology.

An expert in assessment, Dr. Weston focused his presentation on helping educators assess learning technologies for efficient implementation.  According to Weston, “Oftentimes instructors will use a technology just because it’s there. Or it’s a new thing, everyone else is doing it… but it’s not necessarily being integrated to its fullest. So the first thing is just to think about what is this going to ‘afford’ you? How is this going to help you?”  This is where assessment comes in. However, Weston noted that the term “assessment” is imbued with a variety of misconceptions. For example, Weston explained that assessment is often misconstrued as being synonymous with testing.  Weston expounded that “… [standardized tests themselves] are not the assessment…” but that the assessment process involves all methods of gathering information and data. But how does one gather information beyond test taking?  To address this question, Weston laid out a model explaining where and how to gather information.

Benefit/Affordances:

When exploring the benefits and affordances of a technology, Weston employs two prevalent questions: “What does the technology allow that was difficult or impossible to do before?” and “Does the technology supplement or substitute for an existing teaching or learning practice?” According to the case study about Clicker use prepared by Weston, Clickers allow for a multitude of benefits for the instructor in his/her classroom environment. To answer the first question proposed by Weston, these benefits include:

•  Allowing the instructor to assess if students understand the content of a lecture in real time
• Giving an instructor the leeway to modify instruction in real time to explain concepts that seem difficult to understand
• An instructor can gather information from students who are typically afraid to answer questions publicly by providing an anonymous platform

In regards to the second question, Clickers supplement an already existing pattern of interaction between teacher and student in which the subject is discussed. However, students are able to answer anonymously through the use of Clickers and give the instructor a more accurate assessment of student comprehension.

Process/Theory of Change:

The next step in assessing a learning technology involves asking oneself the question, “How will using the technology help students learn?” In the case of Clicker use, the answers are straightforward:

• Students learn more efficiently because the instructor is able to customize the lecture in real time
• Students pay closer attention
• Students have a higher propensity to actively engage when answering questions
• Students receive immediate feedback about whether or not their initial assumption was correct– this allows the students to self-assess where they need to put in more work to learn the lecture content
• Because students are more interested and engaged, there is a higher probability they will study the content after class

But what does this learning look like?

• Students answer questions
• Students are not on laptops/talking with friends and are more engaged
• Students answer questions correctly more often.
• In end of the course surveys, students report a higher level of engagement

Adoption and Implementation:

Often, the act of simply deciding one wants to utilize a technology in the classroom isn’t enough. There is still the question of barriers one must overcome when adopting a new technology. In the Clicker case study, two barriers were observed:

• There is little infrastructure in place for Clicker implementation
• There is a higher economic burden because students are required to buy clickers

But what happens after one overcomes the initial barriers of adopting this technology? There is still the question of implementation and sustaining its use. For Clickers, these issues arise in the following ways:

• Clickers Break
• Clickers aren’t reliable
• There is more time spent preparing lessons that incorporate Clicker use
• The model for Clicker use doesn’t “fit” all lessons
• Not every student will want to participate.

Outcomes and Side Effects

Once the barriers for adoption and implementation are overcome, the next step for an instructor is to explore a series of questions in regards to desired outcome and possible side effects.

What are the desired outcomes of implementation?

1. A better understanding of content
2. Less decay of understanding in class

What evidence will show that the technology had the promised outcomes?

1. A final test shows that there was a better understanding of scientific content material incorporating Clickers
2. Higher achievement on the final in comparison to previous semesters in which Clickers were not utilized
3. According to an end of course survey, students reported higher levels of engagement

How will you collect this evidence?

1. Final test
2. Project to assess acquired knowledge
3. Survey

Could there be any unforeseen consequences for using this technology?

1. There were instances in which students gave their clickers to other students and skipped class
2. Instructors have to put in more work to prepare lessons to incorporate Clicker use
3. Clickers inhibited varied lecture formats
4. It is difficult to write good Clicker multiple choice questions
5. Real discussions were challenging to stimulate when students already know the answer to the Clicker question.

Cost/Benefit Decisions:

After an instructor has gone through each of the previous steps, there are two final questions an instructor must ask his/herself in assessing the efficiency of a technology as a learning aide. Does the benefit of using a technology outweigh the “cost” of adoption, implementation, and adverse side effects and what can one change about implementation to improve a course? For example, in the Clicker case study, the instructor did not see improvement in test scores.  However, the students did report being more engaged in the course. As an added benefit, there were fewer students leaving the course during the first three weeks after the implementation of clicker questions as opposed to prior; as a result the instructor decided to continue using clickers. The instructor then changed the implementation to improve his/her course by adding additional learning activities without the use of Clickers and also made the Clicker questions more difficult.