Designing digital video for learning and assessmentDaniel L. Schwartz & Kevin HartmanSchool of EducationStanford UniversityTo Appear in Video Research in the Learning Sciences; R. Goldman, S. Derry, R. Pea, &B. Barron (Eds.). Mahwah, NJ: Erlbaum.
Summary of article:
When used this way, the video is designed as an assessment that helps researchers learn what the children know.
...Yet, despite the ease of camera use, the array of editing features, and the many video genres, we find it frustrating that the literature provides few resources that can help these students make even more effective use of video for learning. Excepting work on mass media (e.g., Fisherkeller, 2002), there are relatively few empirical evaluations on the use of video for learning, even when compared to computer- aided efforts, as suggested in Table 1.
...To design an effective video, it is important to have a clear target, so in our discussion, we describe some important findings about learning and how to promote and measure it.
In the first section, we describe common learning outcomes, give examples of video genres that achieve those outcomes, and suggest methods for determining whether an outcome has been achieved. ... In particular, we suggest a number of ways to help assess learning (with and without using video), because our experience has been that creating learning assessments is very difficult, until one has seen many, many different examples.
In the second section, we offer some examples of how one might use digital video in a larger, multimedia context. ... This footage rarely makes a self-contained video story, but when embedded within a multimedia environment, it can be used in many creative ways to encourage learning interactions.
...We will describe these behaviors in some detail, because they can help clarify the meaning of the learning outcomes, plus they provide the keys to successful assessments.
...At the other end is a discernment approach that helps people perceive details they might otherwise overlook – the balance point of a painting or the difference between a 5.6 and a 5.8 in a gymnastic routine.
...Moving towards the discernment side of the continuum, there are point of view videos that use camera angles, audio commentary, or interviews to give insight into new ways of seeing, for example, from the point of view of a character, a coach, or a hunted prey.
...People, however, are very good at recognizing images (Shepard, 1967), so this can be made a more sensitive measure by showing things at a different angle or setting. ... One form of assessment might take a forced-choice approach where people have to select which of two pictures or videos is exemplary. ... A more open-ended approach provides the learner with a new video and has the learner describe what is important to notice about it.
...Without relevant prior knowledge, people can have difficulty making sense of a lesson and often have no recourse but to memorize the content rather than understand it.
...So, rather than directly measuring what people learn from the video, one can measure what people learn from the lessons following the video, on the assumption the learners can engage the new materials more fully.
...Other times, when it is too much to expect a learner to imitate an expert’s fully integrated performance, it makes sense to decompose a task into sub-skills that are learned separately.
...For routine skills that require efficiency (e.g., driving a car), it makes sense to require full-blown performance (e.g., a driving test) and evaluate the number of errors and time to execute the skills. ... For example, with intellectual skills, like doing a science investigation, it is useful to create a set of supportive materials a novice can use, rather than just asking the learner to do an investigation cold. ... Talking the talk is not as good a measure as walking the walk, but sometimes, it is an acceptable proxy that can evaluate whether a video is moving people in the desired direction.
...Facts are often seen as the stuff of memorization, but the difference between a good and bad news report is often whether it includes critical facts to help viewers draw their own inferences. ... Bransford, Franks, Vye, and Sherwood (1989) demonstrated that people remember facts better when those facts come as a solution to a problem an individual has attempted rather than as a bald assertion. ... The further one moves from facts, the more important it becomes to create videos that make processes and explanations transparent; talking heads only work if viewers already have sufficient prior knowledge to understand what gets said (Schwartz & Bransford, 1998).
...Typically, single analogies (and examples) are not nearly as effective as pairs of analogies in helping people cull the deep explanatory structure (people tend to focus on surface features of a single analogy or example, Gick & Holyoak, 1983).
...It can also take the form of cued recall, as in the case of showing a numeral and asking a child to state its name or say what numeral comes next. ... Open-ended formats (e.g., essays, making a video) are more difficult to score than right/wrong problems, but they also provide more latitude for students to exhibit what they have learned.
...More than the other outcomes, it is useful to think of explanation assessments before designing a video, because the assessments can help shape what is included in the video.
...The course used a multimedia authoring shell, called STAR.Legacy, that was designed to help teachers use their local resources to create pedagogically sound instruction (Schwartz, Brophy, Lin, & Bransford,1999).
...The pre-service teachers watched the news segment and then had to generate ideas about what they saw that was relevant to improving project-based learning. ... Another expert observed that the children did not measure how high their rockets went, and therefore, they had no way to do science on their designs. ... They found this engaging, and for us, it was a performance assessment of their ability to "do" multimedia instruction based on what they had learned about learning.
...These students also included an assessment page; users saw new clips and had to determine which concept was exhibited in each clip. Another group of students used contrasting cases, like wines side-by-side, that can help novices notice key features (see Schwartz & Bransford, 1998). ... As shown in Figure 3, users had to click on the juxtaposed video portrayals, notice the differences, and then match each teacher portrayal with the appropriate parents who were described in text fields to the side.
...Yet another team of students created a talking-head video in which they explained a visual framework they had invented to guide the design of project-based learning. ... Based on all of the foregoing video implementations, we were impressed by what the students had learned to do with video for learning and assessment and in a relatively short period of time.
...To help people make some headway in light of the limited literature on video for learning, we tried to help people see that there are distinct learning outcomes, and we compared different techniques of assessment to help readers discern what makes these outcomes different. We also gave some examples of how to do videos that help learners learn and help people learn if they have learned.
