Teaching with Classroom Communication System - What it Involves and Why it Works **

A. Louis Abrahamson, Better Education Inc., 4824 G. Wash. Mem. Hwy., Yorktown, VA 23692, USA (la@bedu.com)

ABSTRACT

In July 1992, NSF awarded a grant to a small startup company called Better Education Inc to develop a new tool for teaching known as a Classroom Communication System (CCS). Collaborators on this work included some of the outstanding people in education research in the USA. The results of this grant have been far-reaching. This technology has been shown to act both as a facilitator & as a catalyst for improved teaching and learning. The pedagogy that it spawns seems to have a remarkable quality that not only helps students to learn better but also makes their classrooms more active, lively, & happy places. These results emanate from a wide range of disciplines including chemistry, biology, mathematics, political science, law, psychology, reading comprehension, business negotiation, & history. But, it all started with PHYSICS, because all of the original researchers were physicists. These included Eric Mazur, Bill Gerace, Jose Mestre, Alan Van Heuvelen, Jim Minstrell, David Hestenes, Gregg Swackhamer, Bob Dufresne, Fred Reif, Jill Larkin, George Webb, and Bill Leonard. This session will demonstrate and discuss the pedagogies and describe how and why they work. It will be presented by Dr. Louis Abrahamson - one of the CCS inventors.

Introduction

From the title of this paper, you might think that it is mostly about technology. Specifically about high-powered networked technology for classrooms that can do magical things to help students learn better, and take some of the load off you, the hard-pressed teachers. If that is still your impression at the end of this workshop, then I'm afraid that I will have to take a C- for my teaching because this is an incomplete and in some ways misleading perspective.

So, let me start right away with a direct statement about CCS technology, what it IS about and what it IS NOT. The main idea behind the CCS is about bringing teachers closer to their students. It is not about taking the teacher out of the loop and having computers teach students. And, it is not about interposing a computer system between teacher and students, replacing their regular human contact. Further, it is not some radical idea that has come out of a new-fangled psychological theory. On the contrary, it is an old idea, very very old in fact.

Lessons from Socrates

Two thousand four hundred years ago, the Greek philosopher Socrates realized that people understand more by answering a question, than by being told an answer. This is a very counter-intuitive idea, and it is often REALLY hard to accept an internalize. I remember with my own children when they were growing up. After all, I had struggled hard to understand some things, it was impossible to stop myself from blurting out my answers to all life's questions, before my children had even begun to understand the questions.

The reason that the Socratic method works in teaching is because a teacher through questioning can spotlight an area of knowledge, encourage students to think through the issues, establish positions, and commit to positions. It just so happens that this is exactly what the past twenty five years of cognitive science research has shown are the things that you would like someone to do when you want them to learn something.

But, there is a problem with Socratic teaching: it works well in a small group with perhaps five students. In a class of thirty, one hundred, or three hundred, most students get left out of the interaction. So, if you interact well with five students in a class of thirty, the remainder don't get the full benefit of your teaching. At the dawn of the 21st century this is unfortunate, because now we want to educate EVERYONE, not just the elites of our societies as we did a few hundred years ago. So, it would be appropriate to use the best methods, for example, the tutorial system at Oxford and Cambridge in England, where two to five students meet regularly with their professor ("tutor") in his/her study. Obviously a wonderful environment for teaching and learning - but expensive - much much more expensive than a big class or lecture. This was the reason behind the CCS. We wanted to build a system that would enable teachers to teach interactively, even in the normal class sizes that are associated with common educational practice in the 20th century. Of course, we realized that it would never be as good as a class of five students in the leather armchairs of Cambridge University, but we believed that we could do a lot better that the educational pedagogy that is common practice world-wide.

To change things you have to start with where you are and understand how you got there, but this is a short presentation - not a history lesson, so we're going to skip about 2,300 years and jump straight to behaviorism

Behaviorist Theories about how We Learn Things

The human brain is a funny (strange) subject. One can say that it is an engineering marvel unequaled by any other system on planet earth, but this "compliment" is almost an insult to us humans. Our brain is the source of our innermost selves - the engine and soul of our species. It is almost instinctive to react when it is compared to an engineering feat, because this implies that it could be de-engineered or taken apart and understood. And, it almost certainly will, probably in the 21st century.

We already know a lot about how our minds work. In the 20th century the science of psychology has come a long way. For the first two-thirds of the time, ideas about learning were dominated by the concept of "behaviorism". Behaviorist theories of learning sought scientific, demonstrable explanations for simple behaviors. For these reasons, and since humans were considered to resemble machines, behaviorist explanations tended to be somewhat mechanical in nature. They made use of one or both of two principal classes of explanations for learning: those based on contiguity (simultaneity of stimulus and response events) and those based on the effects of behavior (reinforcement and punishment).

Some principles from behaviorist theory follow:

  • The following are valuable,
    • Repetition
    • Small, concrete, progressively sequenced tasks
    • Positive and negative reinforcement
    • Consistency in the use of reinforcers during the teaching-learning process
  • Habits and other undesirable responses can be broken by removing the positive reinforcers connected with them.
  • Immediate, consistent, and positive reinforcement increases the speed of learning.
  • Once an item is learned, intermittent reinforcement will promote retention.

For many years, these concepts from behavioral theory formed the basis of most of the learning theory applied in child rearing and in classrooms. Parents and teachers still find that, in many instances, individuals do learn when provided with the appropriate blend of stimuli, rewards, negative reinforcement, and punishments. Especially with small children and simpler tasks, behavioral principles are often effective.

Eventually, however, educators began to feel that although stimulus-response does explain many human behaviors and has a legitimate place in instruction, behaviorism alone was not sufficient to explain all the phenomena observed in learning situations. The cognitive approach began to gain attention, while the behaviorist theorists went on to explore the possibilities of programmed learning for the computer age. Today, most computer-assisted instruction is solidly planted on the foundation laid by behaviorist researchers, as evidenced by the emphasis on "drill and practice" techniques.

Good & Bad Results of Behaviorism in Physics Education

There were many good results in education from behaviorist theories and research. Obviously, it is a positive thing for,

  • students to get practice at applying theories and relationships by working out problems, for
  • course material and instruction to be presented clearly one step at a time, and for
  • positive or negative reinforcement to be based on student's work and given in a consistent and timely fashion.

But, there were also some bad results. Mostly these came from an extreme adoption of a behaviorist perspective. For example, emphasis might be placed on solving problems with a huge number of repetitions for similar type problems. This would be done in the belief that these repetitions "train" the student and create beneficial "habits". And this may indeed have been the actual effect for certain students. However, it turned out that more frequently, the habits that students learned were the opposite of beneficial. What students would typically do was to compile a list of "formulas" from their notes or textbooks, and "plug & chug". Each problem was reduced to the simple artifact of picking the right formulas and substituting in the numbers. This was positively harmful to developing a deep understanding and true appreciation of Physics.

To show how this realization came about we need to go back to two seminal papers by Halloun & Hestenes1 in 1985. The astounding revelation from these publications was the unambiguous result that most students could complete and pass an entire course on Newtonian Mechanics at a major US university and still have little understanding of the basic concepts on which the subject depends. This work has since been verified in large scale evaluations of students conceptual knowledge from conventionally taught physics courses2 at a cross-section of educational institutions from high schools to premier universities. As the data became too powerful to ignore, physicists interested in improving education began to take another look at the theories on which their models of teaching were based. And there had been some interesting developments from which to draw lessons.

Cognitive Science and Where It Led

In the 1950s Swiss psychologist Jean Piaget had performed pioneering work on the development of concepts in young children. The structure of these concepts and the way that they were used by children to explain their world, was sufficiently complex that they were difficult to explain by strict adherence to behaviorist dogma. This dogma in its most extreme form was evidenced by viewpoints like B.F. Skinner the famous behaviorist who had said, "The question is not whether machines think, but whether men do." Stimulated by Piaget's work the development of cognitive issues in psychology continued and mingled with influences of information theory, artificial intelligence, linguistics, philosophy, and neuroscience to ultimately create a new field in the mid-1970s.

This field, the science of thinking, came to be called Cognitive Science, and it quickly aided the development of new ideas about the nature of learning. To make a long story short and compress it into one sentence, it said that,

  • learning is, an active process that involves organizing new information, and linking it to prior knowledge, and that the process of striving to accomplish this organization is facilitated by an overall goal or perspective, and that learning is almost never a linear process.

Possibly from the perspective of 1999, this may all sound like obvious stuff, but it had profound implications for the physicists who had been so jolted and alarmed by the results of Hestenes' work.

One such physicist was a young bright charismatic professor at Harvard University. His name was Eric Mazur, and his first reaction to the Hestenes results was, "No!" "Never!" "Not my students!" "Not Harvard students!" But, what he found, was that his students were as subject to the problem as any others. So, he decided to change the way he was teaching the course. Like a number of other leading physics teachers around the country, Mazur decided that the solution was to engage his students actively in the learning process. This decision led to a radical conclusion: passive students sitting in front of him during lectures could not be a part of this new way. He chose to embrace active engagement, interactive lecturing, and especially "Peer Instruction3".

Active Engagement and Constructivism

It was no coincidence that Eric Mazur decided to try actively engaging his students in order to help them to learn better. The year was 1992, and the recent discoveries of the new discipline of cognitive science were becoming known. For thinking people in the forefront of their fields, it was becoming increasingly hard to reconcile what they were hearing with what they say experienced every day in classrooms and lecture-halls. What they were hearing was that learning "is an active process", but this statement contrasted dramatically with the passive bored students sitting in front of them in classes every day. That is, science was saying what many of them had long known in their guts, that large passive lectures were not a very good way to teach.

What the new science was saying was that in order to learn something, a student had to organize new information and fit it to what s/he already knew. Then the student had to strive to "make sense" of it, testing and if necessary, re-orienting, and "constructing4" his or her own knowledge. It follows from this picture of knowledge creation that every person will construct their own unique set. No two people will start with exactly the same knowledge base, and no two people will construct exactly the same knowledge structures from given experiences or information. It also follows that knowledge cannot be poured in, from one person to another. In the 1980's this concept of individual knowledge construction became known as "constructivism".

So, the obvious question had to be asked, "If people don't learn by having knowledge poured in, what is the purpose of lectures?"

The Birth of CCS Technology - First Classtalk Prototype

When new ideas are in the air, they swirl around and land in the most unlikely places. In the late 1980s a NASA scientist, his electronics engineer friend, and their physiologist-turned-physicist son-of-a-Nobel-Laureate neighbor teamed up to build a simple prototype of a new tool for teaching. Their motivation was to make a system to enable Socratic teaching and active learning even in big classrooms.

They called it a Classroom Communication System (a CCS - detailed description follows in a later section) and installed it in a large lecturehall at Christopher Newport (their local university), where it was used by Prof. George Webb for teaching introductory physics. And it worked! At the end of every semester they took surveys of the students. Ninety percent claimed that they,

  • understood the subject better, and
  • enjoyed classes more.

A slightly lower percentage also claimed that they,

  • came to class better prepared, and
  • paid more attention in classes.

The Professor was enthusiastic about the change in his classes. The whole class was transformed into an active, lively, and happier place. Students at this commuter school who had not known each other before, would now come into class laughing and joking together. The drop-out rate decreased because students actually enjoyed the class and would stay in, even if it meant taking a "D".

The Essence of Interactive Teaching with a CCS

So, what techniques did George Webb use to bring about this transformation in his classroom? They were actually very simple, but at the same time, quite profound. And as we've said they were based on ideas two thousand four hundred years old.

All that George Webb did was intersperse questions with his lecturing. For example, he would lecture for ten minutes and then ask a question. The question would be displayed on the overhead projector, and students would be asked to think about the question, discuss it in their small groups (two to four students) and input an answer using a small handheld keypad (computer or calculator) which transmitted their answer to the teacher's computer. If the group was unanimous, then they just had to input a single answer, but if individuals disagreed with the consensus answer, they could input their own dissenting answer. The teacher's computer automatically calculated class statistics and plotted them in the form of a histogram, so they could be shown to the class on the overhead projector. Students liked seeing the histogram because it would give them an idea of how their own answer compared with the positions taken by the rest of the class, and it was not embarrassing even if they were in the minority because no-one else knew what their individual answer was.

This simple device combined with clever questions, was all it took to totally transform George Webb's large physics lecture of one hundred and fifty students. It did several things. It made students think about the issues that they were being taught, it encouraged them establish positions, and the fact that there was a record of their position placed an appropriate amount of pressure on them to commit and stand behind their position - just as in a small class with a good Socratic teacher! Additionally, because the classroom was a natural social situation, the process engaged the students, and the peer interaction helped them deepening their understanding as they articulated their reasons for taking a particular position, or learned from the explanations of their peers.

We need to note at this point, that not all of these things became clear at once. Many were found and understood from the later research which we will now describe.

NSF's Grant to bE - Design of a Newer More Powerful CCS & Pedagogical Research

In 1991 as a result of this early work the original inventors formed a company Better Education Inc. (or bE), established a research team and submitted a proposal to the National Science Foundation to further investigate their findings. It turned out to be an appropriate time. Not only did each member of the research team have their own existing research grants, but they were all ready to use the CCS tool in their pedagogical research. Also, with one exception, they were also all physicists. The team included Eric Mazur (Harvard), Bill Gerace, Jose Mestre, Bob Dufresne, and Bill Leonard (University of Massachusetts), Alan Van Heuvelen (Ohio State), Jim Minstrell (Mercer Island High School), David Hestenes (Arizona State), Gregg Swackhamer (Glenbrook North High School), , Fred Reif, and Jill Larkin (Carnegie Mellon University), and George and Jane Webb (Christopher Newport University). The one non-physicist, Prof. Mary Budd Rowe a Stanford Professor of Education deserves special mention. Her research in the '60s on teacher's questioning had produced interesting results. She had found that, while almost all teacher's used questioning in their teaching, that this was sadly superficial from the cognitive point of view. In fact, her timing data showed that almost all teachers paused only long enough for students to access their short term memories. There was not even time for students to retrieve any information from their long term memories, let alone process the information from a cognitive point of view. In other words the questioning used by most teachers was useful only for short term factual recall. Prof. Rowe died in 1996, but we treasure her advice & support on this project.

When the research grant was awarded in 1992, designs for a new CCS were already well advanced. The original system used simple numeric keypads which enabled limited forms of interaction between teacher and students. We saw the need for a deeper and much more comprehensive information flow, and determined to construct a full low cost network. Handheld computers or "palmtops" were making their first appearance, as were graphing calculators with communication ports. We saw the possibility of using a network of student owned handheld computers that could be carried with them from class to class, that could be used for communication as well as for computation and other pedagogical and learning applications. Along with a network, we saw the need for special software to control the system and allow it to function as an integrated tool. As with Classtalk I the teacher's computer would have to run two monitors, one with private information, the other with public information for display to the whole class. In Classtalk I we had accomplished this by linking two PCs together, but the Macintosh now had this capability built-in to its operating system. As we progressed, we built five question types into the software (multiple choice, numeric, short & long text, & algebraic expressions). In the teacher's computer, student answers were sorted into "bins". For example, in multiple choice there was one bin for each choice. The purpose of binning is to provide a quick way for a teacher to assimilate positions that students in the class have taken. Seat icons mapped on the screen as in the physical classroom contained individual data (name, responses, records, photo, etc.) about students seated there. Bins were color-coded and student seat icons on the teacher's private screen changed color to show into which bin a student's answer fell. Binning could also be more complex. In the case of algebraic expressions, these are parsed, variables identified, random number sets plugged into each variable, and the expressions evaluated. Expressions that evaluate to the same set are assumed to be the same. A sub-binning evaluated functional form. Bins could be predefined by the teacher or created on-the-fly from student responses. Bins were plotted using histograms show aggregate class response. These could be viewed on-the-fly by the teacher on the private monitor as answers were coming in. They could also be projected on the overhead display and shown to the class. The software included three different small group environments with individual responses augmented by two collaborative models: consensus, & consensus with dissent (which was modeled after the US Supreme Court). Overall, the teacher's computer software operated in three domains, future (curriculum preparation), present (active class), and past (class records). Curriculum could include questions (including binning info., individual student feedback, scoring), quizzes, presentation material, notes, etc..

In many ways we were a decade ahead of our time in our concept of the CCS. Even with all of the features described above, the system did not come close to implementing our full vision. The hardware was simply not powerful enough and the software tools had not yet been invented. (Maybe by 2,003, Java with wireless networking will be available on low-cost handhelds, that can be mass marketed to education.)

In the meantime though, it was possible to accomplish a great deal. In February 1993, in the Science Center at Harvard University, two hundred and fifty students logged into Classtalk II in Eric Mazur's first class using the system. This system employed our own low cost home grown network, and had 80 palmtop computers communicating with the teacher's computer as a loosely coupled integrated system. In all well over 100 microprocessors communicated flawlessly with each other that day in that classroom. A significant achievement in 1993! It is worth commenting on such technical aspects only because these are barriers to pedagogical innovation. Technology and pedagogy development are related in a "chicken or egg" type problem - you can't have one without the other, but which came first? That is, pedagogy cannot be developed before the tools, but the tools will not be available till there is a clear market and demand for them. We decided that a step by step approach was the only possible way. And as the results in the following section show, even small steps can be very worthwhile.

Research Results from Interactive Teaching

(1) PEDAGOGICAL DIVERSITY

One of the first things we discovered was that there are many important reasons for asking questions in class. The following list comes from actual records of questions asked from one professor (Randy Caton of CNU) teaching one course (Physics 201) over one semester in 1991:

  • to assess background knowledge,
  • to test if what has been taught has been understood,
  • to provoke a class discussion
  • to emphasize or reinforce a point,
  • for actively reinforcing procedures involved in working through a thought process,
  • to introduce a new topic,
  • to apply and generalize a reasoning strategy, or to see if students can combine past material to reach an understanding of present material,
  • to see if students have read the book,
  • to see if they understood what was done in the lab,
  • to test student's physical intuition before a demonstration, or before teaching a subject,
  • after a demonstration, to evaluate student's interpretation of what happened,
  • to correct a misconception or to lead students to a better understanding.

We think that this list is noteworthy because it illustrates the pedagogical diversity that is possible once a teacher makes the leap to embrace interactive teaching. It is also interesting because of the mix of behaviorist & cognitive perspectives that lie behind the different reasons themselves. For example, testing understanding, seeing if students have done their reading, or if they understood the lab experiment, all have behaviorist and cognitive aspects. From the behaviorist perspective a teacher cannot give positive or negative reinforcement unless you know the behavior. So seeing if students read the book is almost pure behaviorist, but if understanding something is cognitive, is checking understanding behaviorist? More on this later!

 

(2) STUDENT ACHIEVEMENT

It is appropriate to ask if the changes in pedagogy result in improved student achievement. The results are overwhelming and supremely positive. In his book "Peer Instruction3", Eric Mazur gives data (see Table 1) on student performance in pre-tests and post-tests using the Hestenes Force Concept Inventory5 (FCI). The FCI is a diagnostic test that was developed specifically to require a choice between Newtonian concepts and common-sense alternatives. It has become widely used as a standard. Mazur also used another more advanced test, the Mechanics Baseline Test6 (MB), which unlike the FCI requires a moderate amount of computation.

Table 1. Summary of Student Data Before and After Pedagogy Change (from Mazur3)

FCI

Method of teaching Year prea postb gain Gc MB Nd
Conventional 1990 (70%)e 78%  8%  0.25% 67% 121
Peer Instruction  1991 71% 85% 14% 0.49 72% 177
  1993f 70% 86% 16% 0.55 73% 158
  1994 70% 88% 18% 0.59 76% 216
  1995g  67% 88% 21% 0.64 76% 181

a data obtained on first day of class
b data obtained after two months of instruction
c fraction of maximum possible gain realized
d number of data points (students)
e no FCI pretest in 1990; 1991-1995 avg. shown
f no tests administered in 1992
g data for 1995 reflect use of manuscript for forthcoming text

The data in Table 1, show a doubling of student gains on the FCI test, after the change in pedagogy (from 8% to an avg. 17.3%), and a consistent significant increase in performance (avg. 7.3%) on the Mechanics Baseline test.

But Mazur's data are only the tip of the iceberg. In an exhaustive study involving more than 6,000 students from 62 introductory physics courses, Hake2 compared student performance on the FCI & MB from courses taught using conventional techniques versus courses taught using interactive engagement techniques.

Because of the wide range of courses, student abilities, and initial knowledge states, Hake developed an interesting methodology. He reasoned that some classes (e.g. students at an regular high school) could be expected to score low on the FCI pretest, while other classes of students (e.g. Harvard pre-med. students) could be expected to score high on the pre-test. So, the maximum possible gain from pre-test to post-test would be much higher for ordinary high school students than for Harvard pre-med students. However, Hake reasoned, perhaps the percentage of maximum gain actually achieved by both sets of students might be the same, if, the pedagogical techniques used were similar in both cases.

So, Hake divided all of his data into two separate but broadly defined categories. Into one category he placed all classes that had been taught in a "Traditional" format which he defined as "relying primarily on passive-student lectures, recipe labs, and algorithmic back-of-chapter-type problem exams". Into the second category he placed classes which were specifically intended by the instructor to possess an "Interactive Engagement" component. For the purpose of screening these classes, Hake used the following definition of interactive engagement, "heads-on (always) and hands-on (usually) activities which yield immediate feedback to the students through discussion with peers and/or instructors".

After careful analysis of his data with diligent attention to statistical considerations Hake found that his hypothesis had been correct. Courses taught in the conventional manner showed gains (from pre-test to post-test) of 22% (+/- 5%) of the maximum possible gain, while courses taught using interactive engagement techniques showed gains of 52% ((+/- 10%) of max. possible gain. This relationship held true across a spectrum of teachers, educational levels, and educational institutions (see data plotted in Figure 1).

Figure 1. %<Gain> vs %<Pretest> score on the conceptual Mechanics Diagnostic (MD) or Force Concept Inventory (FCI) tests for 62 courses enrolling a total N = 6542 students: 14 traditional (T) courses (N =2084) which made little or no use of interactive engagement (IE) methods, and 48 IE courses (N = 4458) which made considerable use of IE methods. Slope lines for the average of the 14 T courses <<g>>14T and 48 IE courses <<g>>48IE are shown.

There are two important things to note:

  1. Terminology - Mazur talks about "peer instruction" and "conceptests", Dufresne7 et al about teaching from a "constructivist" perspective, ourselves about "interactive teaching", and others simply about "active learning". Obviously, there are important differences between all these terms. Hake chose the term "interactive engagement" with the intent of presenting an all inclusive picture from the perspective that the commonalties of intent and outcome of all interactive methods are greater that their differences. It is hard to argue with his data, except to note that the scatterband from IE methods is wide, which would tend to indicate that there is still a lot to learn about the subtleties of individual methods.
  2. Technology - Mazur used a CCS in his teaching as did a few of the others using IE methods in Hake's dataset. Our perspective is that a CCS is simply a tool to facilitate and catalyse interactive engagement, and that it does this job remarkably well. But that the results ultimately stem from the pedagogy, not the tool.
  3. Other Disciplines - There is also corroborating data from other fields (see McIntosh Elem. below).

 

(2) THE INTERACTIVE CLASSROOM UNDER A MICROSCOPE

Understanding subtleties is one of the most difficult but important aspects of educational research. Data on student achievement, such as that shown in the previous section, paints a broad-brush picture, but it does little to help understanding of "why" and "how" type questions. One of the tools that we have used to help put a microscope on this type of question are surveys, where students are given a list of statements and asked to rate each on a five point scale from disagree strongly to agree strongly. While most questions are phrased in the positive sense, some are also phrased in the negative, to trap rote responses. Results are strictly anonymous, so that student's answers will be dictated by their true feelings.

Figure 2. "A Better Understanding of the Subject Matter": Summary of Student Survey Results from Physics Classes at Four Universities where Interactive Teaching with a CCS Was Used.

The first set of survey data from Spring '94 showed that the original findings in Prof. George Webb's class at Christopher Newport University had not been a mirage. The same results were also evident at Harvard, Ohio State, and the University of Massachusetts. At all these places, students believed by huge margins that answering questions in class helped their understanding of the subject matter (see Figure 2). By similar margins at all four places, they also enjoyed classes more (Figure 3), paid more attention in classes (Figure 4), and felt that the professor was more aware of their problems with the subject matter (Figure 5).

Figure 3. "Enjoying Classes More": Summary of Student Survey Results from Physics Classes at Four Universities where Interactive Teaching via the Use of Questioning with a CCS Was Used.

Figure 4. "Paying More Attention in Classes": Summary of Student Survey Results from Physics Classes at Four Universities where Interactive Teaching via the Use of Questioning with a CCS Was Used.

Figure 5. "An Instructor More Aware of Student Problems": Summary of Student Survey Results from Physics Classes at Four Universities where Interactive Teaching via the Use of Questioning with a CCS Was Used.

This is a interesting and extremely significant result because the differences in the four settings hugely exceeded their similarities:

  • All four professors used different questions. They compiled these themselves and shared no data on them. They also had no guidelines from us or anyone else on what to put in the questions. They simply chose what they regarded as the most appropriate questions for the course and students they were teaching.
  • They had different reasons for asking the questions. Mazur used reading quizzes (to check reading) and conceptests (to check understanding of what he had just lectured on). Webb used "lure & trap" type questions at the beginning of a new topic (with a simple, seductive, common-sense, but wrong answer) to motivate students (when they found out the whole class was wrong about something simple, they would become interested); later after learning more about physical principles the whole class would get this type of question right (and cheer!). Mestre used questions to stretch students thinking from a constructivist perspective: (a) to examine their own ideas, (b) to determine the extent to which new experiences make sense in the light of these ideas, (c) to consider a number of possible alternative explanations for what they have experienced, and (d) to evaluate the usefulness of a number of different perspectives. Van Heuvelen used questions as drivers to have students work through innovative "Active Learning Problems" (ALPS) of his own authorship.
  • They were very different types of institutions. Among the four universities, there were two large State Universities but one was city based and the largest university in the country, while the other was in an isolated campus in a small rural west Massachusetts town. The third was an institution with low admission requirements catering mainly to the children of blue-collar families who are likely to be the first generation in their families who are attending university. And fourth, there is only one Harvard, and it is uniquely different to any other university in the country.
  • The courses were different. CNU's course was a physics survey course for non-science majors (physics for poets), Harvard's was introductory calculus-based physics, Ohio State's was physics for engineers, and U.Mass's was on a specific branch of physics, the Theory of Sound.
  • The students were different. CNU's were mainly business or arts majors fulfilling their one General College science requirement; Harvard's were mostly pre-med students who tended to regard physics as an unpleasant medicine to be taken on the way to becoming doctors; Ohio State's were engineers who regarded the material as necessary, and serious stuff to be used in later life; the U.Mass class were audiologists in training and almost 100% female.

No experimentalists in their right minds would ever have chosen this diverse, eclectic, crazy, mix of people and situations if they ever hoped to get serious data that could be correlated with theory in a sensible fashion. The essence of good experimental technique is to isolate variables, change one thing at a time, and measure the effects of this change. In well funded studies, control groups are carefully constructed to accomplish this end. But well funded studies are by definition, expensive. We did not have the luxury of a well funded study, we had to piggy-back on existing research programs that were funded for other ends, and everyone participating in this project was effectively working for free.

So, we took what we could get, and we got lucky! For all these variations, the level of unanimity in the data is amazing. And this is why we were so lucky, because if you don't follow the rules of carefully controlling variables, and still your experiments show clear unambiguous revealing data that is totally consistent with theoretical predictions, then the conclusions are clear. That is, this data must represent some fundamental scientific truths about teaching & learning (and even perhaps about how people think). Because, as Hestenes had said in a far-sighted 1979 paper, "A science of teaching, pre-supposes a science of thinking8."

But we are getting a little bit ahead of ourselves here, because these first survey results (which were presented at the Second CCS Conference in Williamsburg Virgina in 1994), were not sufficient to show full correlation with theory. More in-depth evaluations were needed. We wanted to understand from a constructivist cognitive science point of view, what was really happening in students heads in these classes that was different to what usually happened in other classes.

First, we wanted to get some measure of thinking. Thinking is obviously critically important to learning, and not just the quantity but the quality. But, how do you measure it, what parameter can define quantities of thinking, and as to quality, what does that even mean? The strategy we adopted to answer these questions was to amend the student survey forms, adding new questions and modifying others. The results of the first key questions are contained in Figure 6(a). We simply asked students to rate the statement, "I do more thinking in classes where the teacher lectures, than I do when we use Classtalk." We reasoned that thinking in this context was directed mental effort, and would be perceived as such by the students, in rating this statement. From a constructivist point of view, taking new information and fitting it to existing knowledge, is work, that is mental effort. So we believe that the ratings of this statement give a reasonable comparative measure of directed mental effort in the context of interactive lectures versus passive ones. From the data in Fig. 6(a) it can be seen that less than 4% of the students in both Mazur's and Mestre's classes disagreed with the statement, while more than 80% agreed or agreed strongly.

Second, we wanted to assess the "direction" of the mental effort. How well was it focused on the issues at hand, as motivated by the teacher's questions? To make this determination, we asked students to rate the statement, "Some Classtalk questions made me try really hard to make sense of the subject matter." We reasoned that the phrase "trying really hard to make sense of the subject matter", represents exactly the directed mental effort that is striven for in all of education. Here 90% of the students in both classes, agreed or agreed strongly, with almost no disagreement (Figure 6(b)).

This results were powerful stuff! The data strongly supports the position that within these two interactive classes, both the level of mental effort as well as it's direction and focus in the vast majority of students, was exactly what might be wished by all teachers. And it is not too hard to explain. If students are hearing, seeing, talking about new ideas, having to answer questions about them, and commit themselves to answers (put themselves on the line) - no wonder they are trying REALLY HARD. So, if the questions are good questions, one would naturally expect RESULTS from all this beneficial effort.

 

Figure 6. "Students Think More in Lectures, Try Really Hard to Make Sense of the Subject Matter, Have the Experience of Suddenly Understanding Concepts, and Find Interactions Within their Small Groups Beneficial": Summary of Student Survey Results from Physics Classes at Harvard and The University of Massachusetts, where Interactive Teaching via the Use of Questioning with a CCS Was Used.

And the students did get results. Over 80% of them had the experience of SUDDENLY UNDERSTANDING CONCEPTS IN CLASS, during the specific times that they were actually engaging in these activities (Figure 6 (c)).

One final point that we wanted to study specifically: we had always suspected that peer-group interactions were very important in the overall process of interactive teaching. Beginning with George Webb, all of the researchers had used class discussion or small group interaction to some degree, but notably Eric Mazur gave them the prominence that we believe they deserve. He called his entire method "Peer Instruction", and this was before the data shown in Figure 6(d), was acquired. This data shows that 80 to 90% of the students found small group interactions helped them with their understanding of the subject. Mazur's explanation for this phenomenon is that students who understand a topic have only recently learned it. They still remember HOW they learned it. So, they can explain it. In contrast, a professor may have assimilated these knowledge structures many years before. He has some difficulty understanding how anyone can not understand the topic, and he certainly has no memory of how he ever learned it in the first place.

We think that there are also other factors involved:

  • people are social animals, they like interacting with each other, and the classroom is a social setting in which small group communication is natural and unthreatening; also,
  • there is a saying, that "to learn something fully, one needs to teach it", so students learn when they are explaining something to someone else, as well as when they are being explained to.

Why One Sometimes Learns a Lot in a Lecture

There are some other issues that this work has raised that we feel need to be dealt with directly. One of these is the obvious question of why one sometimes learns a lot in a lecture. We feel that cognitive science, constructivism, as well as our current work, provide some obvious answers to this question.

Consider for a moment the great orators of history, how they moved nations, brought armies to the battlefield, and shaped situations to their will. Closer to our own experience is the at least one great teacher that has influenced and shaped our individual lives. And, we probably can remember how we were held spellbound by such a teacher as they reached into our inner selves and made connections that changed our perspectives. Or, consider plays, theater, and movies, where we sit absorbed for hours, our attention riveted on the people and the story. These are ubiquitous events and they are not that different from each other from a cognitive science perspective.

The common elements between these events and great lectures are what they do to the audience, and more specifically, to individuals within the audience. Let's look at the process in detail. The movie with a story that absorbs us and touches our emotions, does not do that directly. In fact, it does not even do it indirectly. It is actually us that do it to ourselves. The movie is just a piece of celluloid, it is our brains that interpret the images and sounds and "make sense" of them. It is the evolving "sense" of the story within our minds that affects our sensibilities, assaults or conforms with our principles, and leads us to play "what if" games as though we were in the story ourselves.

That is, the story itself is new information, and we are busily engaged for one and a half hours, in fitting it into information, knowledge structures, concepts, rules, ideas, and nuances that we already possess within our brains. In this sense, it is clearly untrue to claim that we are not interacting with the story. We demonstrably are interacting with it, and you have only to look at the teary faces emerging from a popcorn reeking movie hall, after a powerful but sad drama to know that it is not the celluloid that did it. Similarly, it is not the great orator or teacher who reaches down inside our innermost selves and turns certain switches. It is we who do it ourselves in response to the information, questions, or conclusions presented (however subtly) by that person.

Good lectures work in just the same way as movies, theater, or other presentations, and obviously people learn a great deal in them. It would be ridiculous to pretend otherwise, just as it would be the height of folly to try to make every public presentation interactive. Think of interactive preaching, interactive presentations of research results, or an interactive State of the Union address by the President. But, these examples may be as inappropriate as passive lectures on introductory physics! Doubly so perhaps, because the attention level of the world is changing. At the end of the 20th century, people are used to having their attention sought and beguiled by multi-million dollar budgets per minute. And these become the de-facto standards for public presentations which include teaching. It is hard to give a great physics lecture on Newtonian mechanics to live up to this type of standard. Fortunately, interactive teaching presents a better and more productive alternative!

Work Inside & Outside Classrooms

Another conclusion from our research is that interactive lectures strengthen the need to have closer links between work inside and outside classrooms. For students to benefit most from interactivity inside classrooms, they need to come prepared. Mazur insists that his students read ahead in the book, and he enforces it by grading them on work that has to be turned-in BEFORE lectures. Webb has always started his classes with daily quizzes, on advance material, for the same reason.

Also, it makes sense that if the old raison d'être for lectures was to "present" a totality of the material that students would need to be exposed to for a course, that in shifting to interactive teaching, something has to give. There is just not enough time in classes to do everything just as before, and also add interactivity. But as Mazur articulates so well, we have books, and we don't have to be in class to read them!

Why Problem Solving is Good (and When it Isn't)

Obviously in physics, simply reading books is not enough. It is very important to apply what one has read, to make models, to solve problems, and to do experiments. This is where the behavioral and the cognitive ideas really come together, because it is in their synthesis that a greater truth lies. For, although reading, problem solving, and experimental work, are all associated with behavioral models of education, they are in fact also active engagement activities, provided of course, that they are done thoughtfully and actively.

This last statement may seem trite, but "plug & chug" problem solving was always an aberration, that could and should have been recognized earlier by sensible people.

Stories from CCS Classrooms

We would like to end this little paper by narrating some brief stories about interactive teaching from actual classrooms. The first one is a description of a Mazur class in physics, but there are several others from different disciplines. They serve to illustrate better than other ways, the educational diversity inherent in this work.

Harvard Introductory Physics

Eric Mazur is Gordon McKay Professor of Applied Physics and Professor of Physics at Harvard University. He has taught introductory physics at Harvard since 1984 usually in classes of about 250 students. In 1991 Mazur developed Peer Instruction9, the basic goals of which are to exploit student interaction during lectures and focus students' attention on underlying concepts. Instead of presenting all material at the same level of detail as covered in textbook or lecture notes, lectures consist of a number of short presentations on key points each followed by a "ConcepTest" - short conceptual questions on the subject being discussed. The students are first given time to formulate answers (and enter them via a CCS) and then asked to discuss their answers with each other. This process (a) forces the students to think through the arguments being developed, and (b) provides them (as well as the teacher) with a way to assess their understanding of the concept.

In this lecturing format Mazur uses about one third of each lecture period for ConcepTests. He does not reduce the amount of material covered in the course, but instead requires students to read the textbook & lecture notes ahead outside lectures. At the beginning of each class students receive a brief quiz (via the CCS) on the reading assignment. Mazur can see before he begins the class how well the assignment has been done.

The advantages of Peer Instruction are numerous. Students report that they understand the subject better, enjoy classes more, do more thinking in classes, come to class better prepared, and feel that the professor is more in touch with their difficulties in the course. In addition gains in conceptual understanding are double those when he taught the course conventionally9.

Univ. of Massachusetts Physics & Harvard Business School Managerial Economics

Prof. Elon Kohlberg at the Harvard Business School uses techniques similar to those pioneered by the team at UMass led by Profs. Bill Gerace & Jose Mestre. Both address key issues by presenting relatively difficult conceptual questions likely to cause splits in class position and stimulate discussion in their 100-person classes. In the following example from one of Kohlberg's classes, the first histogram was split more-or-less evenly three ways. Kohlberg said, "I see that there are some differences of opinion on this topic. Would someone whose answer fell into the green bin like to explain your reasoning?" then he continued in a similar way for the other bins . "Now would anyone like to change their answers?" the new histogram was split two ways "This is interesting, I see we are still in disagreement, "

At this point a good-looking young man in the front row - his face flushed red - said, "You can see we don't know where you're going with this. Why don't you just tell us what you want?"

Elon replied, "Suppose you were in a town & wanted to find a bar. If I told you where it was, you might learn less about the town than if you had to find it for yourself, - you also may have less fun doing it."

The young man hesitated, pulled out a scratch pad and began writing. A few minutes later he presented an argument that produced unanimity in the class.

Tabb Middle School, Tabb Virginia, 8th Grade - Math

Jan Andrews is an 8th grade math teacher. Students push to get into her class to enter their homework on a networked calculator. Jan uses a free-form five-question skeleton set for collecting homework. Every day she identifies five of the previous night's homework problems on the board. She uses a free text binning because of its simplicity and manually checks exceptions as they come in. Five minutes after the start of class she knows who did their homework, who had problems, & what these are. She will deal with them before moving onto new material.

Sandhills Community College, Pinehurst, North Carolina - Physics

Profs. Rick Swanson & Chris Roddy run a class/lab of 20 students. They intersperse lecture with questions, measurements, & discussion that can go in different directions depending on students' understanding. Students have TI-92s which they use to take data or plug into the CCS network. Swanson & Roddy cite a noticeably more positive classroom environment and students like knowing how they're doing compared to others, in a non-threatening environment. Test scores have gone up 10-15%.

University of Texas at El Paso, Political Science

Prof. Bob Webking teaches a 550 student class in Political Science in the poorest congressional district in the USA. As he pauses to ask a question one can hear a pin drop in the lecture hall. He uses a CCS to have students express their initial "common-sense" opinions. Then, through continued questioning he leads them to see pitfalls that can result from naïve points of view, points of view to which the students are already aware that they are subject.

McIntosh Elementary School, Newport News, Virginia, 5th Grade Reading Comprehension

Carol Wiatt is a 5th grade reading teacher at a school attended by children from economically disadvantaged families. She uses a CCS to have students input their interpretations of a reading exercise. She shows them the histogram on a TV screen and holds a class discussion about the different answers. On other days they answer puzzles. Students get so involved they barely realize that they're reading the puzzles. At the beginning of the 96-97 school year only 56% of children from her three 5th grade classes were expected to pass the state-mandated reading comprehension test. At the end of the year 89% passed with over 30% of the students growing 5 years (2nd to 7th grade level) in reading comprehension.

Conclusions

There appear to be three clear lessons from this work:

  1. Good questions asked in the right context have a remarkable property to transform a classroom. The environment becomes more lively and active. The atmosphere changes and becomes more "happy"! Students report that they understand the subject better which is confirmed by quantitative studies. They work harder in class, but enjoy it more. There is also evidence that they do more work out of class. Teachers become more aware of student problems with the subject matter.
  2. There is substantial science behind the pedagogy associated with CCSs. This paper sets the context from Socrates to behaviorism to cognitive science & constructivism, and uses this theoretical background to explain why and how the pedagogy works.
  3. The benefits of a classroom communication system (CCS) extend over a remarkable range of disciplines, educational levels, and institutions. From 5th grade reading in an inner-city to Harvard Business School, is quite a range. Currently (to our knowledge) the disciplines included are physics, chemistry, math, biology, sociology, economics, political science, & reading comprehension.
  4. This work is truly still in its infancy. There is a great deal still to learn about teaching, learning, and thinking. There is also a great deal to be done in pedagogy development, and in the development and use of Classroom Communication Systems.


**Invited paper presented at the 7th International Workshop "New Trends in Physics Teaching", Puebla, Mexico, May 27-30, 1999.

1 Halloun, I., and Hestenes, D., (a) "The Intial Knowledge State of College Physics Students," Am. J.Phys. 53, 1043, 1985; (b) "Common Sense Concepts about Motion", ibid 53, 1056, 1985.

2 Hake, R.R., "interactive-engagement vs. Tradional Methods: A Six-Thousand Survey of Mechanics Test Data for Introductory Physics Courses", American Journal of Physics, 1996.

3 Mazur, Eric, "Peer Instruction - A User's Manual", book published by Prentice Hall, 1997.

4 von Glaserfeld E. "Learning as a Constructive Activity". In C. Janvier (Ed) Problems of representation in the teaching and learning of mathematics. Lawrence Erlbaum, Hillslade, NJ., 1987.

5 Hestenes, D.,Wells, M., and Swackhamer, G., "Force Concept Inventory", The Physics Teacher, Vol.30, March 1992.

6 Hestenes, D., and Wells, M., "A Mechanics Baseline Test", The Physics Teacher, Vol.30, March 1992.

7 Dufresne, R.J., Gerace, W.J., Leonard, W.J., Mestre, J.P., and Wenk, L., "Classtalk: A Classroom Communication System for Active Learning, Journal of Computing in Higher Educaion, 7, 3-47, 1996.

8 Hestenes, David O., "Wherefore a Science of Teaching?", The Physics Teacher, 17, 235-242, 1979.

Publications List