Dan Tobin: Judy, NCIP (the National Center to Improve Practice) was one of EDC’s first major experiments with online professional development—actually predating the Internet. Did NCIP start out with the agenda of researching emerging technologies for professional development, or did that evolve over time?
Judith Zorfass: From the beginning, we envisioned using an electronic network. With WGBH [Boston’s public television station] as our partner for the first three years, we were very much focused on new media. We were interested in testing ideas about how we could foster better communication within the field, so we ran discussion forums and special events that lasted three to five weeks. Every couple of weeks, we would bring in a new guest expert. We even tried doing action research online around when and how keyboarding should be introduced to students with and without disabilities. We designed a structure that asked participants to describe what they planned to do, report back after they tried activities in their classrooms, and then reflect on what they were learning. But we found that structure too rigid; it tended to interfere with communication rather than facilitate it.
DT: Was it difficult to get people talking to one another?
JZ: Not really. We started with a small group. Little by little, as each person’s identity and personality emerged, people became “visible” to one another. And they contributed to this feeling that they were becoming a community. In fact, one of our biggest problems was figuring out how to deal with people who were using NCIP as a way to talk about their personal problems, like their own children’s disabilities. We worried about keeping the conversation professional while still allowing people to discuss their personal issues.
Glenn Kleiman: We had a similar issue recently with an extension class I am teaching at Harvard. One of the participants wrote in his introduction about leaving his job as vice president of a major corporation and becoming a teacher as a result of discovering he had a life-threatening illness. It opened up a good discussion about why we do what we do. He said he hesitated before he pushed the “send” button, but figured he’d go ahead. What was interesting is that the class is part online and part face-to-face, and people were more revealing about themselves in the online introductions than when they introduced themselves in class.
Robert Spielvogel: In the early days, there was anonymity in being part of an online community. In the Math Learning Forums, we had several teachers who said, “I can talk about things in this forum that I wouldn’t want the teacher next door or my principal to know about—such as my weaknesses. In my school, you don’t expose yourself in that way.” That was certainly a powerful factor when people were just getting used to this new medium. There was a sense of community without a lot of risk.
Facilitation
JZ: A lot of it also comes down to the skills a facilitator brings to the discussion. We used to have facilitated topical discussion forums, and we worked very hard to find facilitators who were not only respected in terms of their content knowledge, but also comfortable enough with the medium to draw people out. Part of our role became supporting the facilitators, who were not part of our project staff. We had to guide them so that they could create a safe and comfortable environment.
DT: You actually made the art of online facilitation a key area of the project research.
RS: Very much. Projects like NCIP weren’t focused primarily on trying out new technology. These were actually deliberate experiments with the goal of bringing a community together for reflective discussions. We treated them like serendipitous communities, where we could try out different tools and see how people responded. One of the important innovations in NCIP was that it wasn’t simply a discussion forum or an online workshop. Judy put a lot of energy into other things as well, such as a Web tour using text, video clips, and photographs, and a resource library that would attract people even if they weren’t initially sure about participating in the discussion. There was a lot to attract people to the NCIP environment, and hopefully some would stay and take part in the community discussions.
That’s similar to what we did with the Math Learning Forum, which had three stages. In the first stage, teachers watched classroom videos focused on a particular theme of the National Council of Teachers of Mathematics Standards, which were new at the time. The teachers followed up by doing a mathematics activity themselves and talking about their own mathematical understanding. And then they tried the activity with their students and reflected on the experience online. It was a sequence of getting teachers to think deeply about math, and each stage provided a common denominator for a good discussion.
JZ: We were really able to develop the NCIP environment much further when we moved to the Web. Starting with what we knew about best face-to-face professional development, we asked ourselves what strategies we could transfer to this new environment, given the Web’s capabilities. For example, people love to visit other classrooms and then debrief about what they’ve seen. So we designed an online tour of early childhood classrooms where students with disabilities were using a range of technology tools, and we used the same kind of conceptual framework we would use for an actual visit. When you go on a visit, you have an orientation, you visit classrooms, you talk to teachers afterward. We tried to build all that into our Web environment. We even added a “souvenir” stop—a resource library with information about all the technologies on the tour.
DT: Glenn, how did the Leadership and the New Technologies (LNT) website and online workshops evolve from the face-to-face LNT Institute?
GK: When we first started planning the Institute, Bob, Margaret Honey, and I (along with Marilyn Resnick from the AT&T Foundation) talked about the idea of making the one-week institute part of something that continued online. We also found that we had, even in the first year, about four times as many applicants as we could accept. By using the Web, we could extend the reach of the Institute to many more people. At first, we simply wanted to document the Institute in great detail. Then we started experimenting with follow-up online discussions. We found early on that our audience consisted of very busy professionals—principals, curriculum coordinators, media specialists, and others in leadership roles—and we had to convince them that the online community could be valuable to them.
DT: Was it hard to disabuse people of the notion that online discussions equal chat?
GK: There are many online communities consisting of college students who hang out in the dark at night and like to mess around on the Web. Our audience was not about to hang around online hoping something beneficial would come along. They need to know that their time will be well spent to meet their professional needs. We did try some listservs without success, so we moved on to the workshop model. We found that when we framed the discussion as an online workshop with a focus and facilitators, running over six or eight weeks, there was a lot of interest.
RS: Some formats lead to deeper, richer learning experiences. The workshops we run have an order to them; they have a beginning, a middle, and an end. Most online listservs are amorphous. You jump into the middle of the stream, and you stay as long as you want—which is fine, if you just want a how-to or something like that. But it is not good when you are trying to build a sense of community for reflection on deeper issues. You need groups with common experiences, groups who can be mutually supportive.
DT: Let’s talk about the structure of some of these workshops. Judy, you’ve experimented with many different ways to structure online workshops. What has worked well?
JZ: In one course, we focused the whole discussion on the case study of one child. Everybody had to work through the case—understand the child’s strengths and needs, the curriculum goals, the environment, and the tools, and then jointly work through the question of how you match technology for the specific needs of a child with disabilities. That was very successful. We also designed an online workshop around the early childhood tour I spoke about, which served as a terrific springboard for conversation.
One of the other big challenges is structuring the discussion itself. As the discussion grows, some people report that they don’t have the time or energy to follow the conversation thread. They are afraid they are going to say something that has been said four messages back. We needed to find ways to capture the ideas and create a semantic map—the kinds of things you might do with chart paper at a workshop. So you have to think about group dynamics, different ways people approach reading and writing, and information processing.
GK: That’s a critical point. We write weekly summaries of the discussions. That kind of summary is incredibly valuable—particularly because you can link from the summary to the original messages. But we’ve found that it’s also very time-consuming, trying to distill 75 or 80 messages. It takes someone who really understands the field and who can think very carefully.
DT: Can you predict who is going to be a good online facilitator? Are there good face-to-face trainers who aren’t good online?
GK: Absolutely. Online facilitation is very different from face-to-face teaching because you don’t have as many control points. I feel that I have much more control to set a style of interaction and the culture of a class in a face-to-face classroom than I do in an online environment. When people go off-topic online, you may not even notice for a day or two. And then you don’t have the equivalent of giving them a look or using humor to get the discussion back on track. You don’t have the sort of casual tools that are very effective in a classroom setting.
To make up for that, some people tend to overfacilitate online—saying too much too quickly. Often, saying less is more effective. Jamie McKenzie, who is well known in the field of educational technology, served as a guest facilitator for us, and he was great. He just had a way of putting out provocative statements that got people rolling. He came in with very clear, succinct questions and observations that captured the essence of the issue and moved the discussion along well.
JZ: We had one facilitator who felt responsible for sending a message to everyone who posted a comment. That was interesting because although it was a lot of work, she was able to establish a personal bond with participants, and they in turn became more invested in learning online. But this was an unusual situation. It is more common for a facilitator to post some sort of general statement to the whole group, followed up with selective, pointed comments. It’s also interesting to look at what the facilitators reveal about themselves; what are they are willing to say about themselves and their lives?
RS: There are levels, like the guy Glenn mentioned who wrote about his dramatic life change and health issues. That pushed the border, but the class responded well. If the facilitator had said something like that at the beginning of the discussion, it might have been too revealing. It might have made people uncomfortable.
Subtle Differences
DT: In some ways, it sounds like you are trying to replicate the power of a face-to-face workshop. But you also have tools—like archives and hyperlinked resources—that are over and above what you could do face to face.
GK: I wouldn’t say over and above—just different. For example, there is the tremendous energy you can get when you are brainstorming in a classroom—the way people build off one another’s ideas. We haven’t found a way to replicate that online. On the other hand, you often see more thoughtful, reflective contributions online. People have listened to others more carefully because they have been able to read and reread what others have contributed. They don’t have to do that dual tracking of listening to what others are saying while thinking about what they want to say next.
JZ: Being known by what you put in writing is very different from being known through speech. The persona that you present to the world looks different; it may push you to be more reflective, or it may inhibit you because you can’t capture the ideas in writing the way you would like to. It has to do with the literacy skills you bring to the online discussion, and that is a critical dimension.
GK: There are many categories of people who, for one reason or another, are not comfortable speaking up in class. Sometimes you find that students posting very thoughtful pieces online won’t even say much in class—such as ESL students who may feel more confident with their written English than their spoken English.
Professional Development Models
DT: It occurs to me that since this is a new medium, we might have more power to model ways of interacting than we do in more traditional forms of professional development. Do online tools provide new possibilities for professional development?
GK: Again, there are advantages and disadvantages that come with the medium. For example, we’ve discovered that a six-week online workshop, as a rough guideline, can cover about the same content as an intensive weekend workshop. But that gives it a very different character. Because it is spread out over time, it can be more “job embedded,” to use the current jargon. We can, for example, have teachers design a lesson with some of their colleagues, try it out, and then report on what the students did—even sharing some of their students’ work.
RS: I think it is pretty universally agreed that the one-shot model of professional development is not as good as continuous professional development over the length of a career. Online environments can be an important tool for encouraging people to take a long-term approach to their own professional development.
JZ: When you talk about collaborative professional development, the context and the goals of the project matter more than the medium. In our LINK*US project, we work within school districts, so everything is job-embedded and aligned to the district’s initiatives. It’s co-constructed with the district so that it meets the district’s specific needs and standards. That’s very different from trying to, say, build an online community made up of teachers around the country. It’s a different goal.
GK: You can also use an online adjunct to support the work you’re doing within a district—as we do in EdTech Leaders™ Online. Online and face-to-face professional development can support one another; it’s not an either/or situation.
RS: In Schools Around the World (SAW), most of the change in practice will come as a result of local teams working together to examine student work across grade levels and to study their own expectations. We’re finding that even the process of preparing for a SAW course sparks valuable offline conversations for local teams as they work together to decide which cases to submit to the database and who will participate in the international exchange. One reason that all of us are working with school teams is that the research on systemic change says that you have to build a critical mass within a district in order to change the culture. You don’t want to end up with just a few isolated pockets of people invested in changing their practice.
JZ: I’m currently involved in designing an online component for a 12-week literacy seminar. We expect that this seminar will be used by study groups that are part of whole-school reform efforts, such as ATLAS. We’re trying to figure out how to use the Web to connect these study groups across the country. What is the value added to their face-to-face collaborative experience? We haven’t fully answered that question yet.
RS: Right—we have to take a hard look at the value added in online professional development overall. We have gone from being pioneers in online courses to the point where almost every university in the country is offering them. People are inundated with courses from almost every professional organization and society and technical systems provider. But what do these courses offer that you cannot get from other places? We add value by creating databases and other online resources, and by linking the course to face-to-face professional development. Or we design the course in a way that allows the work participants do to contribute to a teacher portfolio they are developing for national board certification. All those kinds of things embed the course in a larger piece of professional growth.
DT: Are you worried about the number of providers rushing to do online professional development?
GK: I worry about the noise level. Too many organizations are providing online courses of all sorts—some of high quality, but others with little understanding of what’s involved in effective online professional development, and minimal quality control. As the field develops, we just focus on doing our work well, while also conducting careful research and working toward articulating principles and guidelines that can help others in the field.
RS: To toot our own horn a bit, I think EDC’s strength is that our approach is never about the technology alone. It is always about adding value. So, we’ve gone beyond the point of simply offering a course; we’re now focused on creating an environment. In SAW, for example, the database of student work is, in itself, incredibly valuable. But now we are creating Web tours in which an educational expert like Grant Wiggins comes in and comments on the student work along with the teachers. That will give people who are not yet ready to take the course a feeling for the power of looking at student work. The end goal is not necessarily to have people take a lot of online courses. In SAW, the goal is to get teachers from around the country to study student work as a basic professional activity.
Impact
DT: How do you measure the impact of online environments on classroom practice?
GK: While our focus up to this point has been on developing and refining the models, we do distribute fairly extensive surveys to every participant in our workshops, which provide very useful feedback. We ask specifically about each component of the course, each type of activity, and we use that to refine the workshops. Like Judy’s group, we’ve spent a lot of time hashing through examples, dealing with design issues, getting the kind of feedback we need. By running a large collection of courses, we have a body of what I think of as “naturally occurring phenomena.” Now we’re ready to move forward with more formal research projects and will be looking for funding to add a larger research component.
JZ: In the early days of NCIP, we used Nancy Brigham’s tracer design methodology to see if our target audience of change agents was passing the information on to others. Did they download material, share the resources with others, and find ways to use the resources in workshops or training sessions? The goal is to see why and how your materials found their way to teachers in classrooms. That kind of evaluation helped us measure the impact NCIP was having in the classroom.
RS: For the Passport to Knowledge evaluation, we developed a system of message analysis for online discussions. We examined how the discussion changed over time by coding each message throughout the discussion. Who is the author? What is the content of the message? Is the participant responding to the facilitator or to another participant? In the early weeks of a discussion, a lot of postings may have simply been requests for technical assistance. In the later stages, there were more messages about the student work or the course content.
The point is that all of these research strategies attend to the context of the discussion. That’s a principle that cuts across a lot of EDC’s work: Context is everything. So we’re not going to research the effectiveness of online professional development by using a pure experimental design—where you control all the variables to isolate a comparison between online and face-to-face professional development. That is almost a worthless endeavor because it’s so artificial. Rather than viewing these technologies as new variables to test in isolation, a more sophisticated approach is to ask, how do they contribute to the field and change the culture? The value of an approach should be measured by how well it can be replicated to meet the real needs of a wide variety of people. That’s what Glenn and Judy are doing: replicating effective approaches with many different people. As you do that, you start to learn more than you would by waiting to do very controlled studies.