Research Findings on Educational Technology

Provided by the ESD 112 Ed Tech Support Center
Debbie Tschirgi, Director

Research review conducted by:
Rita Hale, School Improvement Specialist
Educational Service District 112

Organization:

Impact of Technology on Student Achievement
Impact of Technology on Student Engagement
Impact of Technology on Educational Systems
Factors for Technology that Favor Desirable Outcomes

Impact of Technology on Student Achievement

GENERAL FINDINGS:

Ten meta-analyses synthesized research from 946 studies, ranging from preschool to college. Each meta-analysis concluded that instructional programs that included technology show a positive impact on student achievement, resulting in higher test scores. The key findings from these studies show:

1. Classrooms in which computers were used to support instruction usually showed gains in student achievement as measured by standardized achievement tests. The magnitude of gains varied from study to study. There was usually a good match between the desired outcome of the treatment and the outcome that was measured.

2. The effectiveness of different applications of computer-assisted instruction varied by the content area and the skill being taught. In general, applications fared better if delivered in a content area with a defined structure, such as mathematics.

Meta-analyses of computer-based instruction and multimedia applications indicate that the effectiveness of educational technology on improving student achievement depends on a match between the goals of instruction, characteristics of the learners, the design of the software, the technology, and the implementation decisions made by teachers (Sivin-Kachala, & Bialo, 1993).

The Software Publishers Association (SPA) commissioned an independent meta-analysis of 176 studies focusing on the effectiveness of technology in schools. This report concludes that the use of technology as a learning tool can make a significant difference in, among other things, student achievement as measured by standardized tests (Sivin-Kachula & Bialo, 1996).

In a study conducted in New Zealand, researchers found the use of computers contributed, with other instructional innovations, to higher performance on English, mathematics, and science test. The study was conducted with eighth-, ninth-, and tenth-grade students (McKinnon, Nolan, & Sinclair, 1996).

An extensive study of 55 New York State school districts also points to the same conclusion: increased technology supports, facilitates, and encourages student achievement (Mann & Schaffer, 1997).

Research reported by the National Center for Education Statistics (NCES) indicated that the primary factor in test score differences was family/social backgrounds. The second most important factor was instructional opportunity. According to NCES, it is the way that computers are used to change and enhance curriculum that represents the most important factor in determining whether or not computers have an impact on achievement. One of the strengths of this was that the control group received a valid, well-designed instructional reading program. Analysis of the standardized test scores pre- and post-treatment indicates that all groups improved to well above the mandated cutoff score by the end of the study. Mean scores for the treatment groups were higher than the control group means (Coley, Cradler, & Endel, 1997).

LEARNING “FROM” TECHNOLOGY

CAI – computer-assisted instruction
CBI – computer-based instruction
CEI – computer-enriched instruction
CMI – computer-managed instruction
ILS – integrated learning system

Kulik and Kulik (1991) conducted one of the most comprehensive studies of the effectiveness of using computers to increase student achievement. In 81% of the studies examined, the students in the computer-based instruction (CBI) classes (experimental group) had higher exam scores than students who were taught by conventional methods without computer technology (control group).

One type of computer application that usually results in positive gains in achievement at the elementary and secondary levels is computer tutoring (Kulik, 1994).

The West Virginia Department of Education and the Milken Exchange on Education Technology released a very comprehensive study entitled “West Virginia Story: Achievement gains from a statewide comprehensive instructional technology program” (Mann, Shakeshaft, Backer, & Kattkamp, 1993).

Solomon, in an Afterword of the study, indicated “that not only is there a statistically significant relationship between Basic Skills/Computer Education (BS/CE) and test scores gains; and not only can these gains be translated into effect size comparable to those of other interventions, but also, the gains from programs that update BS/CE’s positive features can be achieved at a much lower cost than could similar gains from a currently very popular alternative intervention, namely class size reduction” (p.50).

An examination of research studies that investigate the impact of technological tutoring systems on student achievement shows mixed results (Wilson., 1993; Butzin, 2000).*

Some reports suggest that CBI, CAI and ILS(s) can improve student’s basic skills in such disciplines as mathematics (Koedinger, Anderson, Hadly & Mark (1997).*

Other studies report that, in some instances, the use of technology to teach basic skills had a negative impact on academic achievement (Wenglinsky, 1998).*

Evaluations showed that Algebra students in a high school in Pittsburg, PA who used “Cognitive Tutor” out-performed students in traditional classes, having achievement gains up to 25 percent in skill and up to 100 percent in problem solving. Retention in mathematics classes and attendance also improved among students using this tutor. (Hubbard, 2000)*

In a decade-long study, researchers investigated the impact of a computer-integrated instructional program called Project CHILD (Computers Helping Instruction and Learning Development) found that elementary students in project classrooms from kindergarten through fifth grade have consistently had “higher test scores and better discipline than their counterparts in traditional, self-contained classrooms” (Butzin, 2000, p3).*

A variety of meta-analyses conducted between 1985 and 2000 on the impact of CBI, CAI, ILS, drill-and-practice software, and computer tutorials on student achievement report that students using computers had higher test scores, typically measured on standardized achievement test. In 1994, for example, Kulik aggregated findings from over 500 individual studies of computer-based instruction. These studies showed percentile gains on achievement tests of 9 to 22 percent over control groups. On average, students who used computer-based instruction scored at the 64th percentile compared to students without computers who scored at the 50th percentile. Kulik also found that computer-based instruction can decrease the amount of time required for students to learn basic skill. (Kulik, 1994)*

LEARNING “WITH” TECHNOLOGY:

In a meta-analysis that examined the impact of technology on student learning, researchers found increased teacher-student interaction, cooperative learning, and most importantly, problem-solving and inquiry (Statham & Torell, 1996).

Many researchers investigating the use of technology in education have found that technology is most powerful when used as a tool for problem-solving, conceptual development, and critical thinking (Culp, Hawkins, & Honey, 1999; Sandholtz, Ringstaff, & Dwyer, 1997; Means, 1994).*

One of the most powerful uses of technology in education is to tailor instruction to students’ individual learning needs. Technology can provide the means for students with special needs to communicate via email, and to use the Internet for research. It can also can help teachers accommodate students’ varying learning styles (Silverstein et al., 2000).*

The Apple Classrooms of Tomorrow (ACOT) project was a 10-year study that set out to investigate how routine use of technology by teachers and students would affect teaching and learning. When compared to their non-ACOT peers, students “routinely employed inquiry, collaborative, technological, and problem-solving skills uncommon to graduates of traditional high school programs” (Sandholtz et al., 1997).*

In a longitudinal study researchers investigated the impact of project-based learning using multimedia classroom projects were much more student-centered than non-project classrooms, and were “organized around the collaborative construction of complex products” (Penuel, Golan, Means, & Korbank, 2002).*

A meta-analysis that examined the impact of technology on student learning found increased teacher-student interaction, cooperative learning, and most importantly, problem-solving and inquiry. One essential condition for student learning to take place: Computers should be used less for drill and practice in the classroom and more as open-ended thinking tools and content resources. (Statham & Torell, 1996).

Technology supports exploration, which helps students set achievable goals, form and test hypothesis, and make discoveries of their own (Collins, 1990).

Using technology to support collaborative knowledge integration includes tools that enable group thinking, problem-solving, and task orientation. Sharing real data from primary sources with experts leads to students thinking and working the way experts do (Evard, 1996; Federman & Edwards, 1997; Jonasses & Reeves, 1996; Lonergan, 1997; Scardamalia & Bereiter, 1991; Wideman & Owston, 1993).

Technology encourages student collaboration, project-based learning, and higher order thinking (Penuel et al., 2000).*

To top

Impact of Technology on Student Engagement

Student Engagement Indicators:
Responsibility for learning – take charge, self-regulated learners
Strategic – develop and refine learning and problem-solving strategies
Energized – excitement and pleasure, intrinsically motivated
Collaborative – have and value skills to work with others
(NCREL Learning with Technology Profile Tool)

Research studies that focused on technology and students’ motivation to learn relied on self-reports of students’ attitudes toward computers and found, in general, that most students considered computer activities to be highly motivating and interesting (Gregoire, Bracewell, & Lafferriere, 1996; Heidmann, Waldman, & Moretti, 1996; Kendall & Broihier, 1992).

Reports of best practice and program evaluations show that students become energized and engaged when given the leeway to explore, inquire, and make connections between their prior knowledge and new-found answers to their questions about the way the world works (Boyer, 1995; Johnson & Johnson 1996).

An early review of the research concerning the effect of computers on students’ attitudes concluded that:
• Computer use most affects attitudes toward school and subject matter.
• Computer use appears to have a positive impact on improving students’ self-image and self-confidence (Roblyer, Castine, & King, 1988).

Teachers also use the Internet for student activities. Fifty percent of the teachers with classroom connections had their students use the Internet at least three times as part of a lesson. However, only 7 percent of the teachers had their students use the Internet to communicate with other students or to post their work (Becker, 1999).

Except for math teachers, there was no statistical difference in use of the Internet in content areas. While studies show that math teachers are among the highest users of technology in their classroom, they were the lowest users of the Internet (Becker, 1999).

Numerous studies have reported that technology can lead to increased student motivation and improves self-concept. Software and Information Industry Association (SILA) examined 311 research reviews and reports from published and unpublished sources. They concluded that technology has been found to have a positive effect on student attitudes toward learning, self-confidence, and self-esteem (Sivin-Kachala & Bialo, 2000).*

Reviews have reported that technology has been found to improve school attendance, decrease dropout rates, and have a positive impact on students’ independence and feeling responsibility for their own learning (Coley, 1997)*

In ACOT classrooms, teachers reported that students displayed increased initiative by going beyond requirements of assignments, and that students spent more time on assignment and projects when working on computers. Students often chose to use technology during free time, and before and after school. (Sandholtz et al., 1997).*

Pervasive perception among teachers is that computers have improved the climate for learning by increasing student motivation in subjects for which they use computers (Gregoire, Bracewell, & Laferriere, 1996).

To top

Impact of Technology on Educational Systems

The more advanced uses of technology support the constructivist view of learning in which the teacher is a facilitator of learning rather than the classroom’s only source of knowledge (Trilling & Hood, 1999; Penuel & Means, 1999; Silverstein et al., 2000; Statham & Torell, 1999).*

In numerous studies of students learning “with” technology , teachers have reported that technology encourages them to be more student-centered, more open to multiple perspectives on problems, and more willing to experiment in their teaching (Knapp & Glenn, 1996).*

Results of a variety of studies suggests that, over time, technology can serve as a strong catalyst for change at the classroom, school, and district level (Hawkins, Spielvogel & Panush, 1996; Means, 1994; Chang et al., 1998).*

Evidence also exists that technology will have a stronger impact when technology integration is a part of the broader-based reform effort (Sandholtz et al., 1997).*

Schools that have embraced technological change in instructional delivery have seen dramatic improvements consistent with school restructuring (Bozeman & Baumbach, 1995).*

“Technology supports exactly the kinds of changes in content, roles, organizational climate, and effects that are at the heart” of constructivist educational reform movements (Means et al., 1993 p.1).*

Many proponents of the current reform efforts see technology as a vital component of a new educational paradigm in which curriculum, teaching methods, and student outcomes are reconceptualized (Means, 1994).**

Student engagement remained highest when technology use was integrated into the larger curricular framework, rather than being an “add-on” to an already full curriculum (Sandholtz et al., 1999).*

Research suggests that when technology integrated into larger instructional frameworks, students will not only learn how to use the equipment and software, but will also gain content knowledge (Silverstein et al., 2000).*

The Learning Return On Our Educational Technology Investment
WestEd 2002

To top

Factors for Technology that Favor Desirable Outcomes

INTEGRATION WITH REFORM EFFORTS:

Researchers examining (Project Explore, Union City, New Jersey) the impact of the technology on student achievement found a substantial improvement in students’ standardized test results. While some of this improvement can be attributed to technology, researchers noted that the importance of other restructuring efforts were occurring simultaneously. “The magic lay not exclusively in the technology, but in the interweaving of a systematic program of educational reform with the judicious use of technology-based resources” (Change et al., 1998, p.34).*

Student engagement remained high in classrooms emphasizing interdisciplinary, project-based instruction (Sandholtz et al., 1997).* Other researchers have reported similar results related to student motivation (Silverstein et al., 2000; Penuel et al., 2000).*

In a study of five technology-rich schools, researchers noted goals for student learning were clearly articulated prior to the introduction of technology (Glennan & Malmed, 1996).*

PROFESSIONAL DEVELOPMENT:

In 1989 the Technical Education Research Center (TERC) launched the Labnet project as “a technology-supported teacher-enhancement program aimed at high school physics teachers” (Roupp, Pfister, Drayton, & Gal, 1993, p.4). Researchers studying the project found that telecommunications helped teachers overcome a number of obstacles to professional development, most notably that of time for developmental activities (Honey, McMillan, Tsikalas, & Griswald, 1995).

Becker was able to determine that certain variables had “important independent relationships to teachers’ use and valuation outcomes” and some did not. Three factors were especially significant: quality connectivity, computer expertise, and teacher pedagogical beliefs and practices that were constructivist-compatible. Three other factors – adequate professional development, home access, and younger age of teachers—were next in importance (Becker, 1999).

Success or failure of technology is more dependent on human and contextual factors than hardware or software (Ryan 1991).

A variety of studies indicate that technology will have little effect unless teachers are adequately and appropriately trained (Office of Technology Assessment, 1995; Coley, Cradler, & Engel, 1997; Silverstein et al., 2000; Sandholtz, 2001)*

Follow-up evaluations of the BS/BC program in West Virginia showed that the greatest gains in student achievement occurred when teachers were trained in the use of technology (Schacter, 1999).*

Rhode Island had 25 percent of all teachers participate in Teachers and Technology Initiative. Researchers concluded that the training (60 hours and sample units and laptop computers) gave teachers “a solid foundation in the use of technology as a core component of their instructional practices” (Henriquez & Riconscente, 1999, p.76).*

In a paper discussing the cost, utility, and value of technology, Wahl (2000)* suggests that organizations should spend 30 percent of their budget on equipment and 70 percent on the “human infrastructure” to support ongoing training and technical assistance.

Helping teachers to learn to integrate technology into curriculum is a critical factor in the successful implementation of technology in schools (Sivin-Kachala & Bialo, 2000).*

In addition to receiving training on how to use technology instructionally, researchers also suggest that teachers need additional help in learning how to assess products created using technology (Penuel et al.,2000).*

Longitudinal research examining teachers’ use of technology suggests that the support teachers need changes as they become more and more proficient in integrating technology into instruction (Sandholtz et el.,1997)*

ACCESSIBLE EQUIPMENT AND SUPPORT:

Although studies are not conclusive about the optimal number of computers per classroom, research is clear that students and teachers are best served if they have convenient, consistent, and frequent access to technology (Mann, 1999).*

Statham and Torell (1999) suggest that a 1:5 computer-to-student ratio would assure students “near universal access.”

If students are to use computers to be better writers, researchers, and problem-solvers, they need to have access to computers when they are engaged in these processes, not only at regularly scheduled times (Knapp & Glenn, 1996).*

Success or failure of technology depends on having significant critical mass numbers and types of technology applications that are appropriate to the learning expectations of the activity [one computer to 4/5 students, located in classrooms or areas where learning is taking place, connected to the Internet, etc.] (Meers, et al, 1993; Sivin-Kachala & Bialo, 1996).

In Indiana, students participating in the Buddy project were supplied with home computers and modem access at school. Project students “showed improvement in all writing skills, better understanding and broader view of math, more confidence with computer skills, an ability to teach others, greater problem-solving skills, and greater self-confidence and self-esteem” than non-project peers (Coley, 1997 p.4).*

Note: Meta-analyses are procedurally objective presentations of study features and outcomes, employing statistical methods to summarize overall findings and explore relationships between study features and outcomes.

Computer-Based Technology and Learning: Evolving Uses and Expectations, NCREL 1999
*The Learning Return On Our Educational Technology Investment WestEd 2002
**Classroom Instruction in Gates Grantee Schools: A Baseline Report, Fouts and Associates 2002

Research Links

To top