Comparative and non-comparative evaluation in educational technology

Non-Comparative Study:Perceptions of Online Learning Quality given Comfort with Technology, Motivation to Learn Technology Skills, Satisfaction, & Online Learning Experience

Michael C. Rodriguez and Ann Ooms
University of Minnesota
Marcel Montanez
New Mexico State University
Yelena L. Yan
University of Minnesota
April, 2005
Paper presented at the annual meeting
of the American Educational Research Association, Montreal, Canada.


A Comparative Study: Students’ Perceptions of Online Learning:
Karl L. Smart and James J. Cappel
Central Michigan University, Mount Pleasant, MI, USA
karl.smart@cmich.edu james.cappel@cmich.edu

The “Students’ Perceptions of Online Learning” is Learners’ perception and performance comparative study and “Perceptions of Online Learning Quality given Comfort with Technology, Motivation to Learn Technology Skills, Satisfaction, & Online Learning Experience” is non-comparative study.
The Students’ Perceptions of Online Learning study examines students’ perceptions of integrating online components in two undergraduate business courses where students completed online learning modules prior to class discussion. However, the Perceptions of Online Learning Quality given Comfort with Technology, Motivation to Learn Technology Skills, Satisfaction, & Online Learning Experience study examines audience comfort with technology, satisfaction with those experiences, and perceived quality, given their experience with online or hybrid courses.
The comparative study the overall sample was fairly evenly distributed by gender (54 percent females and 46 percent males). The participants consisted of 58 percent 4th year students (classified as seniors in the United States), 39 percent 3rd year students (juniors), and 4 percent 2nd year students (sophomores), and most (94 percent) were business majors. The data for this study are based on students’ experiences taking an online learning unit offered by the Information Technology Training Initiative (ITTI) of the Michigan Virtual University (MVU). Then the participants were distributed in two courses were developed by the university:“required course” was taken by third-year (junior) and fourth-year (senior) students and the “elective course” was taken by second-year (sophomore) and third year (junior) students’. The instructors were observing student’s participant team assignments and three group exams and a major group project.In the other hand, non-comparative study was conducted to approximately 700 professional and graduate education students. A survey was as an instrument; it was developed and reviewed by instructional technology experts and researchers, and piloted with a sample of the target audience prior to publication online.

The comparative research was not about assessing the difference between online vs. face-to-face instruction; rather, this research is more about how we can use technology effectively in the classroom and how students react to it. However, the most weakness in the comparative study was that this study compared between the use of integrated online units in both elective and required course, but this integration was done without taking in the consideration of the factors that can affect the results like: learner characteristics, course content, and the learning context. In addition, the largest dissatisfaction factor reported among the participants was the time required to complete the online modules.

The findings of the observation of the comparative one showed that most evaluation measures focus on subjects’ perceptions of the online units. However, the findings of the survey showed that for students with Online course experience, comfort with technology was related to satisfaction with online course experience which was related to perceived quality; motivation to learn more about technology was also related to satisfaction of online learning experience. For students with hybrid online course experience (WebCT), comfort was related to satisfaction and motivation; motivation was also related to satisfaction; and satisfaction was ultimately related to perceived quality. In this group (WebCT), comfort was also directly related to perceived quality. Finally, among students with no online course-related experiences, comfort was related to motivation to learn about technology, but neither of these factors was related to perceived quality of online courses. For students with limited or no online course experience, perceptions of online course quality were more difficult to explain.


PowerPoint Presentation: http://www.slideshare.net/u066945/compartive-and-non