Volume XL, No. 18| May 17, 2018
Launching a Collegewide Online Course Quality Review Initiative
Online learning is growing rapidly in higher education. However, student success metrics in online courses often lags behind face-to-face sections of the same courses. The same is true at Harper College (HC), a community college that serves more than 25,000 students annually in Chicago’s northwest suburbs. To move the needle on students’ success in distance courses, HC launched a collegewide distance course quality review initiative.
Building the Team
The initial drivers behind the quality review process at HC consisted of two instructional design specialists and the dean of teaching, learning, and distance education. This team could not have developed an effective change initiative without the expertise and leadership of HC’s online faculty. The two instructional design specialists and the dean, along with five full-time faculty members with distance education experience, created the Online Teaching Practices Workgroup (OTPW) under HC’s Academic Standards Committee. The OTPW’s task was to identify a rubric and process usable for reviewing online courses and present the findings to the Academic Standards Committee. The Committee then voted on the implementation of the rubric and process determined by the OTPW.
Identifying the Rubric
The first task of the OTPW was to identify the course design rubric that would be used to review online courses. Several rubrics were considered, and the Open SUNY Course Quality Review (OSCQR) rubric was selected. The OSCQR rubric was developed by the State University of New York system and was adopted in 2016 by the Online Learning Consortium (OLC)—an online community of higher education leaders and educators—as their recommended online course review rubric. The OTPW decided that the OSCQR rubric is a good fit for HC because it is research-based, free, customizable, non-evaluative, and easy to use.
The OSCQR rubric approaches course reviews as a professional development exercise designed to help faculty use effective practices and standards to improve the quality, effectiveness, and efficiency of their online courses, rather than as an online course evaluation or quality assurance tool.
The OSCQR rubric contains 50 standards for quality course design and accessibility. The standards are grouped into six focus areas:
- Course Overview and Information
- Technology and Tools
- Design and Layout
- Content and Activities
- Interaction
- Assessment and Feedback
To complete a review using the OSCQR rubric, course instructors and faculty peer-reviewers compare courses against each standard and note whether the standards are sufficiently present in the courses, or whether courses require revisions to meet all of the standards. Peer-reviewers are encouraged to recommend specific actions instructors can take to ensure courses meets all six standards, and to provide positive feedback about areas where courses excels.
The rubric is flexible and designed to be used in a variety of ways. Although the OTPW uses the rubric as part of a formal online course review, the rubric can also be used by individual instructors who want to self-assess, review, revise, and improve the instructional design of their existing online courses.
Training on the Rubric
Once the rubric was selected, the OTPW adjusted some of the OSCQR rubric standards to better meet HC’s needs. Using a two-week online course, faculty members within the OTPW were then trained in how to administer the rubric. During the online training course, faculty members received background information about the OSCQR rubric, resources related to each standard, and access to practice course reviews. OTPW faculty members who completed the online course became the first HC faculty peer-reviewers.
Piloting the First Reviews
After the OTPW faculty members were trained to become OSCQR rubric reviewers, they were provided with opportunities to review actual courses offered for credit at the college. Four online courses were identified for the pilot. These courses were chosen because they were taught by OTPW faculty members who were open to the experience and eager to receive peer-review feedback.
The OTPW set up OSCQR rubrics for each pilot course and created shells of the selected courses in HC’s learning management system. These shells contained the courses’ lesson plans and all of the content covered in the courses. The peer reviewers referred back to the shells when evaluating the courses using the OSCQR rubric. Faculty members whose courses were being evaluated by the peer reviewers also evaluated their own courses against the OSCQR standards. Two members of the OTPW were assigned to each course as peer-reviewers, which gave the faculty self-reviewers two sets of feedback for their courses. In some cases, the self-reviewers and peer-reviewers met after the reviews to discuss the feedback, and in other cases, the self-reviewers independently reviewed the written feedback from their peer-reviewers.
After the pilot reviews were completed, the OTPW convened to debrief about their experiences. The OTPW decided that instructors would complete a self-review for each of their online courses and receive feedback from one peer-reviewer using the OSCQR rubric. The group voted unanimously to recommend implementation of the OSCQR rubric and the process described above to the Academic Standards Committee.
It is important to note that the course review process is a professional development process and not an evaluation. Faculty members who go through course reviews are encouraged to take into consideration the peer-reviewer’s feedback, as well as their own discovery from using the rubric. However, there are no requirements that they make modifications to their courses.
Negotiating the Faculty Contract
At the same time the OTPW was completing their pilot reviews, the full-time HC faculty contract was renegotiated to state that all distance courses are subject to a design review, and that faculty must go through the review to remain eligible to teach their distance courses.
To address the new contract language regarding distance course reviews, two OTPW faculty members presented the OSCQR rubric and process to the HC faculty senate. The presentation provided an opportunity for faculty to ask questions before the final online course review proposal was made to the Academic Standards Committee. Working with the faculty senate provided the OTPW with important feedback, such as making sure that the review results were kept confidential between the self-reviewers and their peer-reviewers.
Final Vote by Academic Standards Committee
Throughout this process, members of the OTPW presented updates to the Academic Standards Committee. After the presentation to the faculty senate, the OTPW presented the OSCQR rubric and process to the full Academic Standards Committee for a final vote, and the rubric and process were adopted.
First Semester of Campuswide Online Course Quality Reviews
The OTPW and administrators identified 17 courses to go through the first review cycle during the fall 2017 semester. Priority was given to courses that were (a) being marketed as part of an online degree program, (b) high enrollment general education courses taken early in a student’s academic journey, and (c) courses with a large success rate gap between face-to-face and online sections. Faculty who taught the 17 selected courses within the past two academic years were required to participate in the course reviews. All 17 fall 2017 course reviews were completed, and 35 more are underway for the spring 2018 semester. The OTPW will increase the number of reviews each semester in order to evaluate all distance courses within five years.
Evaluating Impact
After the first full year of course reviews, we are going to examine student success outcomes to evaluate the effectiveness of the current process. We will compare students’ success rates in courses before reviews to success rates one semester after reviews are completed in order to evaluate the effectiveness of any course modifications that were made. Although course modifications are not required for reviews to be completed, we plan to examine the kinds of changes made to reviewed courses using the peer-review and faculty self-reported feedback.
Key Takeaways
Starting an online course quality review initiative is quite a large task. To gather valuable input and ideas, involve as many faculty and staff in the review process as possible. When beginning the initiative, it is critical that all individuals supporting the review process are on the same page, that they work together, and that they communicate the same goals.
Other colleges considering implementing a collegewide course design review process may want to consider a non-evaluative, peer-based approach such as the one taken by HC. Viewing online course design reviews as a professional development opportunity, instead of as an evaluation, provides a better environment for collaboration and sharing of expertise between faculty and staff.
There are many quality review rubrics available to choose from that focus on a variety of ways to evaluate online courses. If you are interested in completing a self-directed quality review for online classes to increase your students’ success rates, begin the process by searching for a rubric that meets your needs as well as the needs of your courses.
Melissa Baysingar, Instructional Design Specialist
Jenny Henrikson, Instructional and Distance Education Design Specialist
For further information, please contact the authors at Harper College, 1200 West Algonquin Road, Palatine, IL 60067. Email: mbaysing@harpercollege.edu and jhenriks@harpercollege.edu
Opinions and views expressed are those of the author(s) and do not necessarily reflect those of NISOD.
