#IOLchat Report: Evaluating Online Courses

Each week we meet via Twitter for #IOLchat to discuss current issues related to online learning. Participants include students, instructors, eLearning companies, schools, publishers, and instructional designers.

How can students and instructors have an impact on their academic programs? As we near the end of the spring semester at many schools instructors are asking students to fill out "end of term evaluation" forms to provide their feedback on the courses, as well as their instructors. These evaluations often suffer from low response rates and are used on occasion to vent frustration, but they serve many administrative purposes. Can the existing process be improved upon? Here's what our chat participants had to say:

What is the goal of course-level evaluation at your institution?

• This type of data collection is part of the overall process of improving instruction.
• To get feedback from the students' perspectives of the courses they have completed – perceived learning, course materials, course design, instructor performance.
• Analysis that informs the instructor's approach and selection of instructional strategies.
• Forum for student opinions and comments.
• Provide opportunity for student reflection on learning achievement in each course.
• As a way to measure student learning, demonstrate that students are reaching established learning objectives.
• Overall feedback on how the course has met the intended objectives and expectations of all stakeholders.
• In long-term studies, results can indicate trends and help identify needed changes.

How do students provide their feedback on courses in your program?

• Through formal online survey instruments made available on websites, course sites, and via email to students.
• With rubrics that explain criteria for the evaluation.
• With closed and open-ended questions that allow for rankings and comments. A combination of question types is recommended.
• Anonymously – may result in more candid feedback on issues that would not otherwise have been brought to the instructor's attention.
• As a requirement – in order to have final grades released, or other incentive.
• Through non-graded, formative assessments – also an option for gathering input throughout the course, not just at the end of the term.
• Students are encouraged and empowered to communicate with instructors and mentors at any time during a course.
• Book recommendation: Classroom Assessment Techniques
• Report recommendation: Assessment Handbook from Heritage University

How do instructors (especially adjuncts) provide feedback on the courses they teach?

• With online surveys similar to those used to gather student feedback.
• Through open online forums.
• Through a "lead instructor" or "course lead" whose responsibility it is to work with all instructors teaching a course or group of courses.
• As members of a group of instructors communicating regularly about course design, what is working and what isn't.

How could existing methods of course evaluation be improved? What would you change if you could?

• Use of evaluation results as the place to start discussions among instructors, course designers, and curriculum developers.
• Research needed to see if mid-course evaluations are effective at catching things the end-of-course evaluations may not, and comparisons of the two.
• Increase student willingness to complete the evaluations with meaningful input – they are often motivated to complete only because of  "very positive or negative class experience."
• Invite more instructors to participate in various tasks related to online course design and curriculum planning.  
• Course designers and curriculum developers are "often unknown" to instructors – would be helpful to have real-time conversations instead of just relying on survey responses.
• Culture of evaluation may be different across institutions, departments, programs, instructors, etc.
• There's a need to keep lines of communication open throughout a course.
• Communicate clearly to students that evaluations don't affect final grades, as well as more information about timing of results, how they are reported, and when instructors have access to this information. 

Thanks to @SaeedMobarak, @TamraExcell, @HigherEdOnlineL, @DrBruceJ, @_Lazaward_, @ODU_DL, @NoodleEducation, @TruOpenLearning, @aplusdrb, @CourseTech, @AHilbelink, @nicole_legault, @GreatTwitTips, and @HigherEdOnlineL for participating in and sharing the live event! Help us to continue the discussion by adding your thoughts via the comments area on this page.

For more from the most recent live session, review the chat feed below. Our past chats can be found on the archives page.

Follow us (@OC_org) and plan to attend our next chat. We meet on Wednesdays from 12pm-1pm ET and look forward to hearing your perspective.

This week's read-aheads:

Reading Between the Lines of Online Course Evaluations [abstract] from the Journal of Asynchronous Learning Networks

Tips for Designing Course Evaluation Forms from Duke University

UNC's Change in Online Course Evaluation Leads to Participation Decline Across Departments from The Daily Tar Heel

Examples from George Washington University, University of Southern California, and Boston College.

Online Course Evaluation-Literature Review and Findings [PDF] from Columbia College Chicago

This week's chat feed: