Tag Archives: assessment

In praise of the open-book exam

Whether or not students should be required to take proctored exams in an online course has been debated for as long as there have been online courses and I suspect it will not be resolved anytime soon. Nevertheless, there has been a good deal written about the subject that is worth considering.

Open book online exam

CC-BY-NC by Jared Stein on Flickr

A recent study compared proctored and non-proctored student outcomes in online courses. Lee William Daffin, Jr. and Ashley A. Jones from Washington State University, found that with the non-proctored online exams, students took about twice as long to complete their exams and performed between 10% and 15% better than students taking proctored exams.

Despite the instructions that the exam was to be taken closed-book, we might reason that students spent the extra time looking up answers in the text or searching online. Again, this brings up the issue of how to prevent cheating in an online course without requiring students to visit the testing center.

Daffin and Jones suggest trying the following methods to discourage cheating:

  • Explaining the technology used to discourage cheating (e.g. Safe Assign, Bb Monitor)
  • Creating a large enough test bank from which to draw questions
  • Randomizing both questions and answers
  • Including essay questions
  • Posting class and/ or institutional policies and penalties for cheating
  • Shortening the time frames to complete tests
  • Having students sign Code of Conduct / Honor statements

Daffin and Jones offer yet one more option…

A final possibility is to make online exams open notes/book from the start but increase the difficulty of such exams so that they are not simply testing the recollection of facts (Feller, 1994; Williams & Wong, 2009; Stowell, 2015). Though students would be permitted to utilize outside sources, eliminating student misconduct issues, they would still need a good understanding of the material to be able to accurately apply it and could not simply look up answers as they took the exam. – L.W.Daffin & A.A.Jones

In another study, Beth Johanns, Amber Dinkins, and Jill Moore, from Indiana State University, looked at the research around open-book vs closed-book exams and found there are some unique advantages for student learning with both formats. Open-book exams, in particular, were found to help learners develop critical thinking skills. While the closed-book required students expend more time and effort in preparation for the exam, they were also more likely to cram and therefore rely on short-term memorization. On the other hand, open-book exams were shown to engage learners more deeply by encouraging them to gather and critically analyze information – oftentimes from multiple sources.

In their study, Johanns, Dinkins, & Moore explored yet a third type of exam – collaborative testing. In this approach, students prepared for a closed-book exam but were permitted to collaborate with other students while completing the examination. This approach has been shown to promote creative thinking and increased metacognition.

Finally, it is clear there are both opportunities and challenges with any single approach to online assessment. Perhaps the best solution is to provide multiple options. Rather than requiring students to schedule weekly visits to the testing center, consider reducing the closed-book to the midterm and/or final exams and maybe mix things up with open-book and collaborative exams in the interim.


Daffin, Jr., L.W., & Jones, A.A. (2018). Comparing student performance on proctored and non- proctored exams in online psychology courses. Online Learning, 22(1), 131-145. doi:10.24059/olj.v22i1.107


Johanns, B., Dinkins, A., & Moore, J. (2017). A systematic review comparing open-book and closed-book examinations: Evaluating effect on development of critical thinking skills. Nurse Education in Practice, 27, 89-94. doi.:10.1016/j.nepr.2017.08.018



Customizing setting for the Blackboard Retention Center

Subscribing to Student Success

Online student retention is problematic. Online learners drop courses for any number of reasons.

Oftentimes the problem is they aren’t exactly sure what will be required in regards to time and resources for the course and are simply waiting for access before deciding whether to continue. One way to help offset this early drop phenomenon is to make sure the syllabus is available prior to registration. The syllabus should include information about the schedule, whether there are any synchronous requirements, textbook information, and any additional resources needed to successfully complete the course.

As for those students who enroll and decide to give it a go… there are some things we can do to help them get on-board, including fostering a sense of community with other students – especially those who have successfully completed online courses in the past – and making sure new to online students can find the support resources they need to navigate any technical issues (e.g. orientation to LMS, help desk, library, online tutoring, etc.).

Another way we can help is to find ways to reach out to students who are at-risk of falling behind.

Blackboard’s new “Retention Center” is designed to alert both student and instructor at the point a problem begins to emerge. This immediate feedback can help students to know where they stand in time for them to evaluate their own efforts or to seek help. Instructors can reach out to struggling students at the first signs trouble, before they get so far behind they cannot successfully complete the course.

The Retention Center replaces the former Early Alerts feature, permitting faculty to create and apply “rules” to help reach out to students who may be at-risk. Rules might include: logging in at least once every three days, or average grade falls below 79%, more than one late assignment. In the event a rule is broken, an automated message is sent to both the student and the instructor. The instructor (or other users e.g. advisor or program coordinator) may then reach out to the student ASAP to see what they can do to help.

CC-BY by Wolfgang Greller of Flickr

When Online Learning Fails

A recent article in the Instructional Technology Council newsletter shared some of the challenges and frustrations from a student’s perspective when online learning fails.

An high school student tells of her experience enrolling in two online courses during her senior year. While seeking greater flexibility in her summer study schedule, she instead finds the whole experience somewhat frustrating.

Integrating Technology and the Academy

CC-BY by Wolfgang Greller of Flickr

CC-BY by Wolfgang Greller of Flickr

Eight important considerations about online learning to consider from the student perspective…

1) Students need to understand the online course requires at least as much time as the face-to-face alternative. Many online courses require logging into a course site and completing assignments and discussion postings on a regular basis – as much as several times per week.

2) Familiarity with required technology is essential. New-to-online or to a given learning management system (e.g. Blackboard, Moodle, Canvas) will require some orientation. Students and instructors both need to be comfortable with the technology in order that they might focus on the learning.

3) Simply migrating course content and quizzes to the learning management system does not an online course make. In order for deeper learning to be effected students need to actively engage with the content and with one another.

4) Assessments must align with both the content and the learning activities. It can be frustrating when questions on an assessment are not related to the material. Try awarding an extra point to the student who finds an error in either the content or the assessment – and then of course, correct the error as soon as possible.

5) The level of faculty-to-student interaction is arguably the most important factor impacting student retention and academic achievement. Frequent and timely feedback, instructor availability via email / chat / texting, and some type of faculty presence, permit students to better connect with the course through their instructor.

6) Student-to-student interaction creates for the learner, a sense of community – not only for the course at-hand, but to the institution as a whole, and can positively impact both retention and persistence.

7) Support services including online tutoring, technical assistance, self-help tutorials, etc. become all the more critical when the learner is disconnected from the classroom by both time and place.

8) Lastly, Ms. Shriver points out that she prefers “learning in the traditional classroom setting”. Keep in mind that flexibility is the primary reason students enroll in online courses.  Despite the obstacles she fully expects to take more online courses in the future.

Those of us who are responsible for delivering online learning need to be cognizant of the challenges students face and look at what we can do to ensure the learning experience is of the highest quality possible.

9.5 mile marker post along highway

Celebrating milestones

Mile Marker CC-BY-NC-SA by Michael McCullough on Flickr

Mile Marker CC-BY-NC-SA by Michael McCullough on Flickr

According to a recent National Student Clearinghouse report, one in five students completes their college degree at a different institution than the one where they began. For many students the path to a college degree may be a long and circuitous route.

When deciding to run a marathon, we don’t get up one day and say “I think I’ll try to get in 26 miles this afternoon”. We set incremental goals – milestones – and then by building upon these smaller successes we eventually reach our objective. We start out working up to a mile, then maybe a 5 K, then a 10 K, at some point we go for the half marathon, and eventually we reach our goal.

I have on several occasions, heard students, parents and faculty say they aren’t interested in getting their associates degree – or a certificate in their program. Their focus is only on getting the bachelor’s degree. Unfortunately, according the the National Student Clearinghouse report, only about 54 percent of those starting out actually achieve their goal.

I wonder if this is a cultural norm. How is it we do not value the opportunities for smaller successes along the way? And if indeed, this is the case, how might we begin to think differently and start celebrating the milestones and thereby encourage completion?

Some thoughts on how learning technology might support such a shift…

Use online discussion forums, blogs, and rosters at the start of the course, asking students share something about themselves to the class. Instructors can model by providing a simple introduction: my pets, favorite sports, hobby, etc. This can be helpful in creating a sense of community, especially in the online course where students may feel isolated without the advantages of face-to-face interactions available with the classroom environment.

Social networks  like Google+ Communities can offer a means of connecting with other students moving along similar pathways. Google+ Communities may be particularly well suited for establishing strong cohorts across programs and disciplines. Our campus already uses Gmail as its student email solution, thereby providing every student with an account making joining the network practically transparent.

Second year students might serve as moderators or hosts in the community for newly admitted students. Such communities may also include program alumni who are employed in their field or continuing their program of study elsewhere at a four-year institution or graduate program.

To take the marathon analogy a step further (at the risk of overdoing it) the long run is achieved not only with the help of those running along with us in the event, but also with the help of others cheering us on along the way. Hopefully we can help to make the finish line seem all the more within their grasp.

Student learning archery with feedback from intructor

Blended Workshop Presentation: Feedback and Assessment

Sharing the slide presentation from a recent Blended Learning Workshop series along with the source material…

Reference materials

Feedback works both ways – instructor to students and students to instructor. Feedback and formative assessment are powerful tools for learner engagement regardless of the delivery format. Technology provides a myriad of opportunities to connect learners with the content, the instructor, and their classmates.