What can we learn from the unsuccessful online student?

We have quite a bit of information on what it takes to be a successful online student but we may also be able to learn a few things from a couple of studies focusing on the unsuccessful online student.

24566052442_97a919e823_z
Great Expectations by Greg Myers on Flickr CC-BY-NC-SA

Monroe Community College in Rochester, NY, surveyed 438 unsuccessful online students over a ten year span about their online learning experience. “Unsuccessful students” were defined as those who either failed or withdrew from the online course (Fetzner, 2013).

Online course retention rates are on average, between 5 – 10% lower than for on-campus courses. Findings from the study show the group of students least likely to complete include the “first-time, full-time” students, with a difference in the success rate of 32%. The top three reasons given by the unsuccessful students for dropping or failing their courses, were “falling behind in coursework”, “personal problems”, and conflicts with work and/or family responsibilities (Fetzner, 2013).

In another study examining student engagement and online persistence, researchers from from the University of Georgia collected data on 423 students enrolled in thirteen online course sections over three semesters. The withdrawal rate for the asynchronous online courses was very high at 32%. Of those students who stayed enrolled in the courses, only 75% successfully completed the course (Morrie, Finnegan, Wu 2005).

Findings from the UG study (2005) indicated that unsuccessful completers (those finishing the course earning a D or F grade) were much less likely to participate or engage meaningfully in course discussions / postings, etc.

It is apparent from both studies that students may have very unrealistic expectations of what it takes to succeed going into the online learning environment. I have heard anecdotally on numerous occasions of students who have enrolled in an online course for the first time because they thought it would be “easier”.

Our campus is developing an “Introduction to Distance Learning” module in our LMS for students enrolling in an online or blended course for the first time at our college. At the moment they enroll they will be automatically enrolled in the module. An email message will alert them that they need to complete the module before their online course begins. The purpose of the module is to help students better understand what they can expect in taking an online course in regards to organizational skills, time-on-task, the amount of reading and writing required, as well as their access to, and comfort with, technology, etc.

I would be interested to hear of what other colleges have done to help mitigate unrealistic expectations of the first-time online student.

References:

Fetzner, M. (2013). What do unsuccessful online students want us to know? Journal of Asynchronous Learning Networks, Vol 17, No. 1

Morris, L., Finnegan, C., Wu, S. (2005). Tracking student behavior, persistence, and achievement in online courses. Internet and Higher Education Vol 8. 221-231.

 

Advertisements

The value of Student Evaluations of Teaching

A few years back a small group of faculty at our college were charged with redesigning a student survey of instruction for our online and hybrid courses. The resulting evaluation was a significant departure from the previous version with much more focus on student effort and expectations.

Teacher Evaluation Form CC-BY-NC-SA by Kevin Lim on Flickr
Teacher Evaluation Form CC-BY-NC-SA by Kevin Lim on Flickr

Instead of asking students questions along the lines of “Was you professor prepared for class?” or “How knowledgeable was your instructor about the course material?” – questions designed to evaluate instructor performance – the new and improved version took into consideration student investment: What grade do you expect to earn? How much effort did you put into the assignments? How much time did you set aside for course study each week? These kinds of questions help students to consider their own expectations and whether they align with their actual effort.

When asked about the quality of a thing (whether a product, service, or experience), we are actually being asked whether it met our expectations. These expectations may not be very realistic or accurate depending on past experience. When I talk with a group of faculty about innovating instruction in their courses, I suggest they tell students up front that they are trying something new and ask for their cooperation. I also suggest they warn their deans and department chairs that student evaluations may be negatively effected – students don’t always appreciate having to work at their own learning.

Unfortunately, student satisfaction surveys are all to often used as part of the faculty evaluation process. This is not only unfortunate but unfair as there is evidence that such assessments are not very useful or accurate for this purpose. A recent Inside Higher Ed article “Bias Against Female Instructors” reported on a study that showed male instructors tended to be rated higher than females regardless of the gender of the student.

In my opinion student evaluations can be useful for helping instructors, departments and programs to improve course design and delivery but they need to focus not on student satisfaction but on instructional feedback. And, they need to be performed not at the end of the semester but much earlier in the process. By asking students to provide feedback on course delivery and design earlier in the semester, faculty have the opportunity to improve instruction and at the same time, get a sense of how their students are doing in regards to learning the material.

Approaching the finishing line… by Sumeet Mulani Via Flickr:

Can OER improve time-to-degree?

I recently overheard a student complain about being required to purchase a new textbook for their business course. The book cost about one hundred dollars new but they were hoping to save money by purchasing used. Unfortunately, used wasn’t an option as the text included an access code for ancillary publisher materials made available online. Since the access codes are non-transferable, only new texts are made available in the bookstore for students enrolling in the class. Another student said that they were hoping to get by without the text as it was “too expensive”.  Although I don’t know how frequently this happens, it’s not the first time I’ve heard of students trying to manage without the required textbooks for their college courses.

In a recent experimental study published in the Journal of Computing in Higher Education, researchers found that students using open educational resources (OERs), including open textbooks, performed as well or better, in regards to completion rates and final grades, than students using commercial textbooks (Fisher et al 2015).

The study includes a sample size of 16,727 students from four universities and six community colleges with 4909 students in the “treatment condition” using OERs for their courses, and 11,818 in the control group using commercial textbooks.

The study compared student course completion, passing grade, and number of credits students took during a semester. Results indicated student completions were approximately the same for both the control and treatment groups with the exception of a couple of courses where attrition was somewhat higher for students in the control group. Grades were also mostly similar for the students in both groups with the majority of courses, although a few courses indicated students using the commercial text scored somewhat higher.

The most significant difference between the treatment and control groups was in the credit load – students in the treatment condition averaged 13.29 credit hours, while the control group averaged 11.14 hours. It may be that the savings students experienced using the OERs over the commercial text permitted more resources to be used for tuition.

It would be interesting to know some of the details in cases where students using OERs outperformed those using commercial texts. Perhaps these are the students who would otherwise have tried to get by without a “too expensive” textbook and later in the semester decided to drop or fail. Regardless of the reason, students using OERs stayed the course, and by enrolling in more courses may indeed cross the finish line that much earlier.

Reference

Fischer, L., Hilton III, J., Robinson, T.J.,Wiley, D.A. (2015). A multi-institutional study of the impact of open textbook adoption on the learning outcomes of post secondary students. Journal of Computing in Higher Education, Springer. Retrieved from http://link.springer.com/article/10.1007%2Fs12528-015-9101-x

Airman learning online

Two Competency-based Education (CBE) models

Last Friday, I and three faculty members from our campus attended a one-day conference at Columbus State Community College on the topic of Competency-based Education for Ohio.

Dr. Sally Johnstone, V.P. of Academic Advancement described the CBE model used at Western Governors University. WGU is an online non-profit university where students can enroll at any time of the year progress at their own pace. The role of the faculty at WGU are somewhat different than for traditional instructional models. Student Mentors are faculty who work closely with students individually to engage them in their program of study. Student mentors must have a minimum of a Masters degree in the content area. Course Mentors have a minimum of a PhD or terminal degree in the subject area and are responsible for the curriculum and course design. In addition to student and course mentors, Graders or Evaluators oversee student assessment. Students are assessed both using objective proctored exams and performance assessment.

Dr. Johnstone emphasized the importance of secure and reliable assessment – “The integrity of the institution rests on the credibility of the assessments.”

Although students move at their own pace through the course, they still have the opportunity to interact with their peers through the campus portal social media technology.

Course content is provided through KSVs (Kahn Style Videos) much of which is sourced through Carnegie Mellon’s Acrobatiq – Open Learning Initiative.

Dr. Nancy Thibeault, Dean of eLearning at Sinclair Community College, shared about Sinclair’s CBE model and some of the challenges of rolling out CBE at a public two-year institution.

Community colleges in Ohio operate on a semester schedule. Although the semesters are typically sixteen weeks in length, Sinclair permits students to enroll in their CBE courses at any time up until the last four weeks of the semester. Before students are accepted into the CBE program they must meet certain guidelines including a minimum GPA of 3.0 and demonstrated computer literacy skills.

Students may work at their own pace but are expected to keep up with course milestones. An academic coach may work with as many as eighty students individually. Each course unit includes a pre-assessment to determine whether students possess the competencies in a given unit of study – if so, they may skip to the next unit and thereby accelerate their course of study. Coaches assume the role of case-manager, guiding students from point of registration through graduation.

As with WGU, Sinclair awards course credit on a pass / fail basis. Students must successfully pass with an eighty percent or higher in order to continue in the CBE program. Each course includes a comprehensive final exam covering all course competencies.

Pittsburgh Bridges by Don O'Brien CC-BY on Flickr

Navigating the online course

After several trips to Pittsburgh in the past few years, I have learned to turn off my GPS navigation system while driving through the downtown area. With its tall buildings, numerous highway exits, over-passes and bridges, my GPS loses the signal and begins offering less than helpful suggestions at a time when the importance of a clear useable navigation system is most critical.

Pittsburgh Bridges by Don O'Brien CC-BY on Flickr
With its tall buildings, numerous highway exits, over-passes and bridges, my GPS loses the signal and begins offering less than helpful suggestions at a time when the importance of a clear useable navigation system is most critical. Pittsburgh Bridges by Don O’Brien CC-BY on Flickr –

A colleague shared with me an incident with a student requesting a refund when, after week four of an eight week math course, they had yet to log into the publisher’s course site. According to the student they had logged into their Blackboard course and perused the course space but were unaware of the need to log into the supplemental publisher’s site, where the quizzes were to be completed. Of course, by week four, they were already halfway though the course and had missed several quizzes. When the instructor suggested they drop the course, the student stated they were not made aware of the second site and therefore should not have to pay for the course.

Unfortunately, this isn’t the first time I’ve heard these kinds of concerns and it seems to me that this is yet another example of how important it is to make clear the expectations early on in the semester or term. One way to help avoid these problems is to consider the course interface from the student’s perspective.

In a Quality Matters research project, Does Findability Matter? – Findability, Student Motivation and Self-efficacy in Online Courses (2012), researchers found that time-on-task, student self-reported difficulty and frustration levels were significantly higher for students enrolled in control group (non-QM-recognized) courses. On the other hand, students enrolled in the experimental group (QM recognized courses) reported a significantly better experience in regards to ease of finding course materials and assignments. The were more likely to enjoy their course experience and to recommend the course to friends. Instructor ratings were higher and frustration levels, lower than for the control group.

“Findability” was rated using QM Standard 6.3 – Navigation throughout the online components of the course is logical, consistent, and efficient.

When preparing the course ask a colleague and / or student(s) who are unfamiliar with the course, to look over the homepage and provide helpful suggestions. Here are some questions for which they might provide feedback…

  • Is it obvious where to begin?
  • Can you easily find the course syllabus?
  • Is it clear how to contact the instructor, when they are available and when students can expect a response?
  • Are the course goals and objectives clearly stated?
  • Is the schedule of assignments and course activities easy to find and understand?
  • Can students easily navigate to ancillary materials (e.g. publishers quizzes)?
  • Can you see at first glance how to find both technical and course-related assistance?

Keep in mind that people are different and will intuit where to find things by their own experience with similar interfaces. Although you may feel that putting all of the important information in one place (such as the syllabus) should suffice, your students may benefit by finding the same information in multiple locations – or rather – multiple paths to the same information.

These considerations may seem obvious but getting a second or even third perspective can help to ensure students can easily navigate the course and focus on the learning, as opposed to getting lost along the way.

References:

Simunich, B., Robins, D., Kelly, V. (2012) Does findability matter?. Quality Matters.org https://www.qualitymatters.org/files/webform/Quality%20Matters%202012%20Findability.pdf

CC-BY-NC-SA by Luke Chan on Flickr

From Community College to College community

When students drop courses at our community college, we ask them to complete a course drop survey form explaining their reasons for dropping.  The majority state for “personal reasons” the second most frequent response is for “other” reasons. Apparently we are not unique. Neal Raisman recently shared on his blog – “Great Service Matters” – the results of a study of why students leave college. Many of these students at the point of departure state they leave for “personal reasons”.

CC-BY-NC-SA by Luke Chan on Flickr
CC-BY-NC-SA by Luke Chan on Flickr

In the study, 864 students were interviewed after at least six months following their leaving college. The break in time was intended to give students an opportunity to be more reflective and open about their reasons for leaving.

Twenty-six percent of students reported they left because the “college doesn’t care”. Another twenty-four reported “poor service” as their main reason for leaving – which could be interpreted as another way of saying that the college doesn’t care. Together, these two responses account for fifty percent of students leaving school.

That’s huge.

In a previous post I shared the results of our own student survey of online learners asking what we could do to help them be more successful. It was clear from these student responses that they valued courses where the instructor was available and accessible – in other words – cared about them and how they were doing in the course.

What does it mean for a college to care?

Raisman talks about the importance of customer service. Certainly responsiveness is an important component to quality customer service, including something as obvious as having people available to answer calls, emails, texts, etc. in a timely manner. Listening to what students have to say, and then putting ourselves in their shoes is key to responsive and caring customer service.

Inclusiveness is another essential consideration. Many students at the community college are first generation college students. Their parents, friends and family may not be in a position to advise them about college life, expectations, and what it takes to succeed. Keeping this in mind, we need to consider ways of connecting students to the campus. For on-campus students this may mean student organizations or study groups. For online students we need to consider virtual connections that may leverage social media, or collaborative class projects, virtual office hours, etc. With more than 80% adoption of mobile technology by college students, there has never been a better time to leverage social media as a means of connecting all students to the college community.

Student support is more than a responsive friendly help desk or online tutoring – it includes a creating a sense of community for both the on-campus and the online student. If we are to succeed in retaining our distance learners,  the community college must become more of a college community.

Antique Video Camera

Students ask for more video with their learning

A few years back we invested in the streaming video service – Films on Demand. The service integrates with our campus portal and the Lakeland Library to permit faculty the means of integrating educational video into their course materials. Compared with the former system of ordering individual media (VHS, DVD), this has proved to be a very good investment. Nevertheless, the level of adoption has been less than expected.

Fims on Demand - Collections
Films on Demand – Collections

In our recent survey of online learners, students reported that they would like to have more video content in their online courses. Streaming video is becoming an extremely popular way for students to view content as is evidenced by the rate of adoption of such services as iTunes U and Khan Academy.

According to a recent article in Inside Higher Ed reporting on thePearson Higher Ed Survey on Student Mobile usage

“Eight in ten (83%) college students regularly use a smartphone, up significantly from 72% in 2013.  Smartphones are now close to laptops (89%) as the mobile device students are most likely to use on a regular basis.”

Perhaps the increased use of smartphones by students helps to explain the demand for more video content. Although I enjoy reading journal articles and other text-based material on my iPad because of the size and orientation, I am not a big fan of reading text on my iPhone. That being said, the iPhone works great with streaming video – especially with WiFi available both at work and home.

iPhone CC-BY-NC-SA by Alex Bartok on Flickr
CC-BY-NC-SA by Alex Bartok on Flickr

Films on Demand is available to both faculty and students and although we have promoted its use primarily with the faculty – demonstrating how to search for relevant content from among the more than 15,000 titles and 200,000 plus segments – students also may access these materials and find value in searching through the collections themselves.

Gamification and Student-to-content Interactivity

Learner engagement is considered to be an effective predictor of student success. We can increase learner engagement by focusing on interactivity in course design.

Dice CC-BY-NC-SA by Daniela Hartmann on Flickr
CC-BY-NC-SA by Daniela Hartmann on Flickr

The Community College Research Center (CCRC) at Teachers College, Columbia University published a paper on the topic: Predicting Online Student Outcomes from a Measure of Course Quality (Jaggars & Xu.) The purpose of the study was to gather empirical evidence regarding the link between online course quality and student outcomes.

The study looked at four main areas:

  • Course design: organization and navigation
  • Learning objectives and assessment
  • Interpersonal interactions (student-to-instructor & student-to-student)
  • Effective use of technology

Much of what was learned through the research reaffirms what we have shared here before – that there exists a positive correlation between student-to-instructor interactivity and student success.

What the study did not reveal was a correlation between student success and online course design, or for that matter, between student success and the alignment of learning objectives and assessment. That isn’t to say these aren’t essential considerations when designing the online course – they are. But rather, that the study did not provide evidence that these factors are directly related to student achievement.

Another interesting finding was that although student-to-instructor interactions showed a positive correlation to student outcomes, this was not necessarily the case with student-to-student interactions. In fact, students indicated their experience with online discussion and group projects was, to paraphrase, pretty much a time sink.

Initially, the findings appeared to support a relationship between the effective use of technology and course grades, but after controlling for student characteristics the relationship became less apparent. Most quality assurance rubrics and accepted practices in online delivery suggest that educational technologies need to be current or state-of-the-art. However, this study suggests when designing rubrics for online course design…

“…quality ratings for technology may wish to focus on not just the use of “current” technologies but how these technologies are used to support user interaction, confidence, motivation, and learning.”

It occurs to me that interaction, confidence, motivation and learning can all be supported by the introduction of games or gamification in learning design.

The use of games in education has gained momentum in recent years. Games can be very motivating (perhaps even addicting). In a game we interact with the medium, often are given problems to solve or challenges to overcome, make choices, and as we progress in skill we become more confident.

Tools like Respondus StudyMate and Quia can be used to turn objective type quizzes into word games: (matching, hangman, crossword, fill in the blank). This is especially helpful when learning new terminology. After integrating games into her Medical Terminology course a couple of years ago an instructor reported significant improvement in student test scores as students began to spend hours reviewing the content as they played games and tried to improve their scores.

The learning is not necessarily the learner’s primary goal when playing a game, but rather accomplishing a task, mastering a certain level, gaining points or credits, and ultimately to win. Of course learning happens as an outcome in the process. Games are by definition, competitive – we may compete against the computer, chance, ourselves, or others (bringing us back to the earlier discussion about student-to-student interactions).

According to Karl Kapp, “Gamification is using game-based mechanics, aesthetics, and game thinking to engage people, motivate action, promote learning, and solve problems.”

“Through the careful application of game elements—such as the freedom to fail, interest curve, storytelling, and feedback—in learning programs, ordinary content can be made more engaging without the development of a full-fledged learning game.” – Karl Kapp

By looking for ways to implement game elements into our courses we may be able to repond to the several questions brought up in the study, including student to student interactions, and effective use of technology (student to content interactions).

For more information about gamification, check out Karl’s article Getting Started with Gamification on ASTD.org.

What students love about online learning…

Students say they love their online courses when their instructor is accessible and responsive, when their instructor is hard to reach or unresponsive – not so much.

CC-BY-NC-SA by John Harvey on Flickr
CC-BY-NC-SA by John Harvey on Flickr

We recently surveyed our online students regarding distance learning support services. The last question in the survey asked for “any other comments” they wished to offer. About two-thirds of the responses were very positive with students telling us how much they appreciated the online options. Here’s the gist…

“Although the instructor had organized the course well on Blackboard, I could never get a prompt response…”

“My instructor was awesome and was always there if you had any questions.”

“…every course I have taken has been great and the professors have all been attentive and responsive”.

“I really enjoyed my online course with [my] professor… he was responsive to my individual questions and he always replied in a timely compassionate manner.”

“I enjoyed my online courses. The professors were always available for help, though at times I had to wait a day or so for them to email me back.”

“I enjoy my online classes. My psychology class this semester has been great. My economics teacher on the other hand, I feel is very distant and not very helpful when I try and reach out to him.”

Note the theme here – “I loved the online course – my instructor was responsive”.

It would appear that instructor responsiveness and availability is key to student satisfaction in the online learning experience. Some thoughts on how to accomplish this in your online courses…

Students need to know the instructor’s preferred communication style.

Which tools or methods will you employ – email, instant messaging, texts, voice calls, Twitter, Skype, or a combination thereof? Posting this information in the syllabus and course introductory pages can help to manage expectations. Students will know that although you are not available twenty-four-seven, you can still be reached and will be getting back with them in a reasonable amount of time. Tell the students you will respond in a timely manner so they know when to expect a response, and be specific. If your intention is to answer inquiries within 24 hours, state this on the course homepage along with your contact information. I knew a teacher, new to online, who told her students that she would be checking her emails on Thursday evenings. Yeah… as you might imagine, that was not very well received.

Online instructors should not feel the need to respond to every text as it is received, but they do need to establish some sort of routine. If you check your email first thing in the morning or before you go to bed at night, students will begin to expect your responses around these times. If the schedule changes – you’re on vacation, or working on a project that takes you away from your normal rhythm – send out a message or announcement that they might not hear back from you until the next day.

Virtual office hours are useful even when students don’t take advantage of them. They know that Tuesdays and Thursdays they can log into the chat or find you on Skype between 2:30 and 4:00 in the afternoon. I know more than a few faculty who regularly schedule virtual conferences with their students using Google Hangouts or Skype, just to add a more personal connection as they review their students’ writing assignments.

Have the students introduce themselves to the rest of the class at the beginning of the semester. This can be very helpful in creating a sense of community in the virtual classroom environment. You can model this by posting your own introduction to a discussion forum as the first assignment. Using the Blackboard video feature or simply sharing a short video from your phone can help students to see you as a real person so they can put a face along with the name of their professor.

For more ideas about improving communication and interaction, check out the Communications & Interactions Plan found at University of Utah’s Center for Teaching & Learning Excellence: https://utah.instructure.com/courses/148446/pages/communication-and-interaction-plan-strategies

CC-BY-NC-SA by Ed Yourdon on Flickr

What can we do to help students be more successful in online courses?

We surveyed our online students this spring and received a strong response to the open-ended question, “What could [the college] do to help you be more successful in online course(s)?

CC-BY-NC-SA by Ed Yourdon on Flickr
CC-BY-NC-SA by Ed Yourdon on Flickr

I tried to categorize the students’ responses around themes – here are the top ten…

  1. Reliable Technology – specifically the learning management system (LMS). Students expect the technology to be reliable and to work as designed when they need it. They do not expect to be logged out, or timed out, or to find the system off-line due to a power outage, etc.
  2. Video – students want their courses to include short videos:  lectures, explanations, examples, demonstrations… “like Khan Academy”.
  3. More Online Courses – students are enrolling in online courses because it meets their schedules and they need more online offerings if they are to complete their programs
  4. Reminders – they want to get alerts, reminders, notifications about what is due and when it is due.
  5. Consistency – students would like for their online courses to have the same look and feel. The layout of the courses, tabs, menus should be the same from one course space to the next.
  6. Instructor Availability – students want to be able to contact their instructor when they have a question or need help and expect to get a response in a timely manner.
  7. Timely Feedback – students are looking for their instructors to keep them apprised of their progress. They would like to get their grades early and often.
  8. Faculty Involvement – students appreciate faculty taking an active role in teaching the course – not so much a third-party website or publisher’s course pack.
  9. Online Testing – they want to be able to take more tests online as opposed to coming to the testing center. They point out that they enrolled in the online class so that they would not need to travel to campus.
  10. Calendar – students would like to know what is coming up ahead of time and for all their courses. A composite calendar of events for all of their courses is their suggested solution.

It is interesting to me that through this survey, students had an opportunity to recommend new and innovative technological solutions, yet they focused much more so on issues of design and delivery – on improving existing processes.

The good news is, we can do a lot of this stuff!