Keeping score at relay race for Melrose Running Club

Measuring Student Success

For many years, community colleges have focused on providing access to everyone seeking a post-secondary education. More recently the narrative has changed to that of completion and how long it takes the student to complete. The literature tells us the longer a student is in college, the less likely she is to complete. The completion rates for first-time full-time degree seeking students are low – especially for community college students. Generally speaking, when we measure student success, what we are talking about is completion rates.

The fact is many – if not most – of our students are not first-time, full-time degree seeking students. Many have already completed a college education, some having attained bachelors and masters degrees, before coming back to start over in a new career. More and more of our student population are enrolling part-time. Others not seeking a degree at all, but instead, looking to update their job skills by taking a course or two. How do we measure student success for these learners?

California Community Colleges studied these non-completers over a two-year period. Referred to as Skills Builders, these students enrolled in a limited number of courses for the express purpose of enhancing their job skills or moving up the ladder within their careers. The study found that the average Skills Builder improved their salary by 13.6% – averaging $4,300 / year! Clearly these students were successful in meeting their educational goals.

EDUCAUSE Review recently published a collection of short essays entitled, Student Success: Mission Critical.  In the introduction, John O’Brien says, “If students don’t succeed, colleges and universities don’t succeed. Our full attention must be concentrated on the mission-critical goal of helping students define – and meet- their educational goals.”

To my way of thinking, this is what we should be talking about when we talk about student success: “helping students define – and meet – their educational goals“.

Regardless of whether students come to college to take a few courses to improve their employability, complete a program of study, or transfer to university, achieving their educational goals requires persistence. We often hear the terms “persistence and “retention” used synonymously. The National Center for Education Statistics (NCES), however, differentiates between the terms: “retention” as an institutional measure and “persistence” as a student measure. In other words, institutions retain and students persist.

This is an important distinction when it comes to measuring student success. As an institution, we can measure retention: did the student drop / complete? Has the student continued their program of study? However, whether the learner persists or not, is really up to the student. For this reason, we can, and should, create a learning environment that encourages student persistence.

“Tinto posits that students are more likely to remain enrolled in an institution if they become connected to the social and academic life of that institution” – Community College Research Center (CCRC).

The CCRC study found that community college students who make academic connections… “create a sense of belonging and attachment that seems to encourage persistence” – CCRC.

In Student Success: Mission Critical, George Mchaffey reminds us to not “avoid the academic heart of the enterprise… But the core of the enterprise is the curriculum and particularly the classroom. Some people avoid tackling that area because it is likely the most difficult. However, substantive change in student success outcomes must include attention to what happens to students in classrooms and in their academic journeys.”

We can measure student success in different ways: retention is measured by the quantity of students who continue and complete, but persistence in measured by the quality of the student’s experience – whether they belong here and how much we care.

References:

http://www.aacc.nche.edu/AboutCC/Trends/Pages/completion_report.aspx

https://www.insidehighered.com/news/2016/03/04/california-community-colleges-find-new-way-measure-success-noncompleters

http://er.educause.edu/articles/2017/5/student-success-mission-critical

http://ccrc.tc.columbia.edu/media/k2/attachments/exploration-tintos-integration-framework.pdf

 

Advertisements

Using LMS Data to Improve Self-regulated Learning

A recent study examined learning management system (LMS) log files to look at course interactions for 530 online students and found that the students’ self-regulated study habits significantly influenced course achievement. The study focused on those interactions related to self-regulated learning, including such habits  as maintaining a regular study schedule, timely submission of assignments, frequency of course logins, and proof of reading course content (You, 2016).

It may seem obvious that students who possess good study habits are more likely to succeed in the online learning environment (or any learning environment for that matter). However, for me, the important take-away from the study is the potential for leveraging the data collection systems and early alert functionality within our LMS toward reinforcing self-regulating learning habits with those students who may be at-risk of dropping or failing their online course.

The Retention Center is an early alert feature included in our campus LMS – Blackboard Learn. The Retention Center comes set up with four default rules: Course Access, Activity, Grades, and Missed Deadlines. Although these “rules” align fairly well with the study habits mentioned in the study, we are able to customize, as well as to design new rules that can be added to the Risk Table to further support and reinforce study habits.

Screen Shot - Blackboard Retention Center - Rules Customization
Blackboard Retention Center – Rules Customization

The course access and activity rules align with the frequency of course logins habit. The default rule is five days since last access, but in light of the research, I suggest shortening the number to two or three days. The User Activity default rule is set to twenty percent below average for a week – again, I would suggest changing it to three or four days.  The timely submission of assignments and proof of reading course content will depend on setting up corresponding columns in the grade book. By developed assignments that require students to review material (possibly video content) and then to complete a related assessment, grade alerts can be triggered in the corresponding grade book column. By tweaking the default rules or adding new rulee, the instructor can quickly identify students who display poor study habits and immediately reach out to reinforce good habits that support student success.

In addition to the Retention Center, the Performance Dashboard tool can be used to view the content items the student has accessed along with the number and length of posts a student has submitted to course discussion forums. By requiring students to review content and submit substantive posts within a discussion forum, the instructor can encourage the reading of course content – another study habit predictive of student achievement. Encouraging students to subscribe to the discussion forums can further support regular and substantive interaction with classmates.

In a survey of unsuccessful online students at Monroe Community College, students reported the number one challenge they experienced in their online courses… “I got behind and it was too hard to catch up” (Fetzner, 2013). By designing online courses that leverage the LMS analytics features to identify and support at-risk students within the first few weeks, new-to-online students can develop the skills and habits required to be successful in the online environment.

References:

You, J. W. (2016). Identifying significant indicators using LMS data to predict course achievement in online learning. The Internet and Higher Education, 29, 23-30.

Fetzner, M. (2013). What Do Unsuccessful Online Students Want Us to Know?. Journal of Asynchronous Learning Networks, 17(1), 13-27.

 

 

 

 

Virtual Office Hours

In their paper, “Using Virtual Office Hours to Enhance Student-Faculty Interaction”, Lei Li and Jennifer Pitts (2009) from Colorado State University found that students enrolled in courses that offered virtual office hours experienced a higher level of satisfaction than those students enrolled in traditional on-campus courses. Apparently, just by making the virtual office hours an option for students, regardless of whether they availed themselves of the opportunity, was enough to increase the students’ comfort level with the instructor and the online course.

11513097184_11306a8c98_z
Skype & Coffee by I. Keller of Flickr, CC-BY-NC-SA

Ll, L., Pitts, J. (2009). Does it really matter? Using virtual office hours to enhance student-faculty interaction. Journal of Information Systems Education, Vol 20. No. 2. http://jise.org/Volume20/20-2/Pdf/V20N2P175-abs.pdf

 

What can we learn from the unsuccessful online student?

We have quite a bit of information on what it takes to be a successful online student but we may also be able to learn a few things from a couple of studies focusing on the unsuccessful online student.

24566052442_97a919e823_z
Great Expectations by Greg Myers on Flickr CC-BY-NC-SA

Monroe Community College in Rochester, NY, surveyed 438 unsuccessful online students over a ten year span about their online learning experience. “Unsuccessful students” were defined as those who either failed or withdrew from the online course (Fetzner, 2013).

Online course retention rates are on average, between 5 – 10% lower than for on-campus courses. Findings from the study show the group of students least likely to complete include the “first-time, full-time” students, with a difference in the success rate of 32%. The top three reasons given by the unsuccessful students for dropping or failing their courses, were “falling behind in coursework”, “personal problems”, and conflicts with work and/or family responsibilities (Fetzner, 2013).

In another study examining student engagement and online persistence, researchers from from the University of Georgia collected data on 423 students enrolled in thirteen online course sections over three semesters. The withdrawal rate for the asynchronous online courses was very high at 32%. Of those students who stayed enrolled in the courses, only 75% successfully completed the course (Morrie, Finnegan, Wu 2005).

Findings from the UG study (2005) indicated that unsuccessful completers (those finishing the course earning a D or F grade) were much less likely to participate or engage meaningfully in course discussions / postings, etc.

It is apparent from both studies that students may have very unrealistic expectations of what it takes to succeed going into the online learning environment. I have heard anecdotally on numerous occasions of students who have enrolled in an online course for the first time because they thought it would be “easier”.

Our campus is developing an “Introduction to Distance Learning” module in our LMS for students enrolling in an online or blended course for the first time at our college. At the moment they enroll they will be automatically enrolled in the module. An email message will alert them that they need to complete the module before their online course begins. The purpose of the module is to help students better understand what they can expect in taking an online course in regards to organizational skills, time-on-task, the amount of reading and writing required, as well as their access to, and comfort with, technology, etc.

I would be interested to hear of what other colleges have done to help mitigate unrealistic expectations of the first-time online student.

References:

Fetzner, M. (2013). What do unsuccessful online students want us to know? Journal of Asynchronous Learning Networks, Vol 17, No. 1

Morris, L., Finnegan, C., Wu, S. (2005). Tracking student behavior, persistence, and achievement in online courses. Internet and Higher Education Vol 8. 221-231.

 

Approaching the finishing line… by Sumeet Mulani Via Flickr:

Can OER improve time-to-degree?

I recently overheard a student complain about being required to purchase a new textbook for their business course. The book cost about one hundred dollars new but they were hoping to save money by purchasing used. Unfortunately, used wasn’t an option as the text included an access code for ancillary publisher materials made available online. Since the access codes are non-transferable, only new texts are made available in the bookstore for students enrolling in the class. Another student said that they were hoping to get by without the text as it was “too expensive”.  Although I don’t know how frequently this happens, it’s not the first time I’ve heard of students trying to manage without the required textbooks for their college courses.

In a recent experimental study published in the Journal of Computing in Higher Education, researchers found that students using open educational resources (OERs), including open textbooks, performed as well or better, in regards to completion rates and final grades, than students using commercial textbooks (Fisher et al 2015).

The study includes a sample size of 16,727 students from four universities and six community colleges with 4909 students in the “treatment condition” using OERs for their courses, and 11,818 in the control group using commercial textbooks.

The study compared student course completion, passing grade, and number of credits students took during a semester. Results indicated student completions were approximately the same for both the control and treatment groups with the exception of a couple of courses where attrition was somewhat higher for students in the control group. Grades were also mostly similar for the students in both groups with the majority of courses, although a few courses indicated students using the commercial text scored somewhat higher.

The most significant difference between the treatment and control groups was in the credit load – students in the treatment condition averaged 13.29 credit hours, while the control group averaged 11.14 hours. It may be that the savings students experienced using the OERs over the commercial text permitted more resources to be used for tuition.

It would be interesting to know some of the details in cases where students using OERs outperformed those using commercial texts. Perhaps these are the students who would otherwise have tried to get by without a “too expensive” textbook and later in the semester decided to drop or fail. Regardless of the reason, students using OERs stayed the course, and by enrolling in more courses may indeed cross the finish line that much earlier.

Reference

Fischer, L., Hilton III, J., Robinson, T.J.,Wiley, D.A. (2015). A multi-institutional study of the impact of open textbook adoption on the learning outcomes of post secondary students. Journal of Computing in Higher Education, Springer. Retrieved from http://link.springer.com/article/10.1007%2Fs12528-015-9101-x

CC-BY-NC-SA by Luke Chan on Flickr

From Community College to College community

When students drop courses at our community college, we ask them to complete a course drop survey form explaining their reasons for dropping.  The majority state for “personal reasons” the second most frequent response is for “other” reasons. Apparently we are not unique. Neal Raisman recently shared on his blog – “Great Service Matters” – the results of a study of why students leave college. Many of these students at the point of departure state they leave for “personal reasons”.

CC-BY-NC-SA by Luke Chan on Flickr
CC-BY-NC-SA by Luke Chan on Flickr

In the study, 864 students were interviewed after at least six months following their leaving college. The break in time was intended to give students an opportunity to be more reflective and open about their reasons for leaving.

Twenty-six percent of students reported they left because the “college doesn’t care”. Another twenty-four reported “poor service” as their main reason for leaving – which could be interpreted as another way of saying that the college doesn’t care. Together, these two responses account for fifty percent of students leaving school.

That’s huge.

In a previous post I shared the results of our own student survey of online learners asking what we could do to help them be more successful. It was clear from these student responses that they valued courses where the instructor was available and accessible – in other words – cared about them and how they were doing in the course.

What does it mean for a college to care?

Raisman talks about the importance of customer service. Certainly responsiveness is an important component to quality customer service, including something as obvious as having people available to answer calls, emails, texts, etc. in a timely manner. Listening to what students have to say, and then putting ourselves in their shoes is key to responsive and caring customer service.

Inclusiveness is another essential consideration. Many students at the community college are first generation college students. Their parents, friends and family may not be in a position to advise them about college life, expectations, and what it takes to succeed. Keeping this in mind, we need to consider ways of connecting students to the campus. For on-campus students this may mean student organizations or study groups. For online students we need to consider virtual connections that may leverage social media, or collaborative class projects, virtual office hours, etc. With more than 80% adoption of mobile technology by college students, there has never been a better time to leverage social media as a means of connecting all students to the college community.

Student support is more than a responsive friendly help desk or online tutoring – it includes a creating a sense of community for both the on-campus and the online student. If we are to succeed in retaining our distance learners,  the community college must become more of a college community.

Gamification and Student-to-content Interactivity

Learner engagement is considered to be an effective predictor of student success. We can increase learner engagement by focusing on interactivity in course design.

Dice CC-BY-NC-SA by Daniela Hartmann on Flickr
CC-BY-NC-SA by Daniela Hartmann on Flickr

The Community College Research Center (CCRC) at Teachers College, Columbia University published a paper on the topic: Predicting Online Student Outcomes from a Measure of Course Quality (Jaggars & Xu.) The purpose of the study was to gather empirical evidence regarding the link between online course quality and student outcomes.

The study looked at four main areas:

  • Course design: organization and navigation
  • Learning objectives and assessment
  • Interpersonal interactions (student-to-instructor & student-to-student)
  • Effective use of technology

Much of what was learned through the research reaffirms what we have shared here before – that there exists a positive correlation between student-to-instructor interactivity and student success.

What the study did not reveal was a correlation between student success and online course design, or for that matter, between student success and the alignment of learning objectives and assessment. That isn’t to say these aren’t essential considerations when designing the online course – they are. But rather, that the study did not provide evidence that these factors are directly related to student achievement.

Another interesting finding was that although student-to-instructor interactions showed a positive correlation to student outcomes, this was not necessarily the case with student-to-student interactions. In fact, students indicated their experience with online discussion and group projects was, to paraphrase, pretty much a time sink.

Initially, the findings appeared to support a relationship between the effective use of technology and course grades, but after controlling for student characteristics the relationship became less apparent. Most quality assurance rubrics and accepted practices in online delivery suggest that educational technologies need to be current or state-of-the-art. However, this study suggests when designing rubrics for online course design…

“…quality ratings for technology may wish to focus on not just the use of “current” technologies but how these technologies are used to support user interaction, confidence, motivation, and learning.”

It occurs to me that interaction, confidence, motivation and learning can all be supported by the introduction of games or gamification in learning design.

The use of games in education has gained momentum in recent years. Games can be very motivating (perhaps even addicting). In a game we interact with the medium, often are given problems to solve or challenges to overcome, make choices, and as we progress in skill we become more confident.

Tools like Respondus StudyMate and Quia can be used to turn objective type quizzes into word games: (matching, hangman, crossword, fill in the blank). This is especially helpful when learning new terminology. After integrating games into her Medical Terminology course a couple of years ago an instructor reported significant improvement in student test scores as students began to spend hours reviewing the content as they played games and tried to improve their scores.

The learning is not necessarily the learner’s primary goal when playing a game, but rather accomplishing a task, mastering a certain level, gaining points or credits, and ultimately to win. Of course learning happens as an outcome in the process. Games are by definition, competitive – we may compete against the computer, chance, ourselves, or others (bringing us back to the earlier discussion about student-to-student interactions).

According to Karl Kapp, “Gamification is using game-based mechanics, aesthetics, and game thinking to engage people, motivate action, promote learning, and solve problems.”

“Through the careful application of game elements—such as the freedom to fail, interest curve, storytelling, and feedback—in learning programs, ordinary content can be made more engaging without the development of a full-fledged learning game.” – Karl Kapp

By looking for ways to implement game elements into our courses we may be able to repond to the several questions brought up in the study, including student to student interactions, and effective use of technology (student to content interactions).

For more information about gamification, check out Karl’s article Getting Started with Gamification on ASTD.org.

What students love about online learning…

Students say they love their online courses when their instructor is accessible and responsive, when their instructor is hard to reach or unresponsive – not so much.

CC-BY-NC-SA by John Harvey on Flickr
CC-BY-NC-SA by John Harvey on Flickr

We recently surveyed our online students regarding distance learning support services. The last question in the survey asked for “any other comments” they wished to offer. About two-thirds of the responses were very positive with students telling us how much they appreciated the online options. Here’s the gist…

“Although the instructor had organized the course well on Blackboard, I could never get a prompt response…”

“My instructor was awesome and was always there if you had any questions.”

“…every course I have taken has been great and the professors have all been attentive and responsive”.

“I really enjoyed my online course with [my] professor… he was responsive to my individual questions and he always replied in a timely compassionate manner.”

“I enjoyed my online courses. The professors were always available for help, though at times I had to wait a day or so for them to email me back.”

“I enjoy my online classes. My psychology class this semester has been great. My economics teacher on the other hand, I feel is very distant and not very helpful when I try and reach out to him.”

Note the theme here – “I loved the online course – my instructor was responsive”.

It would appear that instructor responsiveness and availability is key to student satisfaction in the online learning experience. Some thoughts on how to accomplish this in your online courses…

Students need to know the instructor’s preferred communication style.

Which tools or methods will you employ – email, instant messaging, texts, voice calls, Twitter, Skype, or a combination thereof? Posting this information in the syllabus and course introductory pages can help to manage expectations. Students will know that although you are not available twenty-four-seven, you can still be reached and will be getting back with them in a reasonable amount of time. Tell the students you will respond in a timely manner so they know when to expect a response, and be specific. If your intention is to answer inquiries within 24 hours, state this on the course homepage along with your contact information. I knew a teacher, new to online, who told her students that she would be checking her emails on Thursday evenings. Yeah… as you might imagine, that was not very well received.

Online instructors should not feel the need to respond to every text as it is received, but they do need to establish some sort of routine. If you check your email first thing in the morning or before you go to bed at night, students will begin to expect your responses around these times. If the schedule changes – you’re on vacation, or working on a project that takes you away from your normal rhythm – send out a message or announcement that they might not hear back from you until the next day.

Virtual office hours are useful even when students don’t take advantage of them. They know that Tuesdays and Thursdays they can log into the chat or find you on Skype between 2:30 and 4:00 in the afternoon. I know more than a few faculty who regularly schedule virtual conferences with their students using Google Hangouts or Skype, just to add a more personal connection as they review their students’ writing assignments.

Have the students introduce themselves to the rest of the class at the beginning of the semester. This can be very helpful in creating a sense of community in the virtual classroom environment. You can model this by posting your own introduction to a discussion forum as the first assignment. Using the Blackboard video feature or simply sharing a short video from your phone can help students to see you as a real person so they can put a face along with the name of their professor.

For more ideas about improving communication and interaction, check out the Communications & Interactions Plan found at University of Utah’s Center for Teaching & Learning Excellence: https://utah.instructure.com/courses/148446/pages/communication-and-interaction-plan-strategies

CC-BY-NC-SA by Ed Yourdon on Flickr

What can we do to help students be more successful in online courses?

We surveyed our online students this spring and received a strong response to the open-ended question, “What could [the college] do to help you be more successful in online course(s)?

CC-BY-NC-SA by Ed Yourdon on Flickr
CC-BY-NC-SA by Ed Yourdon on Flickr

I tried to categorize the students’ responses around themes – here are the top ten…

  1. Reliable Technology – specifically the learning management system (LMS). Students expect the technology to be reliable and to work as designed when they need it. They do not expect to be logged out, or timed out, or to find the system off-line due to a power outage, etc.
  2. Video – students want their courses to include short videos:  lectures, explanations, examples, demonstrations… “like Khan Academy”.
  3. More Online Courses – students are enrolling in online courses because it meets their schedules and they need more online offerings if they are to complete their programs
  4. Reminders – they want to get alerts, reminders, notifications about what is due and when it is due.
  5. Consistency – students would like for their online courses to have the same look and feel. The layout of the courses, tabs, menus should be the same from one course space to the next.
  6. Instructor Availability – students want to be able to contact their instructor when they have a question or need help and expect to get a response in a timely manner.
  7. Timely Feedback – students are looking for their instructors to keep them apprised of their progress. They would like to get their grades early and often.
  8. Faculty Involvement – students appreciate faculty taking an active role in teaching the course – not so much a third-party website or publisher’s course pack.
  9. Online Testing – they want to be able to take more tests online as opposed to coming to the testing center. They point out that they enrolled in the online class so that they would not need to travel to campus.
  10. Calendar – students would like to know what is coming up ahead of time and for all their courses. A composite calendar of events for all of their courses is their suggested solution.

It is interesting to me that through this survey, students had an opportunity to recommend new and innovative technological solutions, yet they focused much more so on issues of design and delivery – on improving existing processes.

The good news is, we can do a lot of this stuff!

Adopt a Peer-reviewed Open Textbook

When considering the adoption of open educational resources (OERs) I have heard instructors express concern regarding the quality of the materials – stating a preference for commercially published materials because they are peer-reviewed. That excuse is losing merit on a number of fronts as educators, together with public and private organizations, work together in addressing these concerns.

Book Stack
CC-BY-NC- by Benton Library Media Center on Flickr

We recently learned that the openly licensed Precalculus textbook authored by Carl Stitz, Ph.D. (Professor of Mathematics, Lakeland Community College) and Jeff Zeager, Ph.D. (Associate Professor of Mathematics, Lorain County Community College) has been approved by the American Institute of Mathematics.  Stitz and Zeager have released their textbook using a Creative Commons License.

The text is available free for students to download in pdf format, as well as at a very reasonable price for the print version from Lulu.

In a recent report by U.S. PIRG, entitled “Affordable Higher Education: Fixing the broken textbook market…

  • 65% of students surveyed reported they had decided against buying a textbook because it was too expensive.
  • [despite this fact] … 94% of students who had forgone a textbook were concerned doing so would hurt their grade in a course.
  • Nearly half of all students surveyed said that the cost of textbooks impacted how many / which classes they took each semester.

In an era where the focus in on completion and student success, we can no longer ignore the impact the high cost of textbooks has on our students and college affordability.

OpenStax College, an initiative of Rice University offers free open licensed peer-reviewed Textbooks in several general education subjects including: Physics, Sociology, Biology, Anatomy & Physiology, Statistics, Economics, Macro-economics, Micro-economics. More textbooks are in the queue including: Chemistry, Pre-calculus, History, and Psychology.

The OpenStax textbooks are licensed under a Creative Commons 3.0 license