Online Learning Rubric:
Cal State Chico has developed a rubric for online learning to help instructors design an effective course. Instructors may assess their online course and make necessary revisions, or use it to design a new course (Chico). Six categories are outlined beginning with the student:
1. Learner support and resources includes information about being an online student, course resources, and supporting course content resources.
2. Outline organization and design analyzes how students navigate the course and how well it is organized. This includes the visual presentation of the course, consistency and the course syllabus.
3. Instructional design and delivery should promote communication, align goals to learning objectives and activities. Student learning activities are addressed to include multiple learning styles, problem-solving skills and critical thinking.
4. Assessment and evaluation of student learning includes assessment of student readiness. Multiple assessment strategies are utilized which include consistent feedback and self-assessments.
5. Appropriate and effective use of technology addresses the use of tools to encourage communication, new teaching methods and multimedia elements. Student engagement is vital.
6. Faculty use of student feedback
(CSU, Chico)
The course I will evaluate is called “Fantasy and Science Fiction: The Human Mind, Our Modern World.” It is an online course available from the University of Michigan through Coursera.org. While this is not a course for credit and will not have all the components of a semester-long college course, many of the same requirements for success must be addressed.
This course includes an introductory unit of video clips discussing how one should proceed plus ten content units. Each content unit asks for the reading of a book or book-length selection of writings in the field of fantasy and science fiction and offers the chance to write a brief essay about that unit’s reading and to comment on the writing of four other participants. Each content unit begins with a video clip with some advice about that unit’s reading and later provides a series of clips, totaling about 1 1/2 hours, discussing both the unit’s reading and general matters that that reading helps explore. The course also offers an enrichment quiz (ungraded) for each unit and an on-going forum for participant discussion. That forum will be monitored and may stimulate the creation of a supplementary clip or two per unit. If any participant desires a grade, the grade will be determined by the quality and quantity of the writing and responses to the writing of others. (Coursera.org)
I believe one of the learning theories within this course is connectivism which has been described as a learning theory for the digital age (Herrington, n.d.). The students who participated came from a wide range of previous learning capabilities and over a wide geographical range, including those not speaking English as their primary language. Therefore the learning and knowledge was contingent on this diversity of opinions (Herrington, n.d.). This was evident with the peer-reviewed and peer-graded section of the course.
Students were advised to grade 500-word essays using a rubric number scale of 1-4, evaluating grammar in one score and ideas presented in another score. Students had a set time to write an essay and a strict deadline. The evaluation period began the following day, with only those students who submitted an essay able to perform evaluations on peers’ essays. Directions were clear as to the dates and times and rules, fulfilling the rubric stated in CSU, Chico’s guidelines. Students had to participate in the evaluation portion to receive an evaluation on their own essay. This was clearly set forth: Remember: as with the submission stage, this evaluation stage is required if you want your own assignment submission to be evaluated. (Coursera.org)
Once again, there was a strict timeframe for the submission of the evaluations. The next day, you could access your essay, with four peer-reviewed assessments. This method aligns with the connectivism theory, as learning and knowledge is based on a diversity of opinions (Kinesh, 2012). This was true, as each student had a different way of grading, even with a rubric. The evaluations were as varied as the student population. Within the evaluations of the same essay, students had differing opinions about whether the thesis was supported or if it was acceptable to make comparisons to other literature. One of the drawbacks was the inability to identify English as a second language students and grade them accordingly.
I would rate the visual organization of the course as high on the CSU rubric. The navigability was intuitive with icons on the left side of the screen. The steps were outlined clearly without too much wording. Students did not have to read through pages of information to navigate between screens. Each assignment became available with each new unit and the whole course was broken down into “chunks” of learning or units.
Each unit opens with an asynchronous video – lecture from the professor utilizing screen shots of words and images, to introduce the course material. This method fits into the behaviorist learning theory – transmitting information (Herrington, n.d.). I found this method to be advantageous over straight constructivist theory that may require students to search and find their own information and introduction to the unit ( The video lecture format, asynchronous and visual prompts the learning to “get excited” about the reading and writing assignment. It is immediate and require little effort, using some of the instructionist learning theory.
After the written assignment, the professor provided a set of video lectures for each unit that provided more insight. This method utilizes the constructivist learning theory, where students watched the short video introduction for the reading material, then read the assignment and wrote the essay without receiving more information from the professor. Students were required to make their own conclusions from the reading, and derive their own thesis. It was entirely open to whatever a student might want to write in his/her essay.
While this fits within a constructivist learning theory, it also opens the door to a wide range of essays and consequently, a wide range of opinions by the peer reviewers (Kinesh, 2012). But you could see where some students had more knowledge and previous experience in their writing. The task becomes difficult to grade each student on the increase of their individual learning and not compare one to another. There were students who became discouraged when they did not get a “good” assessment from another student, which became evident on the discussion boards.
Besides the writing assessment, students had access to a “more to learn” icon where they could take small quizzes about the novels with multiple choice answers. These gave immediate feedback with icons and verbiage: “yes” thumbs up and why it was correct, or “nope” why that was incorrect. The immediate feedback with explanation fits into the instructionist learning theory but with immediate feedback learning is accelerated by providing information at a critical step of learning (Herrington, nd.d).
This course includes an introductory unit of video clips discussing how one should proceed plus ten content units. Each content unit asks for the reading of a book or book-length selection of writings in the field of fantasy and science fiction and offers the chance to write a brief essay about that unit’s reading and to comment on the writing of four other participants.
Each content unit begins with a video clip with some advice about that unit’s reading and later provides a series of clips, totaling about 1 1/2 hours, discussing both the unit’s reading and general matters that that reading helps explore. The course also offers an enrichment quiz (ungraded) for each unit and an on-going forum for participant discussion. That forum will be monitored and may stimulate the creation of a supplementary clip or two per unit. If any participant desires a grade, the grade will be determined by the quality and quantity of the writing and responses to the writing of others.
The final category of the CSU rubric addresses course content and end of course evaluation survey, both of which were included in the University of Michigan course. A “post-course survey” of 26 questions was easily accessible by an icon on the left of the main page. Besides questions of age, gender, country and language, the survey addressed issues that students may have had with the course.
Which best describes your motivation for taking this class?
Did you participate in the course with the goal of earning a certificate, or did you choose to audit instead? Are you currently a student in a school or college?
What is your highest level of education?
If you went to college or are currently going to college, what is thename of your college or university?
Have you ever attended the University of Michigan? If so, what is yourcurrent status? If you have attended the University of Michigan, we willbe following up with another survey soon to gauge your interest in
further continuing education opportunities, and determine which types of
courses you feel would be most useful.
How much did the pre-recorded lectures help you feel connected to theprofessor?
How beneficial did you find the weekly "Some Thoughts On..." videos?
How did you feel about the content and delivery of the lectures?
How did you feel about the design of the syllabus and other explanatorydocumentation?
Did you feel that the course was the appropriate length?
How satisfied were you with the grading options for the peer assessments?
What is your opinion of the peer assessment grading scale?
Please use this area to give us your general feedback on the course.
What did you like?
What would you change if you could?
Please feel free to tell us a little about yourself. (Coursera.org)Checklist
References
Herrington A., & Herrington, H. Authentic mobile learning in higher education. Retrieved from http://www.aare.edu.au/07pap/her07131.pdf
Horton, W. (2012). E-Learning by Design. San Francisco, CA: Pfeiffer
Kinash, S., Brand, J., & Mathew, T. (2012). Challenging mobile learning discourse through research: Student perceptions of blackboard mobile learn and iPads. Australasian Journal of Educational Technology, 28(4), 639-655. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=ehh&AN=77978502&site=eds-live
Great description of a course including analysis of the theoretical components.
ReplyDeleteThe correct citation for the theory of Connectivism is
Siemans, G. (2004). Connectivism: A Learning Theory for the Digital Age. Retrieved April 4, 2008, from http://www.elearnspace.org/Articles/connectivism.htm
I was involved in the early efforts to test and verify the theory of connectivism, mostly conducted in Massively Open Online Courses (MOOC). The original MOOC was open in the sense of enrollment but also in the sense of content, participants choose their own focus, online learning environments and learning networks. The concept of MOOC has been appropriated by large learning organizations and has been squeezed back into the original toothpaste tube of prescriptive institutional education. The dialog continues about connectivism and the connectivist MOOC, now designated cMOOC as distinguished from the institutional version or xMOOC.
A useful framework for understanding the theoretical paradigms in elearning can be found at
Williams, R., Karousou, R., & Mackness, J. (2011). Emergent learning and learning ecologies in Web 2.0. International Review of Research in Open and Distance Learning, 12(3). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/883
Good checklist. See also Kathy Schrock's Web site for comprehensive and detailed information for constructing and validating rubrics for online learning.
http://www.schrockguide.net/assessment-and-rubrics.html