Measuring MOOC Learner Success and Post-course Career Development

success_banner_20171123

How do you measure success? Many researchers, when evaluating Massive Open Online Courses (MOOCs), tend to measure the success of a course with a single metric, i.e. the completion rate. However, MOOCs generally have a large audience, and given the diverse background of MOOC learners, every learner differs in goals and level of engagement with the course materials. It may not be comprehensive enough to evaluate learner success by simple looking at whether he/she has finished the course; and the overall course completion rate alone may not suffice in evaluating the success of a MOOC.

This is what we learnt from a seminar entitled “Demystifying Learner Success: Before, During, and After a Massive Open Online Course”, delivered by Dr Elle Yuan Wang, a Research Scientist at EdPlus Action Lab, Arizona State University on August 1, 2017. Organized by the Faculty of Education, this seminar offered us a different perspective on how to measure success of a MOOC, an ongoing debate among researchers.

Learner Success in terms of Post-Course Career Development
Dr Wang believes that learner success can take many different forms – traditional assessment scores, or in other forms of post-course development, such as career development. In her 2014 study, she measured the post-course development of a group of MOOC learners two years after the end of the course using two metrics: whether learners (i) joined a relevant professional society; and/or (ii) submitted a paper in a relevant conference. By comparing learners’ post-course career development and in-course performance, she set out to investigate the relation between the two. The ultimate goal is to find out how career advancers differ from other learners in terms of their in-course performance.

The research targeted learners of the first iteration of The Big Data in Education MOOC, a postgraduate-level 8-week course offered on Coursera in 2013. Dr Wang was one of the teaching assistants of the course. (The subsequent iterations of the course have been offered on edX.)

The study revealed that:

  • Career advancers who joined a professional society or submitted a paper earned better scores and were more likely to complete the course than non-advancers.
  • Career advancers also demonstrated more frequent engagement with course components including course pages, lecture videos, assignment submissions, and discussion forums. For example, the page viewing activities of people who joined a professional society were much higher than non-members.
  • However, even though career advancers tended to have more post-reading actions, they were not significantly more likely to post, comment, or vote than their peers.

Significance of the Research
This study enriched our understanding of how MOOCs potentially impact learners’ career development and the possible association between student behaviors and positive developments. All these findings are crucial for educators in developing and improving their MOOCs in the future.

A Special Note of Thanks
Hereby we would like to thank Dr Wang for not only sharing with us her research endeavours and findings, but also inspiring our work in learning analytics in HKU. Our colleague, Dr Leon Lei, completed The Big Data in Education MOOC on Coursera in 2013 and is now applying principles learnt from the course in developing our own MOOCs in HKU. Thank you, Dr Wang, for inspiring us. We look forward to more opportunities to further explore learning analytics and educational data mining with fellow researchers in the future.

*Note: Dr Wang’s research was conducted in collaboration with Ryan Baker, University Pennsylvania, and Luc Paquette, University of Illinois at Urbana-Champaign. For further details, please refer to the original research paper: Wang, Y., Paquette, L., Baker, R. (2014). A longitudinal study on learner career advancement in MOOCs. Journal of Learning Analytics, 1(3), 203-206. [PDF]

My Little Story with Mom – Jockey Club End-of-life Community Care Project

minimovie_LR-4

“This is part of what a family is about, not just love. It’s knowing that your family will be there watching out for you. Nothing else will give you that. Not money. Not fame. Not work.” – Mitch Albom, Tuesdays with Morrie

How will you spend the days with your beloved one if either of you only has 3 months left? This is Iris’ burning question when her mother’s days were numbered in My Little Story with Mom. Featuring Miss Chau Ka-Yee, Catherine (周家怡) and Miss Shaw Yam-Yam, Susan (邵音音), this mini movie will touch your heart. Get your tissues ready…

*English subtitles are now available.

The video is also available on Facebook. Give us a like!

Cast
Chau Ka-Yee, Catherine (周家怡) as Iris Fung (馮艾雯)
Shaw Yam-Yam, Susan (邵音音) as Jenny Cheung (張鳳珍)
Man Chi-leung, Alex (萬梓良) as Ming Tin (明天)
Cheung Song-Tak, Peter (張崇德) as Doctor

Director
Eda Wong (黃懿德)

Composer (soundtrack and theme song)
Yusuke Hatano (波多野裕介)

Perhaps, in the final stage of life, what matters the most is to apologize (道歉), love (道愛), thank (道謝), and bid farewell (道別) to your beloved family.

It is never easy to say goodbye to our loved ones, especially those with terminal diseases. It is never easy to endure the pain. But as Catherine sings in the theme song,

“The little things every day, however mundane and trivial
Remind me that, it isn’t too late to hug you
To hold you close, survive the pains
While every day the absurdity of life abounds
I wish it will stay within, every sweet moment,
However common they seem, for they will come to an end.”

We have no choice in the face of death. But we can choose to love before it is too late. Treasure every tiny moment while you still can. After all, love is all that matters.

The JCECC Project and the Mini Movie Initiative
This mini movie is produced by TELI as a major initiative of the “Jockey Club End-of-life Community Care Project” (JCECC) in collaboration with the Faculty of Social Sciences. Launched in 2015 and funded by the Hong Kong Jockey Club Charities Fund, the JCEEC project aimed at improving end-of-life care quality, enhancing the capacity of service providers, and raising public awareness in the society. Alongside public seminars, this mini movie was premiered in Grand Hall, the University of Hong Kong on November 1, 2017. The Gala Premiere attracted a full-house audience of over 1,200 and wide media attention.

5 Months of Preparation: A Challenge and A Milestone for TELI
The production of this mini movie was an exciting challenge, a milestone, and a wonderful opportunity for our team to work with multiple parties both inside and outside the University.

It was our first attempt to produce a mini movie and use movie as a tool of public education. Our multimedia colleagues in particular embraced this challenge with 5 months of hard work and preparation. From scripting, storyboarding, casting, props and costume design, scene-setting, filming and post-production, it took almost half a year of intensive work to bring this mini movie into life. The completion of this project is a milestone for us. It was also a valuable opportunity for us to collaborate with public health care professionals, famous actors and award-winning musicians.

Behind the Scenes
Do you want to see how we made a movie from scratch? Here is a peek behind the scenes of the mini movie. Enjoy!

  1. Scripting and storyboarding
  2. The production process began with a meticulous storyboard.
    minimovie_LR-4The scene of Iris and Jenny taking a taxi to the hospital

    minimovie_LR-5The scene of Iris and Jenny cooking and dining at home

  3. Filming
  4. One of the biggest challenges in producing the movie was to film multiple scenes in a taxi. Filming outdoors required constant adjustment and monitoring of light setting, as well as persistent effort in noise reduction throughout the production and post-production process.

    minimovie_LR-2Filming in a taxi was a daunting challenge!

    minimovie_LR-2Our director Eda Wong viewing the playback

    minimovie_LRFilming of Catherine Chow and Susan Shaw in Centennial Campus, HKU

  5. Production of the theme song and soundtrack
  6. We recorded the theme song with real instruments (strings and guitar) at Shaw Studio, and then with Miss Catherine Chow in Perception Music Production.
    Thanks to this mini movie initiative, our team enjoyed a precious opportunity to work with renowned musicians to produce the theme song and soundtrack, including Mr. Yusuke Hatano and Mr. Adrian Chan. Mr. Hatano is the winner of the “Best Original Film Score” for the film Soul Mate in 2017. Mr. Chan is a talented local song producer who has worked with famous singers such as Leslie Cheung, Faye Wong and Eason Chan.

    minimovie_LR-7Miss Catherine Chow recording the theme song in Perception Music Production

  7. Final testing

After the completion of the mini movie, preparing for the Premiere was another challenge for our multimedia team. Several rounds of adjustment were made to ensure the movie was compatible with the visual-audio system in Grand Hall.

minimovie_LR-6Our team working in Grand Hall to adjust the dialogue and foley design

Let’s Join Hands to Impact the Community
It is our honour to give back to the society in collaboration with our community partners, and we are eager for more opportunities to maximize our impact on students and the greater community. If you are interested in experimenting with new ways to educate students and the general public, contact us.

Further reading
On the JCEEC Project and end-of-life care

  1. Jockey Club End-of-Life Community Care Project
  2. 張亮:【名家搶先睇】未知生、焉知死 [An article on life and death written by Mr. Leong Cheung, Executive Director, Charities and Community, Hong Kong Jockey Club] (Apple Daily, 29 October 2017)

On the mini movie

  1. 邵氏高層女兒黃懿德 從偷拍到明拍 [An interview of our director, Ms. Eda Wong] (Hong Kong Economic Journal, 21 November, 2017)
  2. 陳智思:點畫人生——媽媽和我的小事 [An article written by Mr. Bernard Chan, the incumbent Non-official Convenor of the Executive Council.] (The Headliner, 10 November 2017)
  3. 張亮:【名家搶先睇】媽媽和我的小事 [An article written by Mr. Leong Cheung] (Apple Daily, 5 November 2017)
  4. 「賽馬會安寧頌」微電影 母女共度最後時光好感人 [Translation: JCEEC Mini movie: A Touching Story about a Mother and Daughter Spending their Last Moments Together] (Headline Daily, 2 November 2017)
  5. HKU Press Release, 31 October 2017:
    English: Gala Premiere: “My Little Story with Mom” and End-of-Life Care Public Seminar
    Chinese: 「媽媽和我的小事」微電影首映禮 暨 導演、演員分享會及公眾座談會