Teaching and Learning at The University of Hong Kong HKU

e-learning Blog    > HKU Online Learning & MOOCs    > CETL    > UG Research Fellowship

 
HKUx Blog edx MOOC

Thank you for joining us at the HKU EdTech Day on April 26, 2018! In this event, we showcased HKU’s pioneering efforts in developing e-learning and introduced the latest (and coolest!) technology to teachers, students and visitors. We were overwhelmed by the enthusiasm of our guests!

Highlights of the day

AR Sky Lantern
Make a wish

Wish upon a lantern

AR Dinosaur
Free photo taking with our 3D Gigantoraptor

Filming Station Photo-taking
Free photo taking in our mini studio
A unique souvenir to bring home!

VR Tilt Brush

Unleash your creativity – 3D painting in the air!

VR Rock Climbing

Fire up your adventurous spirit!

Our Place in the Universe

Explore the wonders of the Universe with the Armillary Sphere app
* This app will be used in our upcoming Massive Open Online Course (MOOC) Our Place in the Universe, which will be launched in June 2018. Learn more about the MOOC here. More details will be announced on Facebook.

HKU Online Learning - Courses and tools
A showcase of our online courses and the Video Vox Platform

EdTech Tools Showcase
Experience instant online polling with Mentimeter

Student Initiative

eLearning services provided by ITS

imseCAVE visit

Thank you once again for joining us in this joyous occasion. Check out our Facebook, Twitter and Instagram for more photos!

Contact us if you are interested in using edtech tools in your classroom!

We hope to see you all again in our upcoming events! The next one will be the Authentic Assessment Symposium on May 3!


This blog post is part of the ‘Flipped Classroom Professional Development Series’.

There are multiple ways to assess the effectiveness of your flipped class. While there is no single perfect way to measure teaching effectiveness, practitioners from HKU have come up with a few useful methods and tips for evaluation, which they shared in the Flipped Classroom Learning Symposium – Sharing of Pedagogies and Practices. In general, adopting a mixed approach allows you to evaluate your class more comprehensively.

How Researchers Measured Effectiveness in the Literature

  1. Criteria of evaluation
    Effectiveness of the flipped classroom has been measured by multiple ways in the literature, most palpably by examining the course’s direct and indirect educational outcomes. A scoping review conducted by O’Flaherty and Phillips (2015) summarized how educators evaluated the effectiveness of a flipped class by measuring various direct and indirect educational outcomes.

    While different researchers may have different definitions of “educational outcomes”, direct educational outcomes usually refer to (i) students’ scores and grades in traditional summative assessment and (ii) attendance. In particular, students’ performance in tests, exams, group work and group presentations are often used for evaluation in research (Cheng, Lee, Chang & Yang, 2017; Cotta, Shah, Almgren, Macías-Moriarity & Mody, 2016; Gilboy, Heinerichs & Pazzaglia, 2015). In contrast to direct outcomes, indirect educational outcomes include (i) students’ course experience; (ii) their attitudes, perceptions, and feelings towards the course; (iii) student engagement and learning behavior (measured by learning data); and (iv) student empowerment and development in the course, e.g., development of high order thinking skills, such as creativity, problem-solving and critical thinking skills, etc.

    According to O’Flaherty and Phillips (2015), limited research had been conducted on evaluating student learning outcomes in terms of their development of high order thinking skills; more researchers chose to evaluate (i) student satisfaction of the flipped class; (ii) student-teacher interactions; (iii) student engagement in using e-learning gadgets such as apps in mobile devices; and (iv) the opportunity for real-time and immediate feedback (Gilboy et al., 2015)

  2. Tools for data collection
    Apart from evaluating students’ performance in assignments and reports, various tools can be used to collect data. Examples include student evaluation surveys and interviews. Some researchers also supplement their findings with their own observations.

Strategies Used by Practitioners in HKU
In the Flipped Classroom Learning Symposium, practitioners from HKU shared with us how they evaluate the effectiveness of their flipped classes. In general, they tend to adopt a mixed approach in evaluating the effectiveness of flipped classes, i.e. analyzing both direct and indirect educational outcomes, instead of only using one instrument to evaluate a course. This allows them to evaluate their courses more comprehensively.

Criteria of evaluation
When evaluating the effectiveness of their courses, the practitioners usually collect the following types of information:

  1. Students’ grades: For example, Mr. Mathew Pryor, course instructor of CCHU9001 Designs on the Future, considered grades as strong evidence of students’ improvement.
  2. Students’ comments and perceptions on (i) quality of teaching (in terms of clarity of delivery, clarity of goals and standards, opportunities for skill development, etc.); and (ii) assessment design and workload.
  3. Students behaviour in face-to-face interactions and online

Methods of data collection
Students’ feedback can be obtained through formal and informal means.

  1. Formal feedback can be obtained through surveys and interviews.
    • In HKU, the Student Evaluation of Teaching and Learning (SETL) questionnaire is issued at the end of each course as an official way to evaluate course and instructor effectiveness. In Mr. Pryor’s case, SETL scores served as useful reference for his own performance. Both the quantitative scores (direct ratings by students) and the qualitative response (in the form of open-ended comments) provide vital information for him to improve his course. Using this questionnaire, he discovered that his student evaluations “go up by 10%” after flipping his class. The questionnaire provides concrete evidence that proves the effectiveness of the flipped class approach.
    • In 2014, Professor Rick Glofcheski collected students’ feedbacks on his Tort Law flipped class using a survey with TeLi’s support. The survey collected both quantitative and qualitative evidence of the effectiveness of his flipped classes. Below are some examples:
      Quantitative evidence:
      60% students found the classes “useful”, and 34% “very useful”.
      (Image credit: Professor Rick Glofcheski)

      Qualitative evidence (anonymous comments from students):

      • “It helps me better understand and remember the consideration factors of duty of care.”
      • “It also is an opportunity to discuss with other classmates and get ideas and inspirations from them.”
      • “The class also acts as a useful preparation for future legal practices as it encourages students to articulately express themselves in both oral and written forms.”
      • “Very useful, made me understand the problems better and engage in debate with other students.”
    • Dr. Ng Ming Yen from the Department of Diagnostic Radiology also collected feedback from students in his tutorials on chest pain imaging using a questionnaire. It was part of an experiment he conducted in 2016-2017 to examine the effectiveness of the flipped class approach. 60 students first attended lectures and completed a questionnaire. They then attended flipped classes 6 months later and filled in the questionnaires again. The result showed that the students generally appreciated the videos and over 75% of them thought that the flipped class was an improvement.
       
      Apart from quantitative data, Dr. Ng also collected qualitative comments from students. For example, some students asked for more cases and more time for discussion. These comments provide references for improvement in the next cohort.
  2. Informal feedback can be quickly obtained by teachers in class and online. For example, a quick show of hands gives teachers a rough impression of whether students enjoy an activity. Teachers can also invite students to give anonymous feedback using discussion forums or online polling tools, such as Mentimeter.

    In Mr. Pryor’s case, he highly valued and respected students’ feedback. To understand how students perceive his teaching, he collected informal feedback by asking simple, straight-forward questions such as “Which activity do you like or not like?” or even “Are you happy?” on discussion forums or with Mentimeter. These immediate feedback from students are pivotal in course planning and strategizing.

  3. Observation of students’ behaviour in face-to-face interactions: It is also important for teachers to observe students’ response and behaviour in class, as their body language honestly reflects their extent of engagement and satisfaction. They provide alternative evidence to support findings generated in formal surveys. For example, For example, Dr. Courtney Fung evaluated the effectiveness of her teaching by observing students’ behaviour and response. In class, students assume roles of different nations and simulate real-world political negotiations to resolve crises. Since this activity was student-led, Dr. Fung acted as a facilitator and an observer during the process of negotiation. She observed that not only students were engaged in class, they even self-initiated further discussions over lunch after class. The level of engagement was high, which in turn reflected the effectiveness of the class.

    Dr. Courtney Fung

To sum up, it is best to evaluate a course from multiple dimensions, as different scales of measurement shed light on different aspects of a course. Direct and indirect educational outcomes, as well as students’ feedback, engagement and learning behavior, all have different advantages in telling how effective a flipped class is based on the nature of the course. Aligning your expected students’ learning outcomes with appropriate ways of measurement is crucial for effective evaluation.

Building a flipped class is a long process of development. From preparing online and pre-class elements, encouraging student participation, designing in-class activities, to evaluating  effectiveness, a lot of support and resources may be needed. It is our mission to support teachers in developing e-learning materials and flipping their classes. Contact us if you need help!

Next step
If you are interested in further exploring teaching and learning with us, don’t miss the Authentic Assessment Symposium: The Transformation of Learning in Higher Education on May 3!

This blog post is part of the Flipped Classroom Professional Development Series. More articles from the series:

References
Cheng, X., Lee, K. H., Chang, E. Y., & Yang, X. (2017). The “flipped classroom” approach: Stimulating positive learning attitudes and improving mastery of histology among medical students. Anatomical Sciences Education, 10(4), 317-327. Retrieved from https://search-proquest-com.eproxy1.lib.hku.hk/docview/1969022918?accountid=14548

Cotta, K. I., Shah, S., Almgren, M. M., Macías-Moriarity, L. Z., & Mody, V. (2016). Effectiveness of flipped classroom instructional model in teaching pharmaceutical calculations. Currents in Pharmacy Teaching and Learning,8(5), 646-653.

Gilboy, M. B., Heinerichs,S., & Pazzaglia, G. (2015). Enhancing student engagement using the flipped classroom. Journal of Nutrition Education and Behavior,47(1), 109-114.

O’Flaherty, J. & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. Internet and Higher Education, 25(8), 85-95. Doi: 10.1016/j.iheduc.2015.02.002.

The University of Hong Kong (2018). Educational aims and institutional learning outcomes. In Undergraduate Handbook. Retrieved from http://www.handbook.hku.hk/ug/full-time-2017-18/important-policies/educational-aims-and-institutional-learning-outcomes


Organised by Technology-Enriched Learning Initiative (TELI)

Authentic Assessment Symposium banner

Details of the event:

Date : 3 May, 2018 (Thursday)
Time : 9:30am – 1:00pm
Venue : CPD-LG.18, LG/F, Central Podium, Centennial Campus, HKU

Abstract

In the higher education sector, assessment has been characterized as “driving student learning” – it determines students’ learning strategies and affects their learning outcomes. Authentic assessment strategies, which draw connections between the subject matter and real-world problems, have demonstrated high effectiveness and efficiency in clinical education. But is that the only context where authentic assessment could be applied? Can authentic assessment be adopted in day-to-day classroom teaching and learning across different subjects?

In this symposium, practitioners from law, medicine, dentistry, education, science, social sciences, architecture, arts and CAES will share their philosophy and practices in applying authentic assessment in their classroom. Student representatives will also be invited to share their learning experience and how authentic assessment has enabled deeper learning.

Registration

Enquiries should be directed to enquiry@teli.hku.hk.


Inter-professional team-based learning (IPTBL) is an innovative teaching approach initiated by the Li Ka Shing Faculty of Medicine in the University of Hong Kong (HKU). It is a pioneering effort to promote peer-to-peer learning and inter-professional education across healthcare disciplines using team-based learning (TBL), flipped learning and e-learning. First piloted in 2016, the initiative engaged 500+ students across 7 programmes from HKU and the Hong Kong Polytechnic University (PolyU). Based on the feedback collected, the event was further scaled out in 2017, serving more than 1,000 students (and as many as 501 students in one particular session) from 12 programmes from the two universities.

In this large-scale flipped classroom implementation, we have demonstrated the following attributes:

  • Implementation of an entirely new mode of teaching, pushing teachers and students to step out of their comfort zone;
  • Use of technologies to facilitate teacher-student/student-student interactions; and
  • Effective collaboration among teaching units and teaching-support units.

The event served more than 1,000 students (and as many as 501 students in one particular session) from 12 programmes from the two universities.

We are happy to share that the initiative has been recognized locally and internationally. In 2016, the IPTBL team won the Bronze Discipline Award (Life Sciences) in the prestigious QS Stars-Wharton Reimagine Education Awards. Taking one step further in 2017, we had been shortlisted for the HKU Professional Services Awards 2017 (Team category), an annual award programme at the University level to recognise and promote excellence in professional services. We would like to express our gratitude to the University, our colleagues and students for their support, without which our work would not have been possible.

Raising Standards of Classwork Activities

The motivation behind implementing this initiative is to enhance the quality of teaching and learning by pushing both teachers and students out of their comfort zone.

Facilitating teachers to step out of their comfort zone

Traditionally, many teachers tend to focus on their own discipline in teaching or even a particular topic when delivering a lecture. In the case of IPTBL, all members of the teaching team stepped out of their disciplinary silos and collaborated with teachers from different disciplines and institutions.

Facilitating students to step out of their comfort zone

Many health professional graduates in the work field found it difficult to communicate with their colleagues with different backgrounds, as they mainly received training with students from the same discipline in university. In IPTBL, students from multiple disciplines were grouped in ways that reflected the realities of the healthcare sector. This created ample opportunities for them to develop communication and collaboration skills essential for their future career.

Enhancing Teaching and Learning with Technology

Since a large number of students were involved in each session, an online learning platform and a peer evaluation system were developed to facilitate teacher-student and student-student interactions.

Facilitating teacher-student interactions

It is not easy for teachers to engage hundreds of students and monitor their progress simultaneously in face-to-face sessions. To facilitate teachers in managing the class, an online Learning Activity Management System (LAMS) was introduced. In class, students needed to log-in the system and answer questions individually and in teams. The system allowed teachers to track their real-time progress through the different stages of TBL. This system also made it easier for teachers to decide on which groups to interview in the interactive feedback session.

The Learning Activity Management System (LAMS)

Facilitating student-student interactions

A peer evaluation system was developed to enable students to deliver real-time feedback to their teammates for timely self-reflection. The system also came with an online discussion forum to facilitate student interactions after class.

Overcoming Challenges in Implementing Large-Scale Classroom Activities

Careful scheduling

As IPTBL involved teachers and students from multiple departments and universities, careful scheduling of each session was necessary. To accommodate all parties’ schedules, all six sessions were held on Saturday mornings.

Briefing sessions

With a large number of students and teachers from various disciplines involved, it was essential to ensure all necessary operation details were effectively communicated to all parties. To overcome this challenge, the team conducted multiple briefing sessions in both universities, not only with teachers, but also with students. In the briefing sessions, the importance of IPTBL, the session formats and rundown were explained in great details.

Creating an environment suitable for large-scale in-class activities

As it was necessary to accommodate a large number of students in each session, the team chose the Centennial Campus Lecture Hall II as the core learning space as it is a large flat area with mobile chairs and strong WiFi connectivity. Standard large lecture halls such as Grand Hall or Rayson Huang Theatre were not chosen as they might not be suitable for large-scale group discussion, both online and offline. Since the halls are not flat and the chairs are not movable, it will be difficult for students to move around and work in groups. It may also be difficult for teachers and students to communicate online as the WiFi connectivity in these halls are relatively weak.

As the number of students varied in each session, floor plans were drawn for each session to ensure that all students enjoyed the best learning environment. In the actual implementation, balloons with group numbers were attached to each group so that teachers and students could quickly locate individual teams.

Balloons with group numbers were attached to each group so that teachers and students could quickly locate individual teams.

Collaborations with teaching-support units

Logistics of setting up a suitable environment for large-scale group discussions can be complicated. Reconfiguring the Centennial Campus Lecture Hall II for the IPTBL initiative is particularly challenging, as the hall is usually partitioned into four standard classrooms. Transforming this learning space requires partitions to be lifted up and chairs to be reorganized – a process that would take several hours to complete. As the sessions were held on Saturday mornings, when most HKU offices were closed, the team had to liaise with parties such as Examinations Office and Estates Office to arrange for earlier set up of the venue on the night before.

Each session required materials such as reception tables, refreshments and backup paper tests (in case the online learning system failed to function properly). As most HKU offices were closed on Saturday, transportation of materials was a major challenge to overcome. Extra efforts were made to ensure logistics concerns were addressed – All the materials needed were transported from HKU Medical Campus to Main Campus on the night before. With the assistance from units such as Information Technology Services (ITS) and Learning Environment Services (LES), storage areas were made available.

Multiple rehearsals

To ensure the smooth running of the event, multiple rehearsals had been conducted. With limited time available for each IPTBL session (only 4 hours each) and a tight schedule, time management was crucial to ensure students went through the complete TBL process. Intensive rehearsals and prior testing of the online learning system were necessary to minimize technical problems and possible delays in the actual implementation.

Looking Forward

The IPTBL programme had grown continuously from 2016 to 2017. In the future, the course team aims to continuously improve the online learning platform to provide more informative feedback to both the facilitators and students.

If you are interested in using technology in class or experimenting with new ways of teaching, contact us.


« Previous 1 2 3 4 5 52

Copyright © 2018 The University of Hong Kong. All Rights Reserved Contact Us