Thank you for joining us at the HKU EdTech Day on April 26, 2018! In this event, we showcased HKU’s pioneering efforts in developing e-learning and introduced the latest (and coolest!) technology to teachers, students and visitors. We were overwhelmed by the enthusiasm of our guests!
Highlights of the day
AR Sky Lantern Make a wish
Wish upon a lantern
AR Dinosaur Free photo taking with our 3D Gigantoraptor
Filming Station Photo-taking Free photo taking in our mini studio
A unique souvenir to bring home!
VR Tilt Brush Unleash your creativity – 3D painting in the air!
VR Rock Climbing Fire up your adventurous spirit!
Our Place in the Universe Explore the wonders of the Universe with the Armillary Sphere app * This app will be used in our upcoming Massive Open Online Course (MOOC) Our Place in the Universe, which will be launched in June 2018. Learn more about the MOOC here. More details will be announced on Facebook.
HKU Online Learning – Courses and tools A showcase of our online courses and the Video Vox Platform
EdTech Tools Showcase Experience instant online polling with Mentimeter
Student Initiative eLearning services provided by ITS imseCAVE visit
Thank you once again for joining us in this joyous occasion. Check out our Facebook, Twitter and Instagram for more photos!
Contact us if you are interested in using edtech tools in your classroom!
This blog post is part of the ‘Flipped Classroom Professional Development Series’.
There are multiple ways to assess the effectiveness of your flipped class. While there is no single perfect way to measure teaching effectiveness, practitioners from HKU have come up with a few useful methods and tips for evaluation, which they shared in the Flipped Classroom Learning Symposium – Sharing of Pedagogies and Practices. In general, adopting a mixed approach allows you to evaluate your class more comprehensively.
How Researchers Measured Effectiveness in the Literature
Criteria of evaluation
Effectiveness of the flipped classroom has been measured by multiple ways in the literature, most palpably by examining the course’s direct and indirect educational outcomes. A scoping review conducted by O’Flaherty and Phillips (2015) summarized how educators evaluated the effectiveness of a flipped class by measuring various direct and indirect educational outcomes.
While different researchers may have different definitions of “educational outcomes”, direct educational outcomes usually refer to (i) students’ scores and grades in traditional summative assessment and (ii) attendance. In particular, students’ performance in tests, exams, group work and group presentations are often used for evaluation in research (Cheng, Lee, Chang & Yang, 2017; Cotta, Shah, Almgren, Macías-Moriarity & Mody, 2016; Gilboy, Heinerichs & Pazzaglia, 2015). In contrast to direct outcomes, indirect educational outcomes include (i) students’ course experience; (ii) their attitudes, perceptions, and feelings towards the course; (iii) student engagement and learning behavior (measured by learning data); and (iv) student empowerment and development in the course, e.g., development of high order thinking skills, such as creativity, problem-solving and critical thinking skills, etc.
According to O’Flaherty and Phillips (2015), limited research had been conducted on evaluating student learning outcomes in terms of their development of high order thinking skills; more researchers chose to evaluate (i) student satisfaction of the flipped class; (ii) student-teacher interactions; (iii) student engagement in using e-learning gadgets such as apps in mobile devices; and (iv) the opportunity for real-time and immediate feedback (Gilboy et al., 2015)
Tools for data collection
Apart from evaluating students’ performance in assignments and reports, various tools can be used to collect data. Examples include student evaluation surveys and interviews. Some researchers also supplement their findings with their own observations.
Strategies Used by Practitioners in HKU
In the Flipped Classroom Learning Symposium, practitioners from HKU shared with us how they evaluate the effectiveness of their flipped classes. In general, they tend to adopt a mixed approach in evaluating the effectiveness of flipped classes, i.e. analyzing both direct and indirect educational outcomes, instead of only using one instrument to evaluate a course. This allows them to evaluate their courses more comprehensively.
Criteria of evaluation
When evaluating the effectiveness of their courses, the practitioners usually collect the following types of information:
Students’ comments and perceptions on (i) quality of teaching (in terms of clarity of delivery, clarity of goals and standards, opportunities for skill development, etc.); and (ii) assessment design and workload.
Students behaviour in face-to-face interactions and online
Methods of data collection
Students’ feedback can be obtained through formal and informal means.
Formal feedback can be obtained through surveys and interviews.
In HKU, the Student Evaluation of Teaching and Learning (SETL) questionnaire is issued at the end of each course as an official way to evaluate course and instructor effectiveness. In Mr. Pryor’s case, SETL scores served as useful reference for his own performance. Both the quantitative scores (direct ratings by students) and the qualitative response (in the form of open-ended comments) provide vital information for him to improve his course. Using this questionnaire, he discovered that his student evaluations “go up by 10%” after flipping his class. The questionnaire provides concrete evidence that proves the effectiveness of the flipped class approach.
In 2014, Professor Rick Glofcheski collected students’ feedbacks on his Tort Law flipped class using a survey with TeLi’s support. The survey collected both quantitative and qualitative evidence of the effectiveness of his flipped classes. Below are some examples: Quantitative evidence: 60% students found the classes “useful”, and 34% “very useful”.
(Image credit: Professor Rick Glofcheski)
Qualitative evidence (anonymous comments from students):
“It helps me better understand and remember the consideration factors of duty of care.”
“It also is an opportunity to discuss with other classmates and get ideas and inspirations from them.”
“The class also acts as a useful preparation for future legal practices as it encourages students to articulately express themselves in both oral and written forms.”
“Very useful, made me understand the problems better and engage in debate with other students.”
Dr. Ng Ming Yen from the Department of Diagnostic Radiology also collected feedback from students in his tutorials on chest pain imaging using a questionnaire. It was part of an experiment he conducted in 2016-2017 to examine the effectiveness of the flipped class approach. 60 students first attended lectures and completed a questionnaire. They then attended flipped classes 6 months later and filled in the questionnaires again. The result showed that the students generally appreciated the videos and over 75% of them thought that the flipped class was an improvement.
Apart from quantitative data, Dr. Ng also collected qualitative comments from students. For example, some students asked for more cases and more time for discussion. These comments provide references for improvement in the next cohort.
Informal feedback can be quickly obtained by teachers in class and online. For example, a quick show of hands gives teachers a rough impression of whether students enjoy an activity. Teachers can also invite students to give anonymous feedback using discussion forums or online polling tools, such as Mentimeter.
In Mr. Pryor’s case, he highly valued and respected students’ feedback. To understand how students perceive his teaching, he collected informal feedback by asking simple, straight-forward questions such as “Which activity do you like or not like?” or even “Are you happy?” on discussion forums or with Mentimeter. These immediate feedback from students are pivotal in course planning and strategizing.
Observation of students’ behaviour in face-to-face interactions: It is also important for teachers to observe students’ response and behaviour in class, as their body language honestly reflects their extent of engagement and satisfaction. They provide alternative evidence to support findings generated in formal surveys. For example, For example, Dr. Courtney Fung evaluated the effectiveness of her teaching by observing students’ behaviour and response. In class, students assume roles of different nations and simulate real-world political negotiations to resolve crises. Since this activity was student-led, Dr. Fung acted as a facilitator and an observer during the process of negotiation. She observed that not only students were engaged in class, they even self-initiated further discussions over lunch after class. The level of engagement was high, which in turn reflected the effectiveness of the class.
Dr. Courtney Fung
To sum up, it is best to evaluate a course from multiple dimensions, as different scales of measurement shed light on different aspects of a course. Direct and indirect educational outcomes, as well as students’ feedback, engagement and learning behavior, all have different advantages in telling how effective a flipped class is based on the nature of the course. Aligning your expected students’ learning outcomes with appropriate ways of measurement is crucial for effective evaluation.
References
Cheng, X., Lee, K. H., Chang, E. Y., & Yang, X. (2017). The “flipped classroom” approach: Stimulating positive learning attitudes and improving mastery of histology among medical students. Anatomical Sciences Education, 10(4), 317-327. Retrieved from https://search-proquest-com.eproxy1.lib.hku.hk/docview/1969022918?accountid=14548
Cotta, K. I., Shah, S., Almgren, M. M., Macías-Moriarity, L. Z., & Mody, V. (2016). Effectiveness of flipped classroom instructional model in teaching pharmaceutical calculations. Currents in Pharmacy Teaching and Learning,8(5), 646-653.
Gilboy, M. B., Heinerichs,S., & Pazzaglia, G. (2015). Enhancing student engagement using the flipped classroom. Journal of Nutrition Education and Behavior,47(1), 109-114.
O’Flaherty, J. & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. Internet and Higher Education, 25(8), 85-95. Doi: 10.1016/j.iheduc.2015.02.002.
Organised by Technology-Enriched Learning Initiative (TELI)
Details of the event:
Date : 3 May, 2018 (Thursday) Time : 9:30am – 1:00pm Venue : CPD-LG.18, LG/F, Central Podium, Centennial Campus, HKU
Abstract
In the higher education sector, assessment has been characterized as “driving student learning” – it determines students’ learning strategies and affects their learning outcomes. Authentic assessment strategies, which draw connections between the subject matter and real-world problems, have demonstrated high effectiveness and efficiency in clinical education. But is that the only context where authentic assessment could be applied? Can authentic assessment be adopted in day-to-day classroom teaching and learning across different subjects?
In this symposium, practitioners from law, medicine, dentistry, education, science, social sciences, architecture, arts and CAES will share their philosophy and practices in applying authentic assessment in their classroom. Student representatives will also be invited to share their learning experience and how authentic assessment has enabled deeper learning.
Imagine, experiment, partner with students, build capacity – These are some tips in implementing the blended learning approach shared by expert practitioners in a seminar on February, 26, 2018. Entitled “Blended Learning: Are we Blending and at the same time, Enhancing Student Learning?”, this seminar featured Professor Bob Fox, Pro-Vice-Chancellor Education Portfolio, the University of New South Wales (UNSW); Dr. NS Wong, Associate Professor in the School of Biological Sciences; and Dr. Allan Yuen, Associate Professor in the Faculty of Education, HKU. The three speakers shared exemplary cases from various disciplines and their lessons learnt from implementing blended learning.
What is Blended Learning?
“Blended learning is the thoughtful fusion of face-to-face and online learning experiences,” (Garrison & Vaughan, 2008, p.5). In addition to that, it is the fusion of formal and informal; as well as synchronous and asynchronous elements in the curriculum, as pointed out by Dr. Yuen. With more and more teachers shifting to this approach, “blended learning is now the norm,” said Professor Fox.
Elements of Blended Learning
Blending different modes of learning in a course is an effective way to enhance student learning. The following is a list of examples introduced by the speakers:
Replace traditional lectures with tutorials and group activities:
In a Civil Engineering course in UNSW, traditional lectures were transformed into small group and individual tutorials, where students work through the content with a study guide. This mode of learning is problem-based, activity-led and self-paced. Weaker students can obtain extra tutorial support, while more advanced students are encouraged to support weaker one for badges.
In Dr. Wong’s course on metabolism, students are guided to form “communities of active inquiry” – together they explore concepts, re-synthesize information, propose new questions and complete in-class exercises. By blending “student-based research-style learning” into the course, “in-depth learning is positively promoted”, commented Dr. Wong.
Use of ed-tech
Videos: In UNSW, videos developed for Massive Open Online Courses (MOOCs) are re-used in flipping on-campus courses. This is termed as the ‘BOOC Flip’ (Blended Open On-campus Course).Dr. Wong also developed videos for his course CCST9006 Scientific and Technological Literacy. Here is a glimpse of his course:
Learning Management Systems, such as Moodle, are widely used as a gateway for students to coordinate learning, including conveniently access course materials, submit assignments, receive feedback and discuss with their peers.
Adaptive learning platforms: In UNSW, the course team of MATH2018 Engineering Mathematics 2D used an adaptive learning platform called Smart Sparrow to develop self-paced adaptive online tutorials for students, replacing F2F tutorials.
The MATH2018 course course team also developed a Ninja game. This game consists of a series of exercises with progressive levels of difficulty. Students need to work their way up until they obtain the black belt. This creates incentives for students to learn better.
In BLDG1021, faculty partners with professional builders and create opportunities for students to get involved in real world construction projects off-campus.
Tips for Developing Effective Blended Learning Experiences
To maximize the impact of blended learning, the speakers offered the following tips in implementing the approach:
Pilot and experiment
While it is exciting to incorporate new modes of teaching into your course, it can also be worrying for some – What if it does not work? One way to minimize risk is to pilot test your new ideas and conduct controlled experiments, advised by Professor Fox.
Partnership with students
Work with students in developing course materials
One effective way to develop materials that best suit students’ needs is to involve them in the development process. For example, UNSW pays students to develop resources for them and help faculty members innovate. In Dr. Wong’s case, he creates student internship and once invited a student to produce a short video for him to use in a practical session.
Obtain student feedback
Dr. Wong collects student feedback through administering questionnaires at the end of the course. The feedback is essential for making modifications to the course.
Capability-building for staff
As pointed out by Professor Fox, teachers need ongoing support in developing blended learning resources and materials. Learning from UNSW’s example, the following should be done on the institutional and administrative level to ensure effective implementation of blended learning strategies:
Develop curriculum models, frameworks and strategies for all faculties to buy into
Offer ongoing support to faculties and staff in team-building and capacity-building
Provide resources, technologies and/or funding
Provide promotion and encouragement to staff to get involved in improving teaching and learning
In UNSW, the promotion and tenure system has been refined to ensure that teachers get rewarded and promoted for their efforts made in teaching and learning. 400 academic positions which are education-focused have been set up. Teachers who choose this academic track can focus on teaching and learning instead of research for 5 years. After the 5-year period, they can decide whether to switch back to research track.
Imagine new ways to teach
“Learning is fundamentally about change,” said Dr. Yuen. “Students nowadays are very different from us when we were still students”. In order to enhance student learning, it is necessary for us to reconsider our pedagogy and imagine new ways to teach, using blended learning and other digital technologies. To Dr. Yuen, “blended learning / e-learning is not a simple technological adoption, but a call of teachers to carefully examine their pedagogical practices from a new perspective.” He believes that only by “allow[ing] our imaginations to be at work”, can we unleash the “enormous potential for growth and engagement” of higher education. He left us with two thought-provoking questions:
How do we shape and reshape new learning environment for the New Generation Learners?
What is your imagination in 5 years? How are you going to teach your subject in 5 years?
Dr. Wong believes that by shifting from traditional lectures to blended learning, we are setting students free from a rigid and often unengaging mode of learning. In order to maximize students’ freedom and effectiveness in learning, we need to set our imagination free in innovating our pedagogies.
Fostering stronger inter-collegiate links and sharing of knowledge expertise have always been high priorities in the World’s No. 1 Dental Faculty in the World. From January 29 to February 2, 2018, Dental Materials Science of the Faculty of Dentistry collaborated with HKU TeLi to run a winter programme in Applied Oral Sciences (AOS) at Prince Philip Dental Hospital.
AOS is a 5-day blended learning programme which aims to share (i) innovative methods and best practices in dental materials science research and on-campus instructions, as well as, (ii) technology-enriched pedagogies in teaching and learning, in particularly, in MOOCs and MOOC-based teaching and learning design to dentistry and medical faculties in mainland China and Korea.
The programme was packed with intensive knowledge sessions of featured lectures, as well as, experiential learning and interactive workshops.
Professor Jukka Matinlinna introducing the newest endeavours of Dental Materials Science Faculty.
Dr. James Tsoi talked about teaching and learning in Dental Materials Science.
Deepening Knowledge in Dental Materials Science Research
Through featured lectures conducted by Dental Materials Science faculty, students gained insights in some popular Applied Oral Health research areas including Biomechanics in Orthodontics, Biomaterials for Our Life and Current Approaches and Future Challenges in Dental Pulp Regeneration. These face-to-face sessions also allowed participants to interact with leading academics and dental professionals, as well as, peers from different universities.
Experiencing E-learning and Innovative Pedagogy
All the participants were enrolled in a customized version of the MOOC course for AOS. This enabled the participants to experience the courseware, in particular, the learning activities and bite-sized pedagogy, as well as, the newest science in Dental Materials Science education through videos such as close range surgery demonstrations, stereo-photogrammetry and digital rendering of oral cavities in authentic clinical cases.
A sneak peek of the MOOC: Materials in Oral Health!
TeLi colleague’s sharing of MOOC development.
A platform to Scale MOOC Learning Initiatives beyond the Region
In addition to experiencing the educational methodologies in the MOOC course, educators and teachers from visiting institutions were engaged with a practical session – learning how to deliver content using MOOCs and integrating trending technology in teaching. TeLi shared experiences in MOOC development and production with the teachers, and guided them through the course framework and various key components of the course. Among the topics discussed were pragmatic skills in storyboarding and video production, managing schedules and resource requirements. Many teachers were eager to ask questions and some also shared their own experience and challenges encountered in blended learning and online courses.
The sharing with mainland educators and students enabled TeLi and the Dentistry faculty to transfer our experience to future e-learning creators in professional Dentistry, and empowered the participants to pass on their new experience to more people in the mainland.
It is our hope to inspire more inter-institutional and interdisciplinary collaboration in teaching oral health care, and possibly e-learning pedagogical design and research.
Contact us if you are interested in creating knowledge exchange opportunities with us!
