4. Monitoring and Review of Teaching and Learning Quality
4.1 Monitoring and review
4.1.1 Regular monitoring and evaluation of T&L are informed by internal and external input, including feedback from students, External Examiners’ input, curriculum reviews or professional accreditation, and input from employers, alumni, professional bodies and the industry. Strategic reviews of T&L are also conducted through formulation of Faculty Development Plans, internal academic reviews, and audits by the Quality Assurance Council (QAC).
4.2 Quality enhancement and improvement
4.2.1 The QA processes are articulated with appropriate QE processes, with the aim of enhancing student learning experiences and achieving student learning outcomes. Clear reporting mechanisms are designed to ensure that evidence of student learning experiences and achievement of student learning outcomes collected through the QA processes are acted upon through data analysis and interpretation, development of recommendations for improvement, and implementation of follow-up actions as planned. Details of these mechanisms are elucidated in the respective sections below.
4.2.2 Faculties and the Common Core Office are required to submit annual reports to the TLQC, upon taking a holistic review of the evaluation and feedback collected from key stakeholders through various channels. In addition, they should submit a triennial T&L strategy report in line with the UGC’s triennial Planning Exercise. These reports form part of the TLQC’s annual report to the Senate. The report templates are at Appendix 4.1.
4.2.3 The CCELE, the HKU Horizons Committee, the BASc Curriculum Committee, CEDARS, GHELC, and AAFYEC will each submit an annual report to the Senate, via TLQC, on QA/QE matters relating to Chinese and English language enhancement, the HKU Horizons programme, co-curricular activities and general education courses/activities, experiential learning activities, and academic advising and first year experience (FYE) respectively (see Appendix 2.2). The Common Core Curriculum Committee should also submit a triennial report on the CC Curriculum’s T&L strategies.
4.3 Institutional surveys of student learning experiences
4.3.1The Teaching and Learning Evaluation and Measurement Unit (T&LEMU) is a central unit that supports the University’s teaching and learning strategies, quality assurance and enhancement through various forms of activities as outlined below:
- collecting, analysing and reporting institutional data through institutional surveys for informing and supporting QA and QE;
- conducting evaluation and measurement studies with the goal of furthering the University’s teaching and learning strategies; and
- advancing the understanding of evaluation and measurement practices and processes through research in support of teaching and learning.
4.3.2 T&LEMU conducts institutional surveys on student learning experiences for Ug and TPg on a regular basis to provide information for analysis and renewal of curriculum and pedagogy. These analyses help ensure and enhance the quality of student learning experiences.
4.3.3 The survey findings are analysed and fed back to individual Faculties or programmes. The Vice-President and Pro-Vice-Chancellor (Teaching and Learning) visits each Faculty to discuss strategic T&L issues and developments, while representatives from CETL and T&LEMU discuss with FTLQC members of each Faculty other operational or general T&L issues arising from the findings for the purpose of curriculum and pedagogy monitoring and renewal. An action plan must be drawn up by the Faculties to address the issues identified, which will be incorporated into the Faculties’ annual reports to the TLQC (paragraph 4.2.2 refers).
4.3.4 All first-, second- and final-year Ug students are invited to complete the Student Learning Experience Questionnaire – Undergraduate (SLEQ-UG) annually. The aim of this survey is to collect data about Ug students’ perceptions on the achievement of their educational aims and their learning experience at the University, with a focus on their experience of the degree curriculum as a whole (Appendix 4.2).
4.3.5The Student Learning Experience Questionnaire – Taught Postgraduate (SLEQ-TPG) collects student perceptions of the curriculum and the learning outcomes resulting from the curriculum, containing questions that are specific to the TPg curricula to cater for the characteristics of TPg curricula (Appendix 4.3). The data are used to provide feedback to enhance the curriculum quality.
4.3.6 Once every two years, the University conducts a survey with graduates to understand their experiences during their full-time study of the Bachelor’s degree at the University. Normally, two different cohorts are invited in each round, viz. graduates of the previous year and those who are five years after graduation. The survey findings inform the University’s quality enhancement. Surveys are also conducted with graduates’ employers every three years to understand their overall impression of the University’s graduates on various attributes and their achievement of the University’s educational aims (Appendix 4.4).
4.3.7 Other surveys are administered from time to time on specific areas of student learning experiences to inform the University’s T&L planning.
4.3.8 The Institutional Survey Reports System serves as an online platform for Faculties to retrieve institutional survey results, for example, on student learning experiences. This System enables Faculties to retrieve survey results and generate reports at programme, curriculum, Faculty and institutional levels, as appropriate. Standard report templates are also available for downloading results for the scales and individual items. The user manual is at Appendix 4.5.
4.3.9 Faculties are encouraged to disseminate these results widely to students. They should discuss these findings at the Staff-Student Consultative Committee, and inform students the actions that have been/will be taken to address student concerns. Additional student forums may also be organised. The results and actions should also be posted online, for example, on Faculty website, for access by students.
4.4 Student feedback on teaching and learning
4.4.1 SFTL questionnaires provide a structured feedback mechanism for the University’s courses and teaching. There is a standard questionnaire for courses, and one specifically designed for practicums, internships, fieldwork, capstones and portfolios. In addition to core SFTL items approved by TLQC, Faculty-specific items may be included in the questionnaires if necessary. At the end of each course, students complete the relevant questionnaire, which covers both course and teacher evaluation. The Social Sciences Research Centre (SSRC) is responsible for the development and administration of a central SFTL system and for the analysis of results. The average course effectiveness in an academic year, and the response from the Course Coordinator, if any, are posted on SIS for access by students.
4.4.2 An operational guide for SFTL drawn up by SSRC is at Appendix 4.6. Some key features are outlined below:
- All Ug and TPg courses should normally collect feedback each time they are taught, following guidelines laid down by TLQC.
- Departments and other teaching centres must share the feedback on courses with students by disseminating and publishing course feedback summaries through Staff-Student Consultative Committees and other channels.
- Student feedback is one of the established components of the University’s QA&E system.
- Student feedback on teaching is treated by SSRC as confidential and returned only to teachers and their Head of Department or Centre. Overall teacher effectiveness scores of all courses taught by a teacher should be accessible online by his or her reviewer(s) in the Performance Review and Development (PRD) exercise.
- Feedback from students must be used to improve teaching. The teachers and Departments must take follow-up action with necessary improvements within a reasonable time after students have rated the courses and teaching.
- Students must be assured of the anonymity of results.
4.4.3 The operational guide includes a summary of the dissemination and accessibility of SFTL results as approved by the Senate.
4.5 External Examiners and international benchmarking
4.5.1 Internal Examiners and Chief Examiners are appointed by Faculty Boards to conduct examinations and discharge other associated duties (see Chapter 5). 9 In addition, the University of Hong Kong Ordinance and Statutes provide for the appointment of External Examiners.
4.5.2 The External Examiner system is a key QA and QE mechanism for benchmarking against international standards and practices. External Examiners are appointed to serve as academic advisors to the University’s degrees, certificates and diplomas, and to assist in maintaining the standards of such curricula at a level comparable with other major universities in the world. External Examiners are persons of seniority and with considerable and recent experience in University teaching. Many of them are internationally renowned experts. Additional external examiners from the profession may also be appointed. The policy and regulations governing the appointments of External Examiners and the related guidelines are at Appendix 4.7.
4.5.3 Appointments of External Examiners are approved by TLQC on the advice of the Faculty Boards. External Examiners are expected to comment and advise on the curriculum or programme design and content, pedagogy, assessment, overall performance of the students as compared against students’ academic performance at other international universities, areas requiring attention or improvement, and any other issues relating to T&L. External Examiners overseeing a curriculum with multiple programmes or specialisms are further expected to undertake a holistic review of the coherence of the different learning experiences of the students in the same curriculum.
4.5.4 An External Examiner is required to visit the University once during the period of appointment (except for non-degree curricula). At the end of each academic year, the External Examiner is required to write a general report for the Senate and send the report to the President and Vice-Chancellor in the first instance (Appendix 4.8). Any comments given by the External Examiner during his or her visit or in the External Examiner’s report are considered for response to the comments by the Department through the Chief Examiner and an explicit decision is made about whether or not to introduce consequential changes. The views of the External Examiner and the Chief Examiner’s response thereto on behalf of the Department are considered by FTLQC, the Board of Examiners (BoE) and any other relevant Faculty committees. The Secretary of FTLQC sends the Faculty’s feedback, including the Chief Examiner’s response thereto, and comments on or deliberations of the External Examiner’s report by relevant Faculty committees, to both the External Examiner and the Examinations Secretary. Taking into account the response from FTLQC and other relevant Faculty committees, the Examinations Secretary alerts the TLQC Chair to issues which have not yet been addressed by the Faculty or which require TLQC’s deliberations. The TLQC Chair decides whether the report is to be discussed by TLQC. The discussion/resolution of TLQC is forwarded to the FTLQC for action at Faculty level, as necessary. FTLQC is required to submit a progress report to TLQC normally within 6 months.
4.5.5 The flow of the External Examiners’ reports and the feedback loop described above is illustrated in Appendix 4.9.
4.6 Professional accreditation and external benchmarking4.6.1 In addition to the External Examiner system, some curricula are externally reviewed or accredited in accordance with statutory provisions. Some have attained professional accreditation or recognition by local or international bodies; and a few have benchmarked their curriculum content, pedagogy or assessment against established international standards and professional practice.
4.7 Faculty-level consultations with students and other stakeholders
4.7.1 Departmental or programme-based Staff-Student Consultative Committees are set up to promote communication between staff and students and discuss academic and curriculum related matters, including SFTL course results, for enhancement of T&L quality and student learning experiences. These Committees usually meet twice every academic year.
4.7.2 Faculties also supplement the Staff-Student Consultative Committees with other forms of engagement with students and other stakeholders with respect to T&L matters, e.g. Dean’s Forum, Faculty exit questionnaire, Dean and Associate Dean’s meetings with student societies and alumni associations, programme-based advisory boards to collect views and feedback from employers, professional societies, government officials and industry.
4.8 Programme Learning Outcome Achievement Scheme
4.8.1 With advice from CETL, the TLQC formulated a PLO Achievement Scheme for the evaluation of students’ achievement of PLOs of the programme on which they are enrolled. It comprises the development of a PLOAP for every Ug and TPg programme and the CC curriculum, except externally accredited programmes and the completion of a PLOAR at least every three years (Appendix 4.10).
4.8.2 The purpose of a PLOAP is to ensure that every academic programme can demonstrate the use of direct evidence of student learning for gauging students’ achievement of the PLOs of the programme. The PLOAP should be created shortly after the programme is implemented. The purpose of a PLOAR is to report on students’ achievement of the PLOs, and run on a 3-year cycle so that two PLOAR reports will be available for each curriculum review. All PLOAPs and PLOARs should be considered and endorsed by the relevant FTLQC/Common Core Curriculum Committee.
4.8.3 CETL provides advice and support to Faculties and the CC Office to help them understand the PLO Achievement Scheme, and the completion of PLOAP and PLOAR.
4.9 Curriculum reviews
4.9.1 There are established guidelines for review of Ug and TPg curricula (Appendix 4.11). Through self-reflection and peer review, the University identifies strengths which can be built upon and aspects that can be improved. The aim is to encourage evidence-based reflection and to foster a culture of continuous improvement.
4.9.2 Curriculum reviews are conducted in six-year cycles. TLQC undertakes reviews of Ug curricula while reviews of TPg curricula are conducted by the respective Faculties.
4.9.3 If a curriculum is already subject to external reviews such as accreditation, the external review exercise will be accepted as meeting University’s curriculum review requirement provided that it covers all aspects of curriculum review required by the University. Should there be any required aspects not covered, these can be included in the framework for the external review exercise, or supplemented by an internal review.
4.9.4 In conducting the curriculum reviews, a self-evaluation document is prepared by the curriculum team to address the University’s strategic themes of 3+1 Is as well as each of the six elements of T&L: aims, learning outcomes, curriculum design, content, pedagogy and assessment. The conclusion should be an action plan to address these areas with a timeframe. The review panel is interested to see whether the curriculum team is capable of using evidence to improve the curriculum. It reports on identified strengths and recommendations. The curriculum team is expected to respond to the report with a revised action plan to address the areas of strength and improvement identified by the review panel. The action plan is expected to make specific responses which address in turn each of the affirmations, recommendations, and areas for consideration, if applicable, of the review panel. The action steps should include a timeline. Implementation of the revised action plan is monitored through triennial progress reports. These should detail progress on each action item. For monitoring purposes, the review reports and the relevant curriculum team responses, endorsed by the relevant Faculty Boards, should be submitted to TLQC for endorsement. TLQC is the ultimate QA and QE body and requires Faculties to report progress (or lack of progress) made in action plans.
4.9.5 In addition to curriculum reviews, focused reviews are undertaken for common learning experiences across all Ug curricula. These reviews aim to check whether there are any mismatches between the intended curriculum and the enacted curriculum, and the modifications that need to be made to address the gaps. These reviews necessarily vary somewhat in the review procedures as well as in scale, and are centrally coordinated as they cut across all Ug curricula.
4.10 Faculty development plans
4.10.1 Each Faculty is asked to make a Faculty development plan submission to the Budget and Resources Committee annually updating its strategic priorities in the coming year before the Committee allocates the Faculty’s recurrent budget. Together with a set of key performance indicators, the submissions also enable monitoring of performance and deliverables. In formulating the plans, Faculties are advised to make reference to the University’s strategic development document. As T&L is one of the key areas to be addressed in the submissions, Faculties are able to take the opportunity to present their plans to introduce new programmes or renew existing ones, report on the implementation of T&L and set out action plans in response to institutional survey results.
4.11 Academic reviews of Faculties and other institutes/schools/centres/units
4.11.1 The University conducts comprehensive academic reviews of Faculties and other institutes, schools, centres and units, normally in five-year cycles. These reviews evaluate the mission and goals of Faculties and other units in the context of the University’s vision and mission, their fitness for purpose and success in accomplishing their stated mission and goals. Self-evaluations are submitted against performance indicators, including the ones pertinent to the delivery, QA and QE of T&L (Appendix 4.12).
4.11.2 A review panel is formed in respect of each academic review. The panel comprises members internal and external to the University and is approved by AB via AGAR. Each academic review culminates in the review panel’s report which is submitted along with comments from the relevant Faculty, institute, school, centre or unit and AB to the Senate and the Council for endorsement. AGAR is charged with the conduct of these reviews, and monitoring the implementation of the approved recommendations through the Provost and Deputy Vice-Chancellor with the support of the relevant Vice-President and Pro-Vice-Chancellor(s). AB receives reports on the monitoring of the approved recommendations from the Provost and Deputy Vice-Chancellor via AGAR. Flowcharts showing the review processes are at Appendix 4.13.