What should a Teaching Excellence Framework look like?
Jo Johnson, the new Minister for Higher Education, surprised nobody last week by asserting his determination to deliver the Conservative Party’s manifesto commitment to “introduce a framework to recognise universities offering the highest teaching quality”. This will now be known as the Teaching Excellence Framework.
He did not go into a great deal of detail in his speech about what the framework would look like or what precisely the recognition would be for high quality teaching, although he did indicate during Q&A that he was open to considering financial incentives and rewards. In fact the level of openness and willingness to listen and respond to what the sector thinks was quite striking and positive. I think it is imperative that the sector engages positively with BIS over this, because I’m not sure it is an initiative where a strategy of kicking it into the long grass or hoping for death by a thousand cuts will work.
I think a key consideration in developing the TEF ought to be that it is light touch. I’m sure all universities that participated in the REF would want to avoid replicating the bureaucratic burden involved in that. The TEF should be based on metrics, ones that already exist or could be gathered with relative ease. Jo Johnson was also clear about this in his speech when he said “I have no intention of replicating the individual and institutional burdens of the REF. I am clear that any external review must be proportionate and light touch, not big, bossy and bureaucratic”.
Based on that principle, I think the data should fall into three themes:
- Peer judgement
By input, I mean the quality of the content and delivery of teaching and wider academic experience of students.
There are a number of data that could be used, or newly collected to inform this but I think we have one already in the bag and a couple of others that could be gathered quite easily if we try hard enough.
- Qualified Teacher Status: HESA has already been gathering data on this, I know it’s not a measure the whole sector has united behind yet, but with one more push and some finessing of the HEA’s Professional Standards Framework, we could get some sector-wide, comparable data on this. There will be plenty of wailing and gnashing teeth about this I am sure, but it is notable that this came top of student expectations when HEPI surveyed them recently, we should meet that expectation, it won’t do anyone any harm!
- Research Impact: I was trying to think of a way of quantifying the fact that students should have the opportunity at university to learn and understand the very boundaries of knowledge and understanding in their chosen subject and learn from those involved in defining and discovering the new boundaries of knowledge and understanding. The REF scores would tell some of this story, but it occurred to me that the impact scores are probably a more effective and targeted way of measuring the ability to communicate and engage students with world-leading research.
- Enrichment: it is widely accepted that a students’ academic experience is defined not only in the classroom, but also the co-curricular activities that are available. We are already engaged in an initiative to gather all this data to inform the Higher Education Achievement Record (HEAR). Be it work placements, study abroad, community engagement, leadership of sports clubs or academic representation, these activities improve student academic development and make a significant contribution to an excellent learning and teaching experience. It would take some work to finesse our systems to gather this data, but it can be achieved and should make a useful especially feature of a TEF.
By outputs, I mean demonstrable measures of students gaining knowledge and skills during their their studies. I think there could be three data sets that could be used here, two that we gather already and one that could be derived from existing data.
- Learning Gain: this term means many different things to different people. As a school governor, I have become familiar with the concept of ‘value added’ where a contextual measure of a students knowledge, skills and ability on entry and then looking at progression from that at various points up to a pupil leaving the school. I am not at all convinced that a measure that looks at UCAS tariff through to degree classification is the most effective measure of learning gain, mostly because of all the problems we know exist with the comparability of degree classifications. I think a measure should be developed that looks at the entry qualifications that students arrive with and then how far along they get with their higher education. That way, universities who recruit students with no or few qualifications could be rewarded and see the recognition increase as students achieve higher level HE qualifications. For example a student with no qualifications attending a university and achieving a Level 5 HND would have travelled the same distance as a student with the traditional A levels and then achieving a Level 6 BA (Hons).
- Employment data: we currently have the Destination of Leavers from Higher Education Survey that captures reliable data about what students are doing 6 months after graduation. There is talk of improving this to use HMRC data to show the real earnings of student at 1, 3, 5 and 10 years post graduation which would be a positive development I think. Either way, this is a key piece of data to show what students are capable of doing and achieving after graduation and should be included in any measure of teaching quality.
- Grades: so long as we have the old honours classification system the data is not comparable. But, if a concerted effort is made to move to the GPA system, we might achieve more comparability and be able to develop data that reflects and measures that.
3. Peer review
A purely data driven approach would not give sufficient space for input from peers. I include students as part of the peer community here, because I feel strongly that we should consider their feedback and views of the quality of teaching to be of equal, if not greater, value than that of peers because, above all else students are the only people who can tell us what it is like to be a student.
- The National Student Survey (extended to PGT): I’m pleased that the recent review of the NSS appears to have concluded that the survey is immensely valuable and not in need of a radical overhaul, but needs refreshing to sort out some of the dated and more nebulous questions (personal development, anyone?!). So this survey and its outcomes should undoubtedly form an important part of the TEF, I would argue that it should be heavily weighted in comparison to other metrics too.
- External Review: I am not advocating a return to subject review or inspection regimes of old, they ran out of steam over 15 years ago and everything we learnt about them then still stands (burden, bureaucracy, gaming the system, diminishing quality of the ‘inspectors/reviewers’). But I think a positive judgement from the university’s most recent QAA institutional review should be a pre-requisite for inclusion in the TEF, to demonstrate that core academic standards are being maintained and the teaching is built on solid foundations.
This is an attempt by me, from my perspective, to list what should be included in a Teaching Excellence Framework. Each of the individual sets of data listed above can (and I’m sure will) be picked off on its own and rubbished as an insufficient measure of teaching quality. This may be true, for example DLHE only tells you if students get jobs, it doesn’t tell you if they would have got them anyway or whether their degree actually helped them. BUT nested in with a number of other data sets (some that we gather already, some that we need to start gathering) it can start to piece together a rich picture of the quality of teaching and learning.
This approach of picking apart the minutiae has been very successful for the sector in the past, it may succeed again, but I am doubtful. We have an opening to engage positively and constructively with this initiative and to shape it; if we try to nit-pick and shower the whole thing in treacle, I suspect we’ll get the sort of TEF we deserve!