So, this is going to fun!
I have vivid memories of the last select committee inquiry and report into HE – Students and Universities. I think it’s fair to say that it wasn’t an awful lot of fun for the sector.
So, this is going to fun!
I have vivid memories of the last select committee inquiry and report into HE – Students and Universities. I think it’s fair to say that it wasn’t an awful lot of fun for the sector.
I was tweeted this photograph a couple of days ago by Christina Leston-Bandeira a Professor in the Politics Department at the University of Hull. It’s of an article that was written about Hull University Union when I was student President in 2004/5.
A few weeks ago, I wrote about a polemic, dressed up as research, that had been published arguing that the partnership model of student engagement was fatally undermined by the fact that students’ unions have commercial activity and provide services to their student members (and for that matter, other members of the university community).
If you look at the caption to one side of my photograph, you can see that I gave the game away over 10 years ago:
Independent thinker: Derfel Owen, the union president says that generating its own income has been the answer
I’m clearly a neo-liberal headbanger after all, hell bent on destroying the very fabric of higher education!
I think the point I was trying to make during the interview is that the successful commercial activities of the students’ union had made us a more engaged and visible contributor university life and to the local community, also that those commercial activities had grown our income streams and helped us to invest in representation, volunteering and social activities for students.
I just read this great, succinct piece by Philip Plowden on the Guardian HE pages – How to approach the Teaching Excellence Framework with confidence.
It’s very balanced and sensible. I particularly liked recommendations 1, 4 and 5:
1) Stay Passionate about your subject area
Too often a commitment to learning and teaching is presented in opposition to engagement with research and scholarship, but the two should be inextricably linked. Universities are above all places where knowledge and practice are questioned, and our understandings extended. Without that continuing commitment to the discipline, any teaching can become rote, and the value to students is lost.
The emphasis on the link between research and scholarship and teaching is spot on, universities are meant to be places where students can and should learn about the boundaries of knowledge and understanding in their chosen subject; sharing that knowledge and working together to push the boundaries of understanding is what is should all be about.. Most important is the need for students to feel a sense of passion, commitment and excitement from their lecturers, this can come from linking teaching to their own research but not necessarily, it’s about having a sense that the person in front of you teaching you cares just as much (possibly even more) than you about what you are studying.
4) Talk with the students
“Co-production” has become something of a buzzword, and there is increasing challenge, including from students, to the idea that it is in part their responsibility to shape the learning experience that they are now funding. But if nothing else, it reminds us that students are best placed to comment on how a programme or a module feels to them. And that gives us our first intelligence as to whether we are achieving what we intended.
I’ve written a book about student engagement (get yourself a copy on Amazon!) so get very excited by concepts such as co-production, change agents, partnership etc. However, I have always been of the view that when boiled down to what can and should every member of staff at a university be doing, it’s this, talking to your students. They are the only people who know what it is like to study on your module or programme, taking time to talk to them and listen to them will be invaluable. This can be through formal structures such as student-staff committees or feedback forms, or less formal opportunities at the end of a lecture or around assessment hand-in times.
5) Get assessment right
When we get assessment right, it becomes a seamless part of the learning process. There can be an unspoken anxiety that unless assessment is burdensome – for student and academic alike – it cannot have been rigorous. Perhaps this is why the assessment process often remains the most traditional element in any programme and module. If you can get the assessment right, the rest of the module will follow.
This is what it all comes down to, this is the biggest burden and stress factor for every student and academic. There are so many shibboleths and myths about assessment that lead to over-assessment but also over-burdensome methods of assessment for students and staff.
The other recommendations are good too.
I was really downhearted when I read this article published by the Guardian on its Higher Education pages.
The author argues that the National Student Survey is a waste of time and is detrimental to learning and teaching.
Personally, I think it is a cause for celebration that students report such high levels of satisfaction and positive experiences. In particular this year when the skies were meant to fall down upon us because the first generation of £9k fee payers were filling in the survey.
The NSS is a survey that draws in hundreds of thousands of respondents each year; just in order for the results to be published a response rate of over 50% must be achieved (and this is massively exceeded every year), this provides reassurance that the data is robust and is genuinely representative of student views about their experience.
To complain that the scores are clustering misses the point entirely and ignores the facts. Would we prefer it if more students were having an utterly miserable experience for the sake of having a wider range of scores? I doubt it.
Then there is the argument that a single survey is not appropriate because universities and the courses they provide are different. Of course they are! Universities are different and the courses that are delivered will vary in content, teaching and assessment methods; the survey does not ask students to compare their experience with students at other universities and it would be absurd to do so because students (by and large) only have one experience to reflect on. The NSS, rightly, asks students a series of pointed questions about their experience of teaching, assessment, support, resources etc. at their institution, the questions are identical for all students in their final year so the focus is on comparability of outcome not input. It is perfectly possible for two courses in the same subject to be taught and assessed in completely different ways but for students to come out at the other end highly satisfied with their experience. It would be disastrous to try to bring about uniformity of input.
Another point to consider here is the ‘masking effect’ of the mean scores for institutions. These scores that aggregate the level of satisfaction of all students covers up the fact that there will be significant variation between departments and even within departments between courses. This in itself is not a problem, but it does counter the suggestion that all this is pointless because there isn’t a wider gulf between the highest and lowest scoring universities. This also gives an opportunity for departments to benchmark themselves against colleagues in university and also peer departments in other universities.
The suggestions in the article of changes to the curriculum that have been made because of the NSS seem very odd to me. If methods of assessment are being changed to move away from over-reliance on essays and exams, then I can only cheer it along as a good thing in itself. These traditional forms of assessment are tried and tested and have very important part to play in universities. But why not add other methods of assessment into the mix? Ones that challenge students to develop their thinking and negotiate a final piece of work with a group of other students, ones that encourage and support them to present their research and ideas in new and innovative ways. I’ve just spent the Summer at UCL reading through hundreds of external examiner reports. They are all experienced, senior academics from other universities, and their feedback is routinely positive about new and innovative methods of assessment as a way of maintaining high standards and challenging students in new and interesting ways; far from discouraging new methods of assessment, they actively encourage, support and promote it and these are the opinions of other academics, not students.
Another criticism that is often made (not though in this Guardian article) is that the questions only ask about satisfaction and do not ask student to reflect on their teaching, learning and assessment experiences. I disagree. To take four questions as examples:
Who would not want to know the answer to these questions? If, for example, only 30% of students agree that the course is intellectually stimulating I think it’s very important to know that and to do something about it! Equally if, 70% of student agree with the statement “Feedback on my work has helped me clarify things I did not understand”, that’s a good score, but still one third of students are saying that feedback is not helping them and that’s a worry.
No mass survey, containing only 23 questions is going to be perfect. But I for one welcome this annual opportunity for students, en-masse, to take a moment to reflect on their entire experience and tell us what it’s been like, it is an important feature of UK higher education and a critical part of the relationship we have with our students.
UPDATE: Andrew McRae has ‘fisked’ the Guardian piece with magisterial style on his blog .
I attended an excellent workshop run by the University Alliance last week, officials from BIS and HEFCE addressed an audience about progress developing the Teaching Excellence Framework and the ‘Quality Assessment’ regime. My impression that BIS are genuinely determined to build a framework that the sector can live with (I’m not sure we’ll ever reach a situation where we’re happy with it!) was reinforced.
The event organisers challenged us to think hard about the purpose of the TEF, what we want to measure and what metrics we might want to use. Unsurprisingly, at this early stage, there was a wide range of views and opinions. I’ve already laid my cards on the table in an earlier blog post about what should be measured and I’m not sure I’ve heard anything to convince me otherwise yet.
A few reflections:
My former colleague and Comrade Professor Andrew McRae, Head of the Department of English at the University of Exeter, also posted his thoughts about the TEF on his blog on Monday. Characteristically sensible and challenging it is great to read an academic arguing that this is an opportunity not to be missed.
What should a Teaching Excellence Framework look like?
Jo Johnson, the new Minister for Higher Education, surprised nobody last week by asserting his determination to deliver the Conservative Party’s manifesto commitment to “introduce a framework to recognise universities offering the highest teaching quality”. This will now be known as the Teaching Excellence Framework.
He did not go into a great deal of detail in his speech about what the framework would look like or what precisely the recognition would be for high quality teaching, although he did indicate during Q&A that he was open to considering financial incentives and rewards. In fact the level of openness and willingness to listen and respond to what the sector thinks was quite striking and positive. I think it is imperative that the sector engages positively with BIS over this, because I’m not sure it is an initiative where a strategy of kicking it into the long grass or hoping for death by a thousand cuts will work.
I think a key consideration in developing the TEF ought to be that it is light touch. I’m sure all universities that participated in the REF would want to avoid replicating the bureaucratic burden involved in that. The TEF should be based on metrics, ones that already exist or could be gathered with relative ease. Jo Johnson was also clear about this in his speech when he said “I have no intention of replicating the individual and institutional burdens of the REF. I am clear that any external review must be proportionate and light touch, not big, bossy and bureaucratic”.
Based on that principle, I think the data should fall into three themes:
By input, I mean the quality of the content and delivery of teaching and wider academic experience of students.
There are a number of data that could be used, or newly collected to inform this but I think we have one already in the bag and a couple of others that could be gathered quite easily if we try hard enough.
By outputs, I mean demonstrable measures of students gaining knowledge and skills during their their studies. I think there could be three data sets that could be used here, two that we gather already and one that could be derived from existing data.
3. Peer review
A purely data driven approach would not give sufficient space for input from peers. I include students as part of the peer community here, because I feel strongly that we should consider their feedback and views of the quality of teaching to be of equal, if not greater, value than that of peers because, above all else students are the only people who can tell us what it is like to be a student.
This is an attempt by me, from my perspective, to list what should be included in a Teaching Excellence Framework. Each of the individual sets of data listed above can (and I’m sure will) be picked off on its own and rubbished as an insufficient measure of teaching quality. This may be true, for example DLHE only tells you if students get jobs, it doesn’t tell you if they would have got them anyway or whether their degree actually helped them. BUT nested in with a number of other data sets (some that we gather already, some that we need to start gathering) it can start to piece together a rich picture of the quality of teaching and learning.
This approach of picking apart the minutiae has been very successful for the sector in the past, it may succeed again, but I am doubtful. We have an opening to engage positively and constructively with this initiative and to shape it; if we try to nit-pick and shower the whole thing in treacle, I suspect we’ll get the sort of TEF we deserve!
Jo Johnson has made his second major speech since being appointed Minister for Higher Education today. Contents can be found here.
It is a substantial speech, I wasn’t there but judging by the tweets about the Q&A afterwards, he has really got to know his brief and means business.
The key part of the speech starts like this:
It is striking that while we have a set of measures to reward high quality research, backed by substantial funding (the Research Excellence Framework), there is nothing equivalent to drive up standards in teaching.
That is why my priority as Universities Minister will be to make sure students get the teaching they deserve and employers get graduates with the skills they need by introducing the Teaching Excellence Framework we promised in our manifesto.
My aims for the TEF are:
- To ensure all students receive an excellent teaching experience that encourages original thinking, drives up engagement and prepares them for the world of work.
- To build a culture where teaching has equal status with research, with great teachers enjoying the same professional recognition and opportunities for career and pay progression as great researchers.
- To stimulate a diverse HE market and provide students with the information they need to judge teaching quality – in the same way they can already compare a faculty’s research rating.
- To recognise those institutions that do the most to welcome students from a range of backgrounds and support their retention and progression to further study or a graduate job.
He does make it clear that the TEF will not replicate the bureaucratic burden of the REF, which is welcome.
In response to questions, he did make it clear that there would be incentives for teaching excellence. These will be consulted upon, but it is as clear an indication as any that this will somehow be linked to funding and/or student fee levels. I look forward to the consultation.
There’s a lot more in the speech too about access, standards, graduate prospects. All very interesting and balanced, I’ll try to post something later about these.
As a sideline, I mentioned in a post I wrote on WonkHE earlier in the week that HEFCE’s timing of its consultation on the future of QA was curious given that BIS was going to be consulting on the TEF, which is a much more substantial and significant change for the sector. This just reinforced that.
I’ve posted a piece on WonkHE this morning about HEFCE’s proposals for the future of quality assessment in England. I think it’s a mixed bag of good ideas, well meaning but flawed proposals, some contradictions and some very bad ideas. Overall, I think it has failed to produce proposals that would reduce the burden or cost of QA to universities, it merely shuffles the burden around.
One thing that I didn’t have space to comment on in that piece was the importance of student engagement. A great deal has been achieved on this over the past 10 years, not only in sharpening our focus on the student experience and student expectations but also involving students as active participants in quality assurance and enhancement. This is not very prominent in HEFCE’s proposals and could be one of those valuable features that gets thrown out with the bathwater.
An interesting piece in today’s THE “Students’ unions face uphill battle in anti-consumerism fight”. It is covering an academic article that appears in this month’s edition of the British Journal of Sociology of Education, “Students’ unions, consumerism and the neo-liberal university”. It’s an interesting contribution, but one that I think it seriously flawed.
In the THE, one of the article’s authors is quoted saying “The irony for students’ unions is that one of the main means of retaining independence and being able to resist consumerist agendas, is by embracing commercialism and providing services to paying students”. What utter nonsense!
The first assumption that is made in the article is that consumerism or market-like behaviour by students is something new, a phenomenon that started in 2012, or perhaps the early 1990s. In fact many of the behaviours that are citied as all-of-a-sudden prevalent among students “see a degree as a private investment (rather than a public good); be prepared to accumulate significant debt in order to acquire it; and actively ‘shop around’, comparing institutions and courses to secure the ‘best’ possible education” can be identified in students as far back as 600BC in Ancient Greece (I talk about this and many other historical links through the centuries in one of the chapters in the book I co-edited that’s still available in all good book shops i.e. Amazon!). So this is nothing new, and needs to be accepted by the “we’re all going to hell in a handcart, HE doom” brigade if we are ever to build a meaningful and rich understanding of the very complicated relationship students have with their places of learning.
I feel the authors of this piece have made the error of confusing the multi-faceted and high complex relationship universities and unions have with an idealised, single communitarian model. The existing system is critiqued at length and often badged “neo-liberal”, but no alternative is articulated. The truth is that Universities, and for that matter Students’ Unions are very complex organisations and the relationship with students is equally complex.
At its core, a Students’ Union is established to advocate the views and rights of students, this is certainly best achieved through a spirit of partnership with the University. But unions have also developed sophisticated and valuable structures in addition to this, such as organising volunteering opportunities, job shops, sports and societies and commercial services that reflect the wider needs of students and the university community in which they exist. This article seems to assume that this activity is an unnecessary distraction from partnership. I don’t think it is! Students, while at university want a rich and varied set of experiences and it is perfectly acceptable and reasonable for a students’ union to play a role in providing them. If students want to eat Mars bars, then why on earth should a students union not help them to get one?!
I think also, the mistake is made of assuming that consumerism and partnership are mutually exclusive and totally incompatible. I don’t agree with this. If you look at the student journey through education, it is reasonable to conclude that they might make some decision in very similar ways, irrespective of whether it is a consumer or partnership model on offer. I’ve often shown the following table as an illustration of this when giving presentations on the matter:
|Choice of a range of providers||X||X|
|Choice between a range of courses||X||X|
|Make informed choices||X||X|
|Free to chose course that suits you||X||X|
|You can provide feedback||X||X|
|You can suggest improvements||X||X|
|Provider is the expert||X|
|You are responsible for your education||X|
|If it breaks, you can take it back||X|
The key difference, as is highlighted at the bottom of the table is the level of responsibility students have for their own education. If students do not actively engage with the learning opportunities available to them at University, then they will gain nothing and will not be to go back and request a replacement of themselves after three years of study!
An interesting post on WonkHE yesterday about the future of Quality Assessment.
There are clearly some interesting proposals brewing, and it is quite refreshing to know that sector bodies are willing to be creative and go back to core principles. The most important principle when it comes to the regulation of UK Higher Education, in my opinion, is that of institutional autonomy. The reason UK and U.S. Universities top every global league table going is because our universities, unlike the vast majority of other sectors, have had the freedom to develop and grow free from direct government intervention.
HEFCE is a government body, it’s is established by statute and reports directly to BIS. I was a little concerned therefore when it unilaterally announced, some months ago, that it was going to review ‘Quality Assessment’ in England. I posted about it on my blog.
The statute in the 1992 Further and Higher Education Act under which HEFCE gains authority for quality assessment is as follows:
(a)secure that provision is made for assessing the quality of education provided in institutions for whose activities they provide, or are considering providing, financial support F1. . .,
It is a little ambiguous, I think, but as I wrote back in October, it limits HEFCE to making sure that quality is assessed, it does not imply that HEFCE should be the assessor! A subtle but essential difference. The ambiguity does place a heavy responsibility on HEFCE and a big risk on the sector though; arguably HEFCE could extend it’s reach and interpret this statute in away that allows it to regulate and intervene directly in the management of quality in Universities.
I sincerely hope it doesn’t do that.
Some of the proposals that have been floated could see HEFCE, perhaps unintentionally, cross that line. Annual reviews of data will undermine existing internal processes and give HEFCE the levers to intervene in institutional prioritisation and management decisions. I’m not sure how dismantling external review by QAA and replacing it with a (presumably) ‘non-peer’ review as part of HEFCE’s five-yearly review of governance is going to lighten (or for that matter enlighten!) the system.
One final concern is the unity of the UK system. The existing QA system, just about, holds the UK together with English, Welsh, Scottish and Northern Ireland universities buying in to a broadly common system. Scotland walked away from the review at the outset, I’m not sure whether Wales and Northern Ireland have stayed the course. It would be bad news indeed if the brand of UK Higher Education were to be undermined.
Lots of detail and matters of significant principle to get stuck into.