Friday, October 16, 2009

The currency of knowledge: or, the PBRF in ruins. Part 1

I’ll begin with a story of practical details. The micro-politics of the PBRF reached a new level in mid-2009. After the completion of an internal mock-PBRF assessment at one of our esteemed universities, academic staff whose research output over the previous three years was considered to be well below a level likely to gain a B rating received a letter from the Dean raising concerns about their individual research productivity. The matter was framed in terms of the employee’s contractual obligations and the criteria for the PBRF. Recipients of this letter were summoned to a meeting with their Head of School, at which a member of the Human Resources staff would be present. At this meeting, research objectives would be reviewed and performance targets would be set. If satisfactory progress were not made within three months of that meeting, the letter warned, a performance management process would be initiated. Essentially, this was a written warning about poor performance, following which disciplinary proceedings could eventually be implemented. Under such conditions, dismissal is always a possible outcome if performance does not improve as required, although this was not explicitly alluded to in the letter itself.

Now, one could argue about the lawfulness of this action by the employer. But that is not my business today. In any case, mediation between the employer and the union over the letter has been undertaken and that seems to have produced some change. As if intending to make the PBRF look even worse, however, another, equally-esteemed university shortly afterwards announced that it would ‘fine’ colleges, to the tune of $40,000 per year, if their projections under-estimate the numbers of R-rated staff in the 2012 assessment. Heads of departments were, I believe, asked not only to estimate numbers, but also to provide lists of names.

But let’s suppose now that every university in New Zealand had written a warning letter to every PBRF-eligible staff member who had scored an R in 2006. Let’s omit those with an R(NE) who had the excuse of being ‘new and emerging’ researchers at that time. If we include the colleges of education, which either had already merged, or were about to merge, with universities in 2006, the total number of staff under the gun would be about 1,000 (Tertiary Education Commission 2007). That’s about 16 per cent of the academic workforce of universities, who were eligible for PBRF assessment, at that time. And that’s not allowing for the possibility that some who scored C or R(NE) could also come under scrutiny at some stage.

Imagine issuing poor-performance warnings to 16 per cent of the academic workforce, telling them to produce research within three months or face further disciplinary actions. Dismissal is not ruled out.

Taking the letter I have just cited to its logical conclusion, that’s what could have happened. But if all academics were expected to get a B rating, as the letter suggests, then the proportion of us under such managerial disciplinary surveillance and control would rise to over one half. That’s conceivably more than half of the universities’ academic staff at risk of disciplinary proceedings, and even dismissal, for the alleged failure to produce enough research at a presupposed level of quality.

Now, perhaps this is taking the matter to Orwellian, and hence fictional, levels. And so, before I am (again) accused of indulging in a supposed ‘myth of the despotic regime of production’ (Curtis 2007), let me make a few observations about the complex relationship between members of the Academy and the PBRF. First, the PBRF has been designed, developed and applied, if not misapplied, by academics – by us and by our own colleagues. If we think of the PBRF as a virtual intellectual Panopticon, as a set of constraints and a method of surveillance that produce the internalisation of a moral code, then it is a Panopticon that was designed, built and even guarded by its own inmates.

And let’s not overlook the indignant and the swollen egos, the anxiety and the sheer vanity, produced by those A-to-R grades. While some have argued that the PBRF gave them recognition and validation as researchers that their own managers could never find it in themselves to give, others have railed against unfair assessments, bias among panels, or a failure to recognise the activities that they happen to cherish the most.

Did we really have to take it so seriously? After all, the PBRF is basically a mechanism for allocating some public funds around the tertiary education sector. It’s not the gaze of the Master – though sometimes one would think it were, given the importance that some attribute to its results. The desire for a kind of validation given by such an external judgement – even though it has limited validity – allows it to eclipse, for many of us, the intrinsic estimation of our own work. The exaggerated meanings attaching to PBRF results, and how they are thought to reflect upon the reputations and performance of individuals and academic units, are the product, I suggest, of an excess of signification. And so to strip the PBRF of some of this excess, we need to look again at its intended effects.

Now, as the aforementioned letter does quite correctly point out, the PBRF income to the university is intended to recognise and support the allocation of time to research, and also to research-based teaching, especially at post-graduate level. The freedom that we enjoy as academics to devote ourselves to inquiry and experimentation in fields of our own choosing – and to support advanced students in the same – is reflected in this fund. The PBRF designers intended it to respect institutional autonomy and not to interfere with academic freedom. But no-one ever cogently argued, in the conception of the PBRF, just why an external instrument of fiscal policy should be permitted to influence the priorities and the allocation of time and resources within the universities. Nonetheless, we let the Clark government get away with it. While it is widely thought that New Zealand does not invest enough resources in research, no-one had come up with any analysis, to my knowledge, that showed that universities were doing less than they ought. And certainly no-one has stated what the optimal level of research output – at any specified level of ‘quality’ – should be for any particular university. The PBRF has never come with a clear set of goalposts, to put it simply. Taken to its logical conclusion, the optimal arrangement for PBRF purposes would be an academic staff contingent every member of which gets an A. Not only would this be practically impossible, but any attempt to achieve it would jeopardise the development of the future generation of researchers and scholars – not to mention devotion to teaching.

The message of the PBRF seems simply to be that more – or at least the appearance of more – ‘high-quality’ research is desirable. Growth is good. The sky could be the limit, were it not for the fact that another effect of the PBRF is that the universities are now competing madly with one another to retain much the same real level of funding that they got in previous years. As Adams pointed out, in his independent evaluation, the PBRF is creating a Red Queen race – where the contestants madly scramble to stay in the same place – due to little real increment in the overall fund. But compete they must, so as not to lose their share of the money (Adams 2008: 77).

Hence the pressure reaches down to the individual staff members, who are the ‘units of assessment’ and whose individual scores are aggregated, using a published formula, to create the final figure awarded to each institution.

Now, the statutory basis of our freedom to undertake research and critical inquiry is to be found in Section 161 of the Education Act 1989. There are three features of the wording of that section that deserve to be highlighted.

First, the Act provides clearly for the freedom of academic staff and students of the university, as an autonomous institution, ‘to engage in research’. But, secondly, this is a freedom neither to do nothing nor to evade accountability. Subsection 3 speaks clearly of ‘the highest ethical standards’, of ‘public scrutiny’, and of the need to maintain accountability for the use of resources. That is perfectly reasonable, and we do expect one another to maintain such standards. Indeed, thirdly, subsection 4 requires that, among others, the Councils and Vice-Chancellors ‘shall act in all respects so as to give effect to the intention of Parliament as expressed in this section’. That is, academic freedom is described in the Act as a freedom to undertake research and critical inquiry, provided certain standards are met; and the governors and managers of the university are bound by this, and must (at least) respect this freedom at all times.

So, there is a dilemma here in the expression of our values as academics. On one hand, especially in the humanities, we ideally came to this institution in order to pursue the highest goals as researchers and scholars, and this informs and supports our teaching as well. And so those goals are intrinsic to our careers. Some, especially those from former colleges of education that have been taken over by universities, may feel they have not had such a choice, I admit. But the freedom to engage in research and the quality of the university as ‘critic and conscience’ are clearly defined in the Education Act, as well as in long-standing traditions of the university. And we largely accept those conditions willingly. I cannot defend those who may accept the privilege of academic freedom but who will not use it productively. There have always been such individuals, and there have always been lawful processes for dealing with poor performance by an employee.

What I find troubling about managerial interventions such as that referred to at the beginning of this talk, on the other hand, is that the PBRF creates an environment wherein an external governmental fiscal instrument becomes the rationale for an application of managerial authority, including a disciplinary threat, imposed upon academic staff in order to produce more research, so as to produce more income for the university. This appears to contravene the Education Act’s requirement that the governance and management of the university should give effect at all times to the intent of the academic freedom provisions. Managerial coercion does not give effect to academic freedom, in short. So, how did we get to this? Have the government and the universities been taken in by Rousseau’s idea that some people must be forced to be free? Or is it the exercise of sheer power that satisfies them?

In spite of some critics, I would not put this down simply to some kind of insidious force that one might conveniently label ‘managerialism’ or ‘economic rationalism’ or anything vague like that. The zealous (if not over-zealous) application of the PBRF, and its extension into many aspects of the life of the university and the consciousness of academics, are effects for which we ought to accept some ownership, if we are to be honest with ourselves, and not to project them onto some ill-defined ‘other’.

The PBRF was designed by academics, the TEC consults academics about any redesign issues, and the implementation processes within universities are managed largely by senior academics. These are our colleagues, and, if we are serious about maintaining at least a semblance of an academic community and of collegiality, then we need to accept that the PBRF belongs to us, and that we produce its effects. Furthermore, of those among us who have actually been assessed by the PBRF, how many have not felt some kind of keen anticipation, or apprehension, about the results? Did you feel like a student waiting for final grades? I’m sure there were some – a few – who were completely and sublimely unconcerned. But my point is that, for better or worse, the PBRF and its results clearly mean a lot to most academics – not just intellectually, but emotionally too. It got under our skin. And we need to take ownership of it now that it’s there.

The PBRF has become a source of division and aggravation within the academic community. It has divided opinion between members of different universities and different disciplines, between those who gain and those who don’t, and between academic leaders and their more junior staff; and it has had the unfortunate effect of becoming the source of employment relations problems. It may not be long before the PBRF is the cause of litigation.

Further, the PBRF has encouraged the inversion of the very values that have supported our culture of research and scholarship. In 2003, I was quite well disposed towards the PBRF, but my views rapidly began to change when I heard colleagues say things like: ‘If I publish this there, it will look good on my PBRF portfolio.’ A fairly innocuous statement, but, to me, it implied that the funding of universities had become the reason, rather than the support, for what we do. This thinking becomes an institutionalised constraint when universities coerce staff into doing more research on the rationale that the university must maximize its income. Research has thus become a way of augmenting the institution’s income, more than income being deployed to promote research. Research ought to be valued for its advancement of human knowledge and its social and economic benefits, but the cart has been placed before the horse, I fear.

I also know that there are bright people who are early in their careers and considering a permanent role in the university, and older academics at the point of considering retirement, for whom the PBRF has become one of the main disincentives for remaining in the Academy. That potentially translates into real losses of talent, but this will never be quantifiable.

For many academics, including many very talented researchers and scholars, the PBRF is the most detested aspect of the new ‘university in ruins’ (Readings 1996), this university shaped by auditing and by performativity. For others, the PBRF may, simply and brutally, be the reason why they get the sack.

While the universities need the funding, the assessment system that is used to allocate that funding has become, in my opinion, part of the problem. The present assessment system is fundamentally flawed and I’ll briefly list what I believe those flaws are:

• Undue pressure to produce results assessable in the PBRF discourages the next generation of researchers and scholars who are currently contemplating academic careers. People busy doing doctoral or post-doctoral research are told by their seniors that they are ‘research-inactive’. That can be very demoralizing.

• The PBRF discourages the application of existing knowledge to social and economic problems, including knowledge dissemination in the classroom and beyond. Who, for example, is keen to write a textbook now that the PBRF is in force?

• There has been manipulation by institutions of eligibility criteria, resulting in interference in employment agreements.

• The assessment process is costly in terms of people’s time, and it creates a lot of administrative costs for institutions and for the TEC.

Imagine, furthermore, if I were to plan a research project that would involve evaluating an organization by collecting evidence from all of its employees about their personal performance. That evidence is then to be evaluated and each employee given a rating, and the score for every employee is then to be passed on to the employer to use for internal purposes, including performance management, and also to be passed on to the individual employee. The latter party will have no right to challenge the personal score, even if there is good reason to believe it does not accurately reflect performance. And if any employee refuses to take part in the evaluation, he or she may face disciplinary proceedings, so full participation is guaranteed. Such a research proposal would be treated as an outrage and immediately thrown out by any research ethics committee. And yet, the TEC and university managers consider that they have the right and the need to ride roughshod over those very ethical standards that we, as researchers, are strictly required, for good reason, to abide by. The freedom to engage in research has a statutory basis in the Education Act, and the Act also requires the academic community to abide by ‘the highest ethical standards’ at all times while exercising that freedom. When we, the researchers, are evaluated personally for the quality of our research, why is it that the TEC and our own academic leaders regard themselves as free to abandon such ethical standards? I struggle to find any justification for this. Confronted with this ethical problem, the only reply I have heard from university managers is to resort to the imperative of ‘employment obligations’ – again, an appeal to sheer coercion, as distinct from ethical, consensual conduct.

I am afraid, though, that the likely alternative assessment model – using bibliometric data – could be a whole lot worse, depending on how it is designed. People are already cross-validating PBRF results with publications and citations data from Thomson Reuters (Smart and Weusten 2007; Smart 2009) and even from Google Scholar (Smith 2008). I do not recommend that we condone any such research-quality assessment, as the shortcomings of these methods are well documented (Evidence Ltd 2007). All such governmental assessments are likely to be corrosive of academic freedom, and we should comment critically upon them, as we see fit. If I sensed that there were sufficient disaffection and courage among academics to do so, I would welcome direct resistance to leave the PBRF in ruins. After all, the whole system stands or falls because of us. If we really wanted to end it, we would have by now. But I do notice that many of my colleagues are committed strongly to the PBRF; and even many of those who complain about it are, I suspect, paradoxically in thrall to it.

The really critical issue currently in the redesign of the PBRF for 2012 concerns the reporting of individuals’ scores to the employers. In my opinion, this is in breach of the Privacy Act, as, given the past practice, the employer makes no effort to ensure that each researcher has given informed consent to the disclosure of personal information; and this is the issue that could leave the PBRF in ruins, if it’s not resolved. It is the reporting of individual results directly to universities that makes the system useful, as a management tool, from the universities’ point of view – but it is also the means by which the assessment system can be abused by the universities as employers.

The reporting of individual results, without informed consent and without effective rights of review and appeal, has brought the PBRF into disrepute with the very researchers upon whom the assessment depends. A way of resolving this problem for 2012 will hopefully be reached soon. If it were not for the mismatch between the unit of assessment (the individual) and the unit rewarded (the university), however, the system might not be in such disrepute as it is now.

The currency of knowledge: or, the PBRF in ruins. Part 2

So that was the pragmatist’s tale. Now let me stand back from that for a moment. The metaphor of ‘the currency of knowledge’ in my title today stands for two things: for the demand to be ‘up to date’ – to know what is ‘current’ – and also for the monetary unit of value. To call money ‘currency’ is in itself a metaphor. The currency that flows from the sovereign’s decree can be seen as both cause and effect of knowledge, and the PBRF partakes of this. The PBRF is a fiscal instrument that produces incomes for institutions, and it also uses the commodity logic of knowledge that sees knowing new things as a means to producing wealth and augmenting power. Research and the researcher come to be measurable in monetary terms (Duncan 2008). Some of us enjoy that; others don’t. Out of that situation, there appear to emerge two kinds of discourse within the academic community. One of these seeks to accept the power of this system, and indeed calls upon it to complete its project. Within this discourse, if there are criticisms of the PBRF, it is likely to be of a kind that says that there are loopholes that should be closed, normally involving even more paper-work, or that the assessment should include and hence validate something that presently may appear not to be included, like practice-based or commercially applied uses of research. The PBRF is the gaze of the master, and so we want more of it, we want its recognition, and we want it to be distributed fairly. Some of my comments about design-flaws within the PBRF may fit this type if those flaws are taken as matters to be addressed and corrected.

The alternative discourse places the PBRF in a prominent role within a history of the university that is typified by a corruption of values and an undermining of the traditions of learning and scholarship, within which the future of the humanities, as a traditional foundation of the university, appears to be in jeopardy. Rather than see the PBRF as a productive project that must be progressed and completed, the PBRF is examined and found wanting. It is seen to create a partial or incomplete representation of knowledge. My own comments about academic freedom and the PBRF perhaps fit into this second type of discourse. The danger with such talk is that it may depend upon a nostalgic view of a university that never really was.

Whether one demands more or less of the PBRF – or some alternative method – one may remain stuck within its horizons. Instead of talking about research, academics talk about the PBRF; or, they discuss the purpose and value of their research and publications in terms of the demands of the PBRF. Even if we tried, we cannot be completely innocent or ignorant of it. It does not merely count and evaluate researchers, leaving the field of research in academic disciplines untouched. It does actually change the perception of and motivation for academic activities. Some think that’s a good thing. I don’t.

I think academics need to stop asking for more recognition from this external assessment, and start asking why it has managed to insinuate itself so deeply into the universities.

Is this irritation that I’m expressing a symptom of something? The PBRF is certainly not the only bureaucratic policy structure to be created lately in the universities under the banners of accountability, incentives, performance, and the like. If anything, we can expect the funding, and hence the management, of universities to be increasingly performance-based in future. This represents a dramatic shift in the nature of the university as a bureaucracy: that is, to view the university as a hierarchical organization of offices whose activities are based upon the routine administration of policies and procedures.

Maybe not all of us would like to see the university in that Weberian light, but the university is a bureaucratic organization, and a particular model of bureaucratisation, closely linked to governmental policy instruments, is developing and can be seen quite clearly through the example set by the PBRF. It is worth considering, then, just what bureaucracy represents to us, psychologically. A proposition in psychoanalytic organizational theory, which could work as a premise, has it that participation in organisations generally, and bureaucratisation in particular, serve as a defence against anxiety (de Board 1973; Diamond 1993). At the obsessional extreme this can lead to a lifeless rigidity of behaviour and a narrowing of the field of perception. In academic terms, that would breed the kind of learning that is highly specialized and devoid of much reference to the complexities and pleasures of everyday life. Doesn’t that sound familiar?

Let’s add to that, though, this new form of anxiety that is controlled, if not produced, by the new emphasis on performativity – on the questions of what can our learning do or produce, how it competes, and how much can it earn – as distinguishable from the question of what our learning may tell us about the world and the mind that apprehends it. The PBRF and the concern for external funding and commercialisation of research all participate in that kind of anxiety of performativity, as does the herding and corralling of students when counted as EFTS. Should I suggest, therefore, that the new university is a university of performance anxiety? Were I to do so, I would not deny the implications of an anticipated failure to ‘measure up’ or to achieve pleasure and generativity that the idea of performance anxiety evokes. The only thing I would want to correct there arises from the fact that performance anxiety tends to be a masculine anxiety, and so perhaps the metaphor is too partial.

After all, the university is still referred to as Alma Mater by its alumni. Checking the OED, I learned that Alma Mater translates as ‘bounteous mother’ and was a title given by the Romans to goddesses, such as Cybele, who represented fertility and growth. For some, surely psychologically significant, reason, Englishmen adopted the term in the nineteenth century to refer to the schools and universities that had nurtured them in their youth. Is the notion of the university as an embracing and nurturing maternal institution that encourages ethical-intellectual growth still possible?

If things like the PBRF are symptomatic, they may be products of the anxiety and the excitement that we feel about knowing and not knowing – and this is frequently a matter of who knows, or who uses what they know, before others do, in the pursuit of some strategic purpose. And since we no longer consider knowledge as the thinking of an isolated Cartesian subject – in favour of relational, or even conflictual, understandings and utterances – the idea of competitiveness as productive of knowledge is not alien to us as theorists. So, is the notion of the university as an institution shaped by performance anxiety really a reflection of the dominant present-day ‘discourse of the university’? The performativity and the utilitarian ethics of the disciplines of economics and management do seem to dominate decision-making. If we try to re-establish a voice for the humanities in the university, is that voice doomed to be speaking in a space defined by managers, economists and the like? And, if one were to rebel against that idea, would this only be a form of nostalgia, based upon the old Newmanesque notion of a university of impartial, but perfectly useless, knowledge? Though I hasten to add that I too am proudly capable of producing work of no apparent utility, and I would defend any academic’s right to do so.

Derrida (2001) insisted on even more than what we know of as academic freedom, and said that the university should be granted an unconditional freedom to assert and to question, and to profess whatever may be considered to be the truth. Certainly, the statutory academic freedom that we are permitted in New Zealand is not unconditional. But, can we even profess, truthfully, the notion of an unconditional freedom? Is not the ‘free’ subject, including the scholar, always somehow under the permissive gaze of the Sovereign? Can we support an image of a university as a Platonic Academy, suspended in a pure abstract form of freedom, and not contingent upon the world around it and the norms within it? I don’t think we can. Academic freedom is a product of sovereign and strategic relations of power and cultural norms, and it exists always in a relation of tension and compromise with the world around it. By one means or another, it must be paid for and granted; and it sits alongside similar liberties and privileges, such as parliamentary privilege, freedom of the press and judicial independence. Academics are largely timid people who are afraid to speak out, and thus afraid to exercise the conditional freedom that they do have; and they thus allow others to lead the renegotiation and struggle over the administrative practices, such as the PBRF, that impinge upon, if not undermine, academic freedom. If academic freedom really is being eroded, then the academics themselves are partly responsible for that, due to their own anxiety.

What, then, is a University? Constitutionally, it is a creature of statute, but this fact will not satisfy the motives for that question. For most of my life, I have been occupied within the university, and yet I am reluctant to make a general statement to characterise what it is. Obviously, whatever it is, it is changing, due to new requirements and imperatives. Different people in different disciplines occupy themselves in the university for different reasons and with different understandings about their surroundings. I suppose, then, that there will be different kinds of universities for different kinds of scholars. That’s one of the good things about a university.

What I am sure the university ought not to be is a mere corporation whose business is to maximise its income through the production of skilled professionals and economically useful inventions. Universities may always have achieved such aims, but they also must do much more than that. I work within a University – thought of as a global institution with no walls – that has an intellectual heritage of more than two millennia and that began to appear as a recognised autonomous institution in the middle ages. I have a very tiny part to play in that long tradition, and the worth of what I do – in the sense of a worth that sustains my own work – is based in a number of qualities. That worth begins with my own curiosity and sense of intrinsic interest, but it’s also sustained by a striving to do justice to the authors who have come before me, to use my native language to the best possible effect, to use both my intuition and my capacity for logical argument, to rely upon the best evidence when I can find it, and to acknowledge the merits of a good argument, regardless of whether it pleases me. That includes engaging with the world in a range of different registers, sometimes practical, sometimes scholarly, sometimes poetic, but always examining what has enduring value in what we can know and what we can say. These are some of the intellectual and ethical qualities of learning that I try to use and to display, and to develop in my students. These qualities are needed well beyond the privileged space of the university and will stand our students in good stead as they become informed readers, theatre audiences, parents, barristers, journalists, scientists, politicians, physicians, and leaders in their various fields.

We work here for the intrinsic worth and the self-discipline of our careful reading and writing. The qualities of thought and action that such disciplines can develop are crucial for our ability as a society to conduct our affairs civilly, creatively and progressively. The University is not the only place where people can learn such ethical and intellectual lessons, but it does play a vital and enduring part in that. A life in the University enriches the capacity to speak and to act mindfully.




References

Adams, J. (2008). Strategic Review of the Performance-Based Research Fund: The Assessment Process. Leeds: Evidence Ltd.

Curtis, B. (2007). Academic life: Commodification, continuity, collegiality, confusion and the Performance Based Research Fund. New Zealand Journal of Employment Relations, 32(2): 1–16.

de Board, R. (1978). The Psychoanalysis of Organizations. London: Tavistock.

Derrida, J. (2001). The future of the profession or the university without condition (thanks to the “Humanities,” what could take place tomorrow). In T. Cohen (ed.), Jacques Derrida and the Humanities: A critical reader (pp. 24-57). Cambridge: Cambridge University Press.

Diamond, M. (1993). Bureaucracy as externalized self-system: A view from the psychological interior. In L. Hirschhorn and C.K. Barnett (eds), The Psychodynamics of Organizations, Philadelphia: Temple University, 219–236.

Duncan, G. (2008). Counting the currency of knowledge: New Zealand’s Performance-Based Research Fund. In I. Morley and M. Crouch (eds) Knowledge as value: Illumination through critical prisms. Amsterdam, New York: Rodopi (pp. 23–42).

Evidence Ltd (2007). The Use of Bibliometrics to Measure Research Quality in UK Higher Education Institutions. London: Universities UK.

Readings, B. (1996). The University in Ruins. Cambridge MA: Harvard University Press.

Smart, W. (2009). Making an Impact. Wellington: Ministry of Education.

Smart, W. and Weusten, M. (2007). (ex)Citing Research: A Bibliometric Analysis of New Zealand University Research 1981–2005. Wellington: Ministry of Education.

Smith, A.G. (2008). Benchmarking Google Scholar with the New Zealand PBRF research assessment exercise. Scientometrics, 74 (2): 309–316.

Taggart, M. (no date). Some Impacts of the PBRF on legal education. Unpublished conference paper.

Tertiary Education Commission (2007). Performance-Based Research Fund: Evaluating Research Excellence. The 2006 Assessment. Wellington: Tertiary Education Commission.