Peter Bates > Dilemmas of writing in public

Dilemmas of writing in public

Most of the documents we read are finished pieces of work, carefully crafted and edited in private before being shared with anyone else. My ‘How To’ guides are a different kind of paper – they are shared online from the first day, when the initial handful of ideas are incomplete, poorly phrased and tactless. The work is then edited many times, and on each occasion a revised version replaces the earlier material online. Despite this, the paper may still be lacking crucial concepts, evidence, structure and grammar. As readers continue to provide feedback, further insights are used to update it.

This way of writing is risky, as it opens opportunities to those who may misunderstand, mistake the stopping points on the journey for the destination, and misuse or distort the material. This way of writing requires courage, as an early version can damage the reputation of the author or any of its contributors. At least, it can harm those who insist on showing only their ‘best side’ to the camera, who want others to believe that their insights appear fully formed, complete and beautiful in their simplicity. It can harm those who are gagged by their employer or the workplace culture, lest they say something in a discussion that is not the agreed party line. It can harm those who want to profit from their writing, either financially or by having their virgin material accepted by academic journals.

In contrast, this way of writing can engage people who are not invited to a meeting or asked for their view until the power holders have agreed on the ‘right message’. It can draw in unexpected perspectives, stimulate debate and crowdsource wisdom. It can provide free, leading-edge resources. But it also surfaces the following five dilemmas, all linked together in a tangle of competing issues, illustrated here from the writing process for a paper on the payment rates offered to research participants.

Naive comment as evidence

I wanted to ask researchers for their rationale for the rates of participation payment they offered, to discover what they revealed through what was usually a brief and sometimes a superficial response to the question. After all, this is what potential research participants are likely to be given when they ask for an explanation. I did not want to cue people by suggesting what their answers might cover, but instead remain open to the possibility that the response might raise a completely new topic. In this I was vindicated, as responses were much broader and richer than the somewhat narrow preoccupations of the published literature on this topic.

Once I had received this initial response to my question, I then revealed that this was part of a larger piece of work and the perhaps rather offhand answers given to my initial question were being subjected to a detailed analysis, whereupon some researchers felt as if they had been deceived, since the existence of this larger work had been kept secret up to this point. There are rules about research using deception, but I’m not entirely sure that the boundaries of the concept are clear. How does deception make people feel, and do academic researchers always manage to avoid it through employing Participant Information Sheets and the scrutiny of the Research Ethics Committee? Should I have introduced my question by alerting researchers to the existence of the draft paper? Some respondents may have read it and considered their own arrangements in the light of what was already in the emerging paper before answering my question. Others, on learning that their responses were to be subject to rigorous reflection, may have withdrawn from the exercise.


Keeping faith with contributors

My second dilemma asks how to keep faith with contributors. As mentioned above, there was an immediate risk to good faith when I revealed that I had been collating responses and intended to place the emerging paper on my website. A further threat appeared when I received a response that in my view was poorly informed, erroneous or laughable. I decided to manage this by attributing positive and neutral material and anonymising contributions that could place the researcher in a negative light. Once general phrases such as ‘one research team thought…’ started to appear in the paper, my principle of accountability was lost. At a larger scale, the dilemmas listed here began to suggest that my whole approach to public accountability may have been misguided, and perhaps I should move the attributions to a private record and publish a revised and anonymised version. This action may please new contributors but would break faith with those who had agreed to the previous arrangements. In fact, rather than acknowledging their identity and intellectual capital, it would rob them of both and relegate them to a woefully inadequate ‘acknowledgements’ box, earning the condemnation of those who regard research that merely extracts data as exploitative.

The importance of this issue is amplified when the topic turns out to be more sensitive than first appeared. For example, another paper on funding arrangements for overseas aid workers began with the usual transparency by which every source was named. As more and more people saw the emerging work, it became clear that the topic had ramifications for people working in insecure parts of the world where intolerance or violence might be triggered. A wise, professional researcher would surely anticipate these matters and design in appropriate confidentiality from the start, so that no inconvenient truths emerged later on. But that is the very point of writing in public – that it starts before the investigator has had the chance to become wise, at the stage when issues are dangerously ill-formed. Traditional academic writing is retrospective and so we must wonder how often these late realisations and changes of direction are air-brushed out of the published account of the study. Keeping faith with contributors includes admitting that sometimes we start by intending one thing and end by delivering another, and so we must carry them with us, negotiating changes with them in how they and their work is presented. This is either a massive job for the prime author, requiring repeated invitations to check and recheck the paper as it changes, or involves a shift of responsibility, so that contributors get to choose how often to revisit the emerging paper or trust the prime author to have the last word.

Relationships across the research community

I began by assuming that researchers who are assigning public funds to those who participate in research would be entirely at ease about sharing their decisions with others. A paper in which individual projects are named and compared would permit researchers to contact one another, discuss their practices, learn and sharpen notions of best practice, so it was surely a good thing that would be universally valued. Each time I received an answer, I converted it into additional material for the draft paper, uploaded the revised version and sent an email to the researcher to ask them to check whether I had treated their input appropriately. This was better than simply sending the sentences off on their own, since the researcher would be able to see the nature of my website and the thrust of the argument in this particular paper and then decide if they wanted to appear or not. Suggestions for revision would come back within hours, and it was highly unlikely that anyone else would download an unauthorised version in the meantime from my sleepy and mostly unexplored website. Again, I was mistaken. A few researchers – quite understandably given their workload pressures – took weeks to respond, and then expressed their dissatisfaction that their formal consent had not been sought prior to anything about them appearing in the public domain. Perhaps I should create multiple copies of the emerging paper, each containing draft revisions from a single contributor, each waiting for approval which may never come.

Amateur or professional investigations

Writing in public is based on the idea that everything can be scrutinised, that there are no secrets, especially when public funds are involved. Asking researchers why they set a particular rate for payments to research participants was asking an entirely reasonable question and no-one would be shy about the answer becoming known. As the paper emerges, a growing community of researchers share the results and have the opportunity to interrogate one another, to learn together. There are pragmatic benefits too – version control is easy, since every snippet of new information is immediately added to the paper and every proposed revision instantly implemented. Rather than the writer claiming to know exactly what will interest the reader, sources are cited so that readers can seek out the additional information they need, and the writer can avoid sending out duplicate inquiries to the same respondent. The paper can be written in spare time, emerge over months or years, and is complete and entire in itself, since every source is identified and so meets the scientific principle of accountability.

Some researchers have found this to be wholly unacceptable. There is a clear boundary between the private and the public; evidence drawn from email correspondence is private and should never be posted in a public document; researchers should not be identified, so separate confidential papers should be created and then held in the background to store private information about where each snippet of information arose; the final published work should be anonymised so that no-one who appears in it can recognise themselves or identify anyone else. Formal consent should be obtained before starting, the text of my email inquiry should be approved by a Research Ethics Committee, responses should be loaded into software for thematic analysis; and formal conclusions kept under wraps until they had been reviewed by academic peer, schooled in the art of writing for scientists. My own investigations, driven by little more than personal ignorance and curiosity, shaped by a belief that truth is revealed slowly through dialogue, and unfunded, are simply not the right way to proceed. An army of change agents, lay people with good ideas but no money are locked out – if they want to investigate things, they should go and get trained as researchers, win funding worth a minimum of £100k and then do it properly.

Publishing as power

Large organisations often engage Communications Professionals. Where these people feel secure in the reputation of the organisation, they serve colleagues who wish to engage in dialogue with others. In contrast, when they are driven by fear, their task becomes one of gagging staff, controlling public statements and attempting to manage the corporate image. Staff are routinely prohibited from naming the organisation in online media or speaking to the press, unless they have explicit permission to do so and have been coached in the ‘right’ message. People become nervous about saying the wrong thing, worry about the borderline between private dialogue and publication, and begin every encounter with hesitancy. This is toxic to honest dialogue, truth-seeking and exploration. In practice, it means that projects which could have been co-produced, written in public and enriched by unexpected contributions are diminished, as it is considered less risky to the organisation’s reputation to write in private, even if the resulting output is weaker.

Driving service improvement

Until recently, the scientific community believed that evidence delivers change, which was curious, as there is very little evidence to suggest that research findings influence the provision of health or social care. Service providers are to get on with delivering care and should not be distracted from their task until robust, systematic evidence is amassed that points to a new practice, whereupon it should be quickly adopted and consistently spread throughout services. The most effective way to contribute to this evidence is to publish in high impact academic journals and definitely not on my own, rarely visited website.

A contrasting view is that change occurs as people ask good questions and then own and share their search for answers. Social media creates memes that transmit transformational ideas with increasing rapidity. A wide variety of resources can stimulate service improvement, especially those that provide detailed suggestions for how changes can be implemented – information that is often missing from scientific papers. Like in participative action research, sharing the question at the start and the findings as they emerge can engage people who want to share the journey.

My whole career has been devoted to creating ideas for improving health and social care services and this has often involved harvesting and presenting innovations that are already appearing in the field. After a hundred publications, I have abandoned the contortions of academic writing, and offers to collaborate with academic co-authors to bring my material and theirs to the scientific press have usually fallen on deaf ears. This may be because my discussion papers, whilst they would need serious revision prior to academic publication, are already perceived as in the public domain and so would not meet the journals’ appetite for unseen material, which is ironic, given the pitifully low visitor numbers at my website. Or it may be because the academic community have so many research ideas of their own and so many demands to generate outputs related to their own funded projects that they simply have no capacity to partner with anyone else in speculative writing driven by curiosity rather than a research funder. I read once that patients had generated a million ideas for healthcare improvements, so I am left wondering if there is an untapped wealth of ideas for innovation beyond the academy and how widespread service improvement can be triggered.


What should I do with the idea of ‘writing in public’? I am constitutionally frightened of conflict, so will probably move to a default position of anonymising this and future papers, inviting contributors to indicate if they want to be cited, rather than the other way around. In a space teeming with contradictions, this jars with my wish to honour contributors from the outset, but, in my mind, one criticism has the potential to cancel out ten approvals. Perhaps my efforts to ‘write in public’ are too idiosyncratic to be of interest to others, but I prefer to choose optimism and test the idea that the dilemmas that arise here may be pertinent to others who collect evidence and present it.