The impact of COVID-19 on the debate on open science: a qualitative analysis of published materials from the period of the pandemic

This study is an analysis of the international debate on open science that took place during the pandemic. It addresses the question, how did the COVID-19 pandemic impact the debate on open science? The study takes the form of a qualitative analysis of a large corpus of key articles, editorials, blogs and thought pieces about the impact of COVID on open science, published during the pandemic in English, German, Portuguese, and Spanish. The findings show that many authors believed that it was clear that the experience of the pandemic had illustrated or strengthened the case for open science, with language such as a “stress test”, “catalyst”, “revolution” or “tipping point” frequently used. It was commonly believed that open science had played a positive role in the response to the pandemic, creating a clear ‘line of sight’ between open science and societal benefits. Whilst the arguments about open science deployed in the debate were not substantially new, the focuses of debate changed in some key respects. There was much less attention given to business models for open access and critical perspectives on open science, but open data sharing, preprinting, information quality and misinformation became most prominent in debates. There were also moves to reframe open science conceptually, particularly in connecting science with society and addressing broader questions of equity.

Introduction
Calls for greater openness in science have been debated for two decades or more. Open access (OA) in scholarly communication has been widely discussed in its own right and is now often incorporated into a wider debate on open science (OS), including issues such as open data sharing (OD), open peer review, alongside OA. OS has become an important aspect of the governance of science, and is increasingly seen “as part of a well-functioning research system” (Science Europe 2022). In recent years, critical debates have revolved around OS’s impact on both research productivity, and equity, diversity, and inclusion (EDI) in the research system and beyond. Such issues also came to the fore in relation to the research system during the COVID-19 pandemic. The pandemic was a major test for the global research system and generated debate about the resilience of the system, issues of EDI within research, and the system’s capacity to respond to major emergencies. COVID’s societal impact, from health to politics, has also been widely debated. To date, however, little work has been done on how COVID and OS interacted. During the pandemic, many of its advocates saw the pandemic as finally demonstrating unequivocally the case for OS, with some going as far as to assert “Open science saves lives” (Besançon et al. 2021). Others suggested that significant changes were happening to patterns of scholarly communication caused by COVID-19 (Taraborelli 2020). At the same time, more sceptical voices observed that at least some approaches to OS were in danger of disseminating “unvetted science” which was “fuelling COVID-19 misinformation” (Gitlin 2020). The study reported in this paper was designed to provide a rigorous analysis of the international debate on OS that took place during the pandemic. We explore the debate through a qualitative analysis of a large corpus of key articles, editorials, blogs and thought pieces about the impact of COVID on OS, published during the pandemic in English, German, Portuguese, and Spanish. Specifically, we seek to address the question: how did the COVID-19 pandemic impact the debate on open science?

Our findings comprise a number of key points. First, many authors believed that it was clear that the experience of the pandemic had illustrated or strengthened the case for OS. Metaphors for the pandemic acting as a “stress test” or “catalyst” for OS, and the pandemic marking an OS “revolution” or “tipping point”, were frequently used. Second, it was commonly believed that OS had played a positive role in the response to the pandemic, and this was a key reason for the pandemic, in turn, strengthening the case for OS. The pandemic created for its advocates a clear ‘line of sight’ between OS, on the one hand, and societal benefits, on the other hand, making arguments about the benefits of OS for society more obvious and credible. Most of the arguments deployed in the debate were not substantially new; rather, they were reworkings of pre-existing arguments but, crucially, were set in a new context and given a new sense of urgency. Third, although the arguments around OS did not change, in some key respects the focuses of debate did. There was much less attention given to business models for OA, such as payment of article processing charges to publish works in an open form, and critical perspectives on OS seemed to receive less prominent coverage, particularly those relating to Global North/South relationships. Instead, open data sharing and preprinting (the dissemination of pre-reviewed versions of papers) were the most prominent in debates on OS, with discussion on openness often clustering around the pros and cons of these aspects of OS. Critics often expressed concerns about quality in the context of rising levels of preprinting along with concerns about wider issues of misinformation in society beyond the academy—an “infodemic” accompanying the pandemic. At the same time, questions on the scope and extent of OS itself also came into focus, with moves to give prominence to the aspects of OS that most directly connect science with society and address broader questions of EDI.

The rest of this paper presents these arguments in more detail and is structured as follows. In the next section we provide context about the COVID-19 pandemic itself and the development of the open science movement. We then go on to summarise our methods—the way we formed and analysed a corpus of work on OS published during the pandemic. We then present our findings in detail, unpacking and evidencing the summary above. This is followed by a conclusion, which compares some of the major focuses of attention on OS during the pandemic with some of the areas that received less attention. We discuss some of the possible implications of our findings for the future of OS developments.
Background
The COVID-19 pandemic, at its height between 2020 and 2022, was the most serious global health emergency for a century. It resulted in excess of 18 million deaths worldwide (Wang et al. 2022), and led to major social, economic and political disruption, including ‘lockdowns’ in many countries, severely limiting the movements of citizens beyond their homes (Lilleker et al. 2021). The international scientific community responded—often with additional emergency funding from governments—by focusing attention on areas such as recording and modelling the spread of the virus, developing treatments, creating vaccines, and producing health advice (including mask wearing, hand washing, and social distancing). Communication of scientific research became an important part of the response and issues of timely accessibility of research outputs—such as scientific papers or datasets—received much wider attention than ever before, both in the scientific community and beyond. Greater openness in disseminating research publications (‘open access’, OA) and other forms of openness in scientific practices (collectively known as ‘open science’, OS) (Fecher and Friesike 2014; Vicente-Saez and Martinez-Fuentes 2018) were highlighted by many as essential features of an effective response to the pandemic, particularly in the role they play in making research more rapidly and more widely available (Science Europe 2022). Many actors involved in scholarly communication undertook developments aimed at improving access to scientific outputs, including most scientific publishers who made their COVID-related content freely available, at least temporarily (Arrizabalaga et al. 2020a). In the context of the pandemic, OA and OS became subjects of intensified discussion and debate.

Debates about open science were, of course, not new in 2020. Approaches to achieving open access had been the subject of debate since at least the beginning of the 21st Century (Suber 2012; Willinsky 2003, 2006), albeit in relatively specialised circles. Other aspects of OS, particularly open data sharing, were discussed widely before the pandemic (Miedema 2022). Whilst adoption of OS, and particularly of OA, had increased over two decades, aspects of it have been (and still are) controversial. Issues such as different models of OA, sustainability of OA publishing, the role of commercial players, incentives for researchers, and the implications of OS for global equity have all contributed to ongoing, often heated, debates (Miedema 2022; Pinfield et al. 2020). Approaches have varied across disciplines and also differed across countries and geographical regions (Moskovkin et al. 2021; Simard et al. 2022). The costs and benefits of OS continue to be studied and debated in different contexts. We see many of these different strands of the debate reflected in our study, with the specific context of the pandemic casting new light on them.

Both OA and OS have always had a political dimension, particularly relating to the development of policies of governments or publicly-funded agencies sponsoring research. However, during the pandemic, science became more directly connected with political discourse and politicians than had previously been the case (Claessens 2021). Scientists were involved in advising politicians in ways that led immediately to far-reaching political decisions. Some achieved unusual public profiles, including giving briefings alongside political leaders on prime-time television, presenting evidence to justify policy (Joubert et al. 2023). This often saw politicians and scientists deploying rhetoric about the “battle” against the pandemic, in which we were “all in it together” (Montiel et al. 2021). Of course, political responses to the pandemic differed both within and between countries, with very different positions taken in relation to lockdowns, mask wearing, and vaccines. Political responses to the pandemic differed widely between countries. For example, the Brazilian federal government took a non-interventionist vaccine-sceptical approach, in stark contrast to Germany’s strict lockdowns and coordinated vaccine rollout (Lilleker et al. 2021).

The response of the scientific community to the pandemic was generally more consensual, with national and international science and health agencies, like the World Health Organization (WHO), collating evidence and coordinating responses (Hassan et al. 2021). There were some obvious successes, such as the rapid sequencing of the COVID-19 genome, accelerated by data sharing and open infrastructure (Chen et al. 2022). Of course, there were also differences between countries in how scientific communities responded and were able to respond. The pandemic illustrated global inequalities and power imbalances in scientific and medical capacity, despite international agreement on many scientific and medical priorities.

In some ways, the international medical and scientific response built on work carried out during previous health crises, which had also seen moves towards increased openness. For example, during the Ebola epidemic of 2014–2016, Yozwiak et al. (2015) urged the establishment of principles for rapid and responsible data sharing in epidemics, and for researchers working on outbreaks “to embrace a culture of openness” (p. 479). The 2015–2016 Zika virus epidemic provoked similar calls for data sharing, both among scholars (Chretien et al. 2016) and in the media (Wadhwa 2016). In 2016, over 30 public health journals and funding agencies issued a landmark statement on the importance of data sharing in public health emergencies (Wellcome 2016), which gave impetus to further calls for the use of preprints in such situations, albeit noting the challenges of the format (Johansson et al. 2018). Four years later, in the early stages of the COVID-19 pandemic, the Wellcome Trust again coordinated the creation of an influential statement calling for greater open practices (Wellcome 2020), which can be seen as taking forward these developments linked to previous health crises. Signed by 150 organisations including funders and publishers, the statement committed those organisations to encourage the sharing of research papers ahead of peer review (i.e., preprinting), data sharing, and other modes of making outputs openly available immediately.

During the pandemic, several major international policy initiatives designed to further OS beyond the COVID-19 context were launched. Notable among these were the memorandum on ‘Ensuring Free, Immediate, and Equitable Access to Federally Funded Research’ issued by Dr Alondra Nelson on behalf of the Office of Science and Technology Policy in the USA (Nelson 2022), and UNESCO’s (2021) ‘Recommendation on Open Science’. The first was a highly significant national policy which built on previous policies in the US, and adopted similar approaches to policies already established in some other countries (e.g., those aligned to the European Plan S). The second policy, from UNESCO, deployed an expanded framing of OS, and gained wide international coverage. This policy recommended adopting a definition of OS that includes engagement with actors beyond the academy and connections with diverse knowledge systems. This broader view of OS, as well as other features of the debate on openness during the pandemic, are examined in the analysis that follows. The UNESCO Recommendation is rapidly becoming an important reference point for OS in the way it frames the key issues, and so we mention it at other points in this paper, and we return to it in the conclusion, using it as a lens to view our findings.

Our study was designed to address the main research question: how did the COVID-19 pandemic impact the debate on open science? We aimed to investigate how the pandemic affected the ways different actors discussed their perceptions of the value of OS, as well as how they articulated the barriers to openness as they saw them. We also wanted to gain insight into how different commentators presented their beliefs around how the pandemic would influence future directions of OS.
Methods
In this paper we report a qualitative analysis of the OS debate over the course of the pandemic; a detailed description of the methodology and list of resources can be found in the supplementary materials. We assembled a corpus of published material (including editorials, thought pieces, blogs, media stories, press releases, and journal articles) that related to the research question, representing a range of topics and perspectives. We began by querying the Open Access Tracking Project, a database with a wide range of coverage and material types that focus on OS. Search strings combined terms about OS (e.g., open access, preprints, open data) with terms about COVID-19 (e.g., COVID-19, coronavirus, pandemic) in English, German, Portuguese, and Spanish (the languages spoken by members of the research team) Results were filtered by date (December 2019–December 2022), language, and source type (scholarly literature, grey literature, blogs, mainstream news media, meta-journalistic media, higher education/science press, and professional publications). Results were manually sorted for relevance, based on the research question above, producing an initial corpus of 260 items.

We augmented the corpus by searching additional databases for particular material types: Web of Science and Dimensions for peer-reviewed articles and editorials; Dimensions for conference proceedings and grey literature; BASE for peer-reviewed and grey literature; Overton for policy documents; and Nexis for media stories and press releases. We also searched websites of key policy and professional organisations (e.g., UNESCO, UN, WHO, Science Europe) and collected key sources of meta-journalistic media, higher education/science press, and professional publications recommended by members of our research team who are experts in these areas. These exercises ensured that key source types were thoroughly covered. These steps were performed in each of the four languages spoken by the research team, along with additional searches in language-specific databases (e.g., SciELO for Spanish and Portuguese, idw – Informationsdienst Wissenschaft for German). Results were assembled using Zotero, deduplicated, and manually sorted for relevance, finally yielding a total of 446 items (311 in English, 53 in German, 31 in Portuguese, and 51 in Spanish). The corpus comprised 145 newspaper and magazine articles (including online news sites and press releases), 141 journal articles, 103 blog posts and online opinion pieces, 36 reports and policy documents, eight conference papers and presentations, eight preprints, and five books or book chapters. Whilst we do not make claims that this corpus is exhaustive in its coverage, we believe it is a useful sample of the debate about OS that took place during the pandemic and represents a broad range of issues and perspectives from different actors working in different contexts.

We used NVivo 1.7.1 to perform a qualitative inductive content analysis of the documents using thematic analysis approaches (Terry et al. 2017). We first selected a group of 32 core pieces from the English-language corpus for deeper analysis. These items were identified through discussions among the research team as covering between them the main topic areas, types of sources, types of authors (e.g., journalists, scholars, policymakers) and pro-/anti-OS views. These pieces were coded in detail using open coding. A codebook was generated by one researcher and validated by other members of the team based on a reading of the sources, generating 128 codes. The remainder of the English-language corpus was then coded with a lighter touch to check for other arguments, nuance, and general frequency of each argument. This was done via the use of memos in NVivo, which were then assigned to a small number of codes in the codebook to denote the main content and argument of each piece. This process was repeated in German, Portuguese, and Spanish by members of the research team who were fluent in these languages, who applied codes from the English-language codebook but also added to or amended the codebook as appropriate. Two further codes were added during this process, giving a total of 130. Using the codebook, themes were developed from the data, discussed in the Findings and Discussion sections below.
Findings and discussion
The pandemic as a “stress test” for OS
The idea that the pandemic had strengthened the case for OS was evident in much of the corpus (Breznau 2020; Havemann et al. 2020; Lane and Lifshitz-Assaf 2022; Molldrem et al. 2021; Rijs and Fenter 2020; Shearer et al. 2020; Stuart 2021). There was a noticeable use of terms like “revolution” (Taraborelli, 2020), “disruptor” (Butler et al. 2021a, 2021b) and “paradigm change” (Cárdenas-González and Álvarez-Buylla, 2022), with some items asking whether the pandemic was these things and others asserting that it was, but always creating a sense that significant changes were happening. Some articles clearly stated that the authors, or those they cited, had changed their minds about OS, or at least viewed open practices more positively, as a direct result of observing the adoption and impact of OS approaches during the pandemic (Callaway 2020; Horby 2022; Kim 2022).

OS had been gathering momentum for many years before the outbreak of COVID-19 (Callaway, 2020), and many items in our corpus noted this, often arguing that the pandemic had accelerated OS beyond the gains seen in previous decades. Khamsi (2020) argued that “[t]he pandemic is accelerating a trend, but this was coming already”. Bobrov (2020), writing early in the pandemic’s course, went so far as to claim that “the tide [had] already turned” for OA before COVID-19 and that it was “a widely accepted goal”, citing increasing rates of OA publishing and the rise in preprints as evidence. However, Rebecca Lawrence, Managing Director of open-access publishing platform F1000Research, maintained that in the pre-pandemic period “the pace of change has been much slower than many had hoped”, and suggested that the pandemic could be a “trigger” for “wholesale change” (Lawrence 2020).

The pandemic was sometimes described as a “stress test” or “test case” for OS (Altman and Cohen 2022; Clinio et al. 2022; Hagemann and Joseph 2020; Molldrem et al. 2021; Russell 2020; Science Europe 2022). In this narrative, the pandemic provided a set of conditions where the principles and practices of openness were tested in the context of a real-world emergency. There was a sense recurring in the corpus in various guises that the COVID-19 outbreak demonstrated that an alternative system with openness at its centre was possible, showing what could be done when required. Long-standing OS advocate Leigh-Ann Butler and colleagues (Butler et al. 2021a, 2021b) argued that the pandemic illustrated how quickly the research ecosystem could come together to share results on a global scale, confirming the value of openness, especially during times of crisis. Shearer et al. (2020) felt that the situation had proved that established flaws in the system regarding the time and cost of research could be overcome “with enough political will”, alluding to the role of policymakers in promoting openness. Some of the rhetoric used of the response to the pandemic, as being a “battle to defeat COVID-19” (Perkins Coie 2020) or “combatting COVID-19” (OECD 2020), were in line with the wider political rhetoric and apparently designed to emphasise the positive link between (open) science and society. The thinking that the pandemic had been a successful test for OS is also reflected in policy making, or at least, there is evidence of such thinking being used to justify new policy. The Nelson memo states, “Immediate public access to COVID-19 research is a powerful case study on the benefits of delivering research results and data rapidly to the people” (Nelson 2022, p. 2) and expresses support for opening up different kinds of research outputs. Other authors claimed that the COVID-19 response was proof that scholarly communications could indeed be accelerated and opened up, and that entrenched cultures and mindsets could be changed (Brainard 2021; Callaway 2020; Lawrence 2020).

There was some disappointment expressed that OS in the pandemic had not gone far enough, with levels of data sharing and preprinting still lower than many had expected or hoped (Homolak et al. 2020; Waltman et al. 2021). Brainard (2021) felt that early predictions that preprints would dominate and reshape the landscape were proving to have been exaggerated, and that “hopes for a wholesale revolution are fading” (p. 1182). However, more common were concerns that the perceived gains made during the pandemic would not be sustained in the post-pandemic period. Moore (2020) expressed concerns that gains might be lost once the initial sense of crisis was over—for example, through reintroduction of paywalls—and that the momentum that had built around OS might dissipate. This perspective was especially common in the German literature, where authors conveyed the desire to institutionalise ad hoc measures taken during the pandemic to preserve progress made and ensure long-lasting change (Blasetti et al. 2020; Frick 2020; Haerdle 2020; Taschwer 2022). The view in the Portuguese-language literature was similar, with many seeing the uptake of OS during the pandemic as exceptional and unlikely to continue (Candido 2023; Ferreira 2020; Nassi-Calò 2022; Rodrigues 2022). In the case of Brazil, Rodrigues (2022) recalled how the Zika virus epidemic—which affected many countries in Central and South America—had brought the same debate, and that the increased openness in that situation had not survived as had been hoped.

Clear ‘line of sight’ to the benefits of OS
The common argument that the response to the pandemic had a positive impact on OS was usually based on a parallel perception that OS had a positive impact on the response to the pandemic. This apparent mutually-reinforcing effect was fundamental to the argument of many advocates—the case for OS was strengthened by the pandemic precisely because OS had strengthened the pandemic response. Authors expressing this view argued that OS accelerated and improved scientific and political reactions to the COVID-19 emergency (Barbour and Borchert 2020; Gentemann et al. 2022; Hatch 2020; Lane and Lifshitz-Assaf 2022; Molldrem et al. 2021; Tavernier 2020). After describing the rapid scientific response to COVID-19 in terms of understanding the virus and developing diagnostic testing and treatments, Barbour and Borchert claim that,

“the success in responding to the COVID-19 pandemic has depended fundamentally on open science: scientists being able to rapidly see what others have done, to check its validity by accessing both the underlying data and the researchers’ interpretation of their research, and to build on it for the next advance” (Barbour and Borchert 2020)
The rise of OS was said to have had obvious scientific benefits which then led to clear benefits to society. The societal benefits emphasised most in the corpus were health-related, as might be expected in the context of the pandemic. The development of interventions such as diagnostic tools, treatments, and eventually vaccines are featured in the corpus, alongside other benefits, like informing policy making.

The pandemic seemed to create a clear ‘line of sight’ between OS, on the one hand, and societal benefits, on the other hand, making the arguments about the benefits of OS beyond the academy more visible and credible. Taschwer (2022) emphasised these broader, societal consequences of openness, arguing that “[t]his enormously accelerated flow of communication of knowledge from the laboratory to the public contributed and continues to make a significant contribution to overcoming the pandemic in terms of health, society and the economy.” Others focused on public impacts of particular parts of the OS system. Koerth (2021) quotes one of her interviewees, Richard Sever, a preprints advocate and cofounder of the preprint servers bioRxiv and medRxiv, as saying: “I had one MD who contacted me, and he said, ‘You know, there are probably people who are alive today who would have been dead if not for preprints.’” This kind of argument, linking a mode of OA with medical outcomes, was clearly designed to reinforce the view of the urgent need to make OS more mainstream.

Of course, this emphasis on societal benefits of openness was not new. OS advocates had presented similar arguments for many years, often focused on health, but including other benefits such as economic benefits and evidence-based policy making (e.g., Willinsky 2004). In fact, what we see during the pandemic is the case for OS being made essentially using arguments that have been deployed before. Key arguments (summarised in Table 1a - supplementary materials) included better health outcomes (Besançon et al. 2021; Capps 2021; Hatch 2020; Horby 2022; Lane and Lifshitz-Assaf 2022; Rijs and Fenter, 2020), but also other benefits including positive economic outcomes (Besançon et al. 2021; Havemann et al., 2020; UNESCO, 2021), and improved and more timely communication with the public about science (Fecher 2020; Lane and Lifshitz-Assaf 2022; Nelson 2022). This was argued by some to have had a “democratizing” effect (Lane and Lifshitz-Assaf 2022; Raven and Haigh 2020; Rosa et al. 2021; Tavernier 2020; UNESCO 2021), creating greater inclusivity (Lane and Lifshitz-Assaf 2022; UNESCO 2021) and having benefits for education, particularly in health-related disciplines (De Giusti 2022; Orellana et al. 2020). Although such benefits were commonly featured in OS advocacy before the pandemic, the COVID-19 context appears to have made these arguments sharper and more concrete, with the pandemic providing a clear illustration of the benefits of OS that provided advocates an opportunity to argue more assertively for its mainstreaming as an urgent priority. The pandemic also gave OS and its benefits more coverage, with an awareness of openness reaching different communities previously with low or no awareness of the issues (Molldrem, et al. 2021). Such benefits were wide-ranging, but often presented in either broad-brush or anecdotal terms. This may in part be due to the short nature of many items in the corpus, yet it is noticeable that many of the positive outcomes of OS in relation to the pandemic were often assumed to be self-evident, and therefore needing little elaboration.

These arguments about the benefits of OS for society worked alongside a second related set of arguments used in the corpus which focused on upstream benefits of OS, for science itself. Such arguments were mostly instrumental, emphasising that OS makes science more effective and efficient. Like the societally-focused arguments, this group of arguments had also often been used before the COVID-19 outbreak. Yet they too appeared to be given new impetus by the pandemic, with much of the discussion focused on how the crisis illustrated different strands of the argument. Table 1b (supplementary materials) sets out the main arguments cited in the corpus about how the pandemic demonstrated the instrumental benefits of OS.

Prominent amongst these arguments of the benefits for science was the view that openness accelerates scientific work by making communication of findings faster. This emphasis on speed was understandable in the context of the pandemic (Bermúdez-Rodríguez et al. 2020; Horby 2022; Lane and Lifshitz-Assaf 2022; Lawrence 2020; Yan 2020). Discussing the context created by the COVID emergency, Gentemann et al. (2022) argue, “In this environment, the rapid dissemination of knowledge is critical; closed, siloed knowledge slows progress to a degree society cannot afford.” This argument about the speed of knowledge dissemination was complemented by a related argument that the science itself was speeded up, with new findings able to emerge more quickly (Gentemann et al. 2022; Hatch 2020; Kunz 2021; Lane and Lifshitz-Assaf 2022; Lawrence 2020). There were also arguments around increased efficiency in science due to OS, including the ideas that openness decreases waste, unintended duplication and costs associated with research (Besançon et al. 2021; Lawrence, 2020). The increased visibility of science to the global community during the pandemic (Hatch, 2020; Lane and Lifshitz-Assaf, 2022; Lawrence, 2020) was described as complementing the improved transparency ascribed to OS, making authors and peer reviewers more accountable (Barton et al. 2020; Haerdle 2020; Nelson 2022; Owens 2022). In turn, accountability was viewed by some as contributing to improvements in the quality of scientific work, since mistakes or fraud become more easily detectable (Besançon et al. 2021; Fox 2020; Harris 2022; Shearer et al. 2020; Tavernier 2020). OS was also said to encourage collaboration (Ferreira 2020; Hatch 2020; Lane and Lifshitz-Assaf 2022; Rodrigues 2020; SPARC Europe 2020) and interdisciplinarity (Fecher 2020; Lane and Lifshitz-Assaf 2022). Others argued that OS makes for a more inclusive science globally (Harris 2022; Schaffer 2021; Stewart and Reiners 2022; UNESCO 2021) and a more ethical system, with greater safeguards, possibly ensuring scientific integrity (Besançon et al. 2021; Nelson 2022). All of these benefits of OS for science were, of course, commonly-raised points before COVID-19, but were now said (or assumed) to be illustrated all the more powerfully by the pandemic.

Many advocates of openness in the corpus portrayed OS as a way of correcting major weaknesses or problems in scholarly communication and publishing systems and processes. They argued that the pandemic exposed and magnified these flaws as well as demonstrating and amplifying the comparative benefits of OS. Some pieces highlighted what they saw as general dysfunction in scholarly communication, as well as focusing on particular aspects such as the inefficiencies of the current system (Barbour and Borchert 2020; DeBruin 2020; Larivière et al. 2020), the duplicative nature of the multiple peer review rounds a paper must typically undergo before acceptance (Arrizabalaga et al. 2020b; Barbour and Borchert 2020; DeBruin 2020; Donato et al. 2020; Larivière et al. 2020), and the lengthy or slow processes involved in scientific systems in general (Miller and Tsai 2020; Shearer et al. 2020). Additionally, the current system was viewed as poorly coordinated (Barbour and Borchert 2020; Besançon et al. 2021), insufficiently collaborative or interdisciplinary (Fecher 2020; Homolak et al. 2020), unnecessarily costly (Larivière et al. 2020; Shearer et al. 2020), and inadequately transparent (Shearer et al. 2020). There was also criticism of too much focus on metrics and outputs (Besançon et al. 2021; Fecher 2020), and on journal articles or books at the expense of other contributions such as data, metadata, preprints, and protocols (Fecher 2020; Shearer et al. 2020). They used these arguments as a way to highlight the unfairness of the system to those with fewer resources; for example, researchers working in the Global South (Carlin 2020; Chan et al. 2020; Havemann et al. 2020; Lane and Lifshitz-Assaf 2022; Okafor et al. 2022; Pells and Smits 2022; Shearer et al. 2020; UNESCO 2021); publishing in languages other than English (Chan et al. 2020; Harris 2022; Havemann et al. 2020; Larivière et al. 2020; UNESCO 2021); or employed at less well-funded institutions (Lane and Lifshitz-Assaf 2022; Miller and Tsai 2020; Schaffer 2021), as well as early career researchers (Besançon et al. 2021; Lane and Lifshitz-Assaf 2022; Nelson 2022) and minority communities (Chan et al. 2020; Dey 2022; Havemann et al. 2020; Kadakia et al. 2021). Worryingly, despite research suggesting that the pandemic exacerbated gender differences more generally in society (Flor et al. 2022; Madgavkar et al. 2020; Yavorsky et al. 2021) and in science (Caldarulo et al. 2022; King and Frederickson 2021; Lee et al. 2023; Pinho-Gomes et al. 2020), there was little mention of it in the corpus in connection with OS.

New focuses of debate
Shift away from OA business models and critical perspectives
The arguments deployed to support the case for OS during the pandemic may not have been new, although they were apparently given new impetus in the context of the global emergency, but there were apparent shifts in the focuses of the debate. There was a notable shift away from discussion around modes of OA and business models for OA, which had been a core part of the debate around OS before COVID-19 (Abadal 2013; Harnad et al. 2004; Zhang and Watson 2017). It seems this issue receded to the background following the temporary removal of paywalls on COVID-19 content by publishers, and therefore did not attract much attention during the pandemic, as much as would have been expected had paywalls not been removed. Some argued that the publishers’ approach was an implicit admission that the current paywalled system is “unjustified and inefficient” (Rooryck 2020), or at least sub-optimal. As noted by Tavernier (2020),

“many publishers have tacitly agreed that open access is beneficial to scientific advancement and necessary to move science forward… publishers by their actions have validated the argument that where there are barriers to access to such knowledge, the pace of scientific progress decelerates” (pp. 226-227).
There was also some debate about the specifics of the publishers’ approach: how long the measures should last, what topic areas they should cover, and on what basis such actions should be taken. Several pieces in the corpus questioned what might constitute ‘relevant’ content, noting that research into COVID-19 drew on prior research into, for example, other coronaviruses, ventilators, or mask-wearing not made freely available (Barbour and Borchert 2020; Larivière et al. 2020; van Gerven Oei, 2020). As Barbour and Borchert (2020) contend, “although the corona virus [sic] may be novel, research on the corona virus in fact draws on a long tail of often closed research literature” (Barbour and Borchert 2020). Moreover, such research was not limited to medicine or life sciences, but spanned a broad range of disciplines (Matthews 2020)—including social sciences and humanities (Taster 2020; van Gerven Oei 2020)—and topics—such as economics (Lane and Lifshitz-Assaf 2022), education and behavioural science (Ala-Kyyny 2020), mental health (Kiley 2020a, 2020b; Morrison 2020), and geography and management (Lane and Lifshitz-Assaf 2022)—which were not opened up by publishers unless directly about COVID. Like the broad range of individual topics, the interdisciplinary nature of much research relevant for combating the pandemic was also noted and criticism made of decisions to leave much of it behind paywalls (Fecher 2020; Kiley 2020b; Lane and Lifshitz-Assaf 2022; Larivière et al. 2020; OECD 2020; Stuart 2021).

This argument about the need for access to a broad range of literature to address the pandemic often gave rise to calls to open up all research, regardless of field or date (Finley 2020; Kiley 2020b; Larivière et al. 2020; van Gerven Oei 2020). This argument would often start with literature on public health: “[the] current pandemic makes abundantly clear that the public availability of public knowledge indeed saves lives – but it doesn’t do so only now, it always does” (van Gerven Oei 2020). From there, the question was often posed: if this could be done for COVID-19, why not for other topics? This suggestion was extended to other health crises and diseases (Kamel 2020; Larivière et al. 2020; Lawrence 2020; Legarda 2021; Napolitano 2020), as well as other global challenges such as poverty (Pells and Smits 2022; Stuart 2021; UNESCO 2021), climate change (Madise 2021; Rijs and Fenter 2020; Stihler 2021; UNESCO 2020), or addressing UNESCO’s sustainable development goals (Alemneh et al. 2020; Jones and Campbell 2021; Stuart 2021; UNESCO 2021). Documents in the corpus also emphasised the value of openness for social movements and civil society organisations (Chan et al. 2020; Shearer et al. 2020). Several authors simply advocated for all research being made available (Finley 2020; Rooryck 2020; Van Noorden 2022; Willinsky 2020), and not only in times of crisis (Kiley and Rooryck 2022; Napolitano 2020; Nelson 2022; SNSF 2020). It is significant that this kind of thinking seems to have underpinned policy making. In the USA, the Nelson memo uses exactly this line of argument to justify its far-reaching proposals:

“Immediate public access to COVID-19 research is a powerful case study on the benefits of delivering research results and data rapidly to the people. The insights of new and cutting-edge research stemming from the support of federal agencies should be immediately available—not just in moments of crisis, but in every moment. Not only to fight a pandemic, but to advance all areas of study, including urgent issues such as cancer, clean energy, economic disparities, and climate change.” (Nelson 2022)
However, this emphasis on extending the accessibility of the range of scholarly literature does not seem to have been considered through a critical lens. Indeed, critical perspectives on OS seemed to receive less prominent coverage in general, particularly those relating to Global North/South relationships. Much of our corpus (pieces mostly written from Global North perspectives) seemed to treat the challenges of the pandemic as global problems, and appeared to assume that solutions developed in the Global North would be universally applicable. Any unique consequences of openness during the pandemic for many contexts in Low-Income Countries do not seem to have received much attention.

Focus on open data sharing
At the same time, the debate shifted towards several aspects of OS that took particular relevance during the pandemic—notably open data sharing and preprinting. Discussion on the first of these, data sharing, often started with the widespread claim that OD had been important in combating the pandemic (Barton et al. 2020; Desai et al. 2021; Gonzalez-Zapata et al. 2021; Gutierrez and Li 2020; Larson et al. 2022; Simons et al. 2021). Cited advantages of OD in the response to COVID-19 included benefits to science—such as validation of research conclusions, increased re-use of data, enablement of replication studies, and facilitation of the peer-review process (Bal 2021), plus greater transparency (Barton et al. 2020). Discussions also described broader benefits to society through accelerating development of treatments for COVID-19 (Besançon et al. 2021; CMS Law-Now 2021). However, despite positive feelings following the early use of OD for important advancements such as genomic sequencing and vaccine development, some disappointment and concern around OD’s effectiveness and level of uptake followed, along with concerns about infrastructural issues (Besançon et al. 2021; Li et al. 2021; Lucas-Dominguez et al. 2021; Strcic et al. 2022; Watson 2022; Yao and Park 2020). There were also some caveats around the value of open sharing, with some arguments in favour of promoting data sharing amongst “the people that need it” and could understand it (Shadbolt 2020).

The focus on OD was particularly strong in the German texts in the corpus, especially in media coverage (Amrein 2020; dpa 2020; Gillmann 2020; Hoppe and Specht 2020; Humborg 2022; Schuster 2022; Skinner 2021; Streim et al. 2020; Zindler et al. 2021) but also in scientific literature (Pilgram et al. 2021) and policy documents (Bundesministerium des Innern 2021). Within this German-speaking context, there was a recurring argument that the pandemic exposed major inefficiencies and gaps in the collection and sharing of government data, citing lack of coordination between stakeholders, lack of interoperability between local and national systems, and slow publication of COVID-19 data (Humborg 2022). Several pieces argued that administrative and government data in both Germany and Austria should be made more available to companies and the public or mentioned startups, apps, and services as beneficiaries of OD (Hoppe and Specht 2020; Schuster 2022; Streim et al. 2020). There was a strong emphasis across these pieces on efficiency, innovation and global competitiveness, as well as “[solving] societal challenges” (Streim et al. 2020). They also highlighted how OD could lead to economic benefits, e.g., from commercial developments using data, as well as the gains in public trust. Historically, Germany has not ranked highly on the EU’s survey on OD maturity, placed 14th out of 35 countries (European Commission 2022), which could partly explain the push for more and better OD. We clearly see the pandemic used to further this agenda.

Focus on preprinting
In addition to OD, the topic of preprinting saw extensive and high-profile debate during the COVID-19 pandemic. Preprinting, it was often argued, had been important in responding to the crisis (echoing the claim made in relation to OD). While use of preprints was growing even before COVID-19 (Callaway 2020; Rieger 2020), from early 2020 onward, there was a significant rise, attributed to the pandemic (Callaway 2020; Fox 2020; Fraser et al. 2021; Kiley 2020b). Some early accounts discussed the possibility that a mainstreaming of preprints during the crisis could result in a major, long-term change in the scholarly communication system. Whilst much of this did not materialise (Waltman et al. 2021), there was certainly evidence in the corpus of growing acceptance of preprints (Brierley et al. 2021; Callaway 2020; Coates, 2021). There were notable editorials from The Lancet (Kleinert and Horton 2020) and Nature Reviews (‘Watching Preprints Evolve [Editorial]’ 2021) confirming their acceptance of preprinted work due to the successful use of preprints during the pandemic.

As with much of the debate about OS and the pandemic, arguments surrounding preprinting were not new, but rather used COVID-19 to illustrate long-standing claims about the benefits of preprints (Chiarelli et al. 2019) and post-publication peer-review (Besançon et al. 2022). Apart from being openly available, a point often taken for granted in arguments about them, the key argument for preprints deployed by their advocates during the pandemic was speed—crucial during the pandemic:

“During the coronavirus outbreak, preprints have been increasingly used as a way to quickly share new research prior to going through peer review so that other researchers in the field can quickly assess the outputs and, where appropriate, start to build on them without the normal delay (often months) awaiting formal journal publication.” (Lawrence 2020)
Other arguments in support of preprints were familiar ones, such as the potential for receiving feedback on work that could improve its quality (Besançon et al. 2021; Fox, 2020; Watson 2022). Similarly, arguments against preprinting were also familiar. The most important concerns were about low-quality or misleading information being publicly available (Chiarelli et al. 2019), although the intensity of the criticism arguably increased during the pandemic, with “unvetted science” in preprints said to be “fuelling COVID-19 misinformation” (Gitlin 2020). Criticism of preprinting also seemed to become something of a lightning conductor for scepticism about OS more broadly. Concerns were expressed in relation to the impact on science, noting the potential for misleading information to be taken up in other research, but also in wider society, where there was danger that journalists or other users beyond the academy could be misled (van Schalkwyk and Dudek 2022). In other cases, there was a sense that preprints would remain part of the system in future, but caution would be required. “Preprints are no panacea, but as they have continued to develop in their own right they are putting useful pressure on some of the structures of traditional scientific publishing” (Rieger 2020). Other writers expressed scepticism that preprints had enabled the benefits claimed by their advocates, noting, for example, that most COVID-19 preprints received very low rates of commenting (Flanagin et al. 2020; Krumholz et al. 2020).

Focus on quality, retractions, and misinformation
Some of the debate about preprinting played itself out in ways that specifically related to the pressures of the COVID emergency, often focusing on the tensions of speed versus quality in scholarly communication. Some of the debate about preprints and quality clustered around retractions. One notable preprint claimed to have identified similarities between the DNA of the SARS-CoV-2 virus and that of HIV, the virus that causes AIDS, and suggested that the virus might therefore have been human-caused (Pradhan et al. 2020). Examples of problematic preprints such as these fuelled the case against preprinting, with some arguing that such examples demonstrated the potential for misinformation to spread quickly (Bagdasarian et al. 2020; DeBruin 2020; Koerber 2021; Molldrem et al. 2021). However, some of these same examples were also used in support of preprints; the AIDS-related manuscript very quickly received dozens of critical comments, and was withdrawn within 48 h, demonstrating the rapid self-correcting nature of the preprint system (Kiley 2020b); the utility of post-publication peer review has also been discussed elsewhere (Besançon et al. 2022). Many argued that such a system in fact increased the chances that inaccurate or flawed studies would be spotted, and more quickly and easily withdrawn before they have the chance to circulate widely (Flier 2020; Koerber 2021; Oransky and Marcus 2020; Shearer et al. 2020; Taraborelli 2020; Thomasy 2020). In practice, withdrawals of preprints were rare—a rate of 0.26% was cited by Yan (2020).

Retracted preprints were often compared to other high-profile retractions during the pandemic—notably those of articles published in prestigious, peer-reviewed journals. A study by Mehra et al. (2020a), published in The Lancet, examined the effectiveness of hydroxychloroquine and chloroquine; another piece in NEJM by some of the same authors (Mehra et al. 2020b) investigated the use of blood pressure medications in COVID-19. Both articles relied on data from the discredited Surgisphere database (Offord, 2020), which furthermore was not made openly available, or even to the authors of the studies. Several authors used these high-profile examples to demonstrate that closed, peer-reviewed science can still be fallible (Brainard 2021; Horby 2022; Minari et al. 2020; Oransky and Marcus 2020; Rabin 2020; Redden 2020; Yeo-Teh and Tang 2021).

The potential for misinformation, noted as a key challenge facing the preprint system, was also discussed more broadly. Many authors discussed the possibility of incorrect information being circulated, or of information being misinterpreted, as a potential risk of increased openness (Besançon et al. 2021; DeBrui, 2020). This was seen as a particular concern in a crisis, such as the pandemic, where information can potentially spread faster and wider, often driven by social media, and create an ‘infodemic’ (Koerber 2021; Mogensen 2020; Molldrem et al. 2021). Several noted the potentially dangerous consequences for public health if incorrect information was used in medical treatments or policymaking (Bagdasarian et al. 2020; Bramstedt 2020; Breznau 2020; Flanagin et al. 2020). Others emphasised that research—whether peer-reviewed or preprinted—is hard to fully withdraw once made public (Ala-Kyyny 2020; Besançon et al. 2021; DeBruin 2020). The public and media may continue to spread the misinformation, intentionally or not (Flanagin et al. 2020), and “flawed and fraudulent papers continue to be cited approvingly, even following retraction” (Molldrem et al. 2021, p. 1476). Here it was seen as important to ensure that more accurate information was brought to the attention of the public, with many highlighting the crucial role of journalists, including data journalists, in doing so (Besançon et al. 2021; Desai et al. 2021).

Reframing OS
The link between OS and wider society was at the centre of a final focus of debate within the corpus, the debate around the scope of OS itself. We saw some moves during the pandemic to reframe OS, which took two main forms: first, conceptualising OS as a coherent whole, emphasising how the different components of OS integrate with each other (Besançon et al. 2021; DFG 2022; Minari et al. 2020; United Nations 2020); and second, extending the boundaries of what constitutes OS (Chan et al. 2020; Lane and Lifshitz-Assaf, 2022; UNESCO, 2021). As with many aspects of the corpus, there is evidence of these moves prior to the pandemic (e.g., Albornoz et al. 2017; Fundación Karisma, 2020), but the experience of the pandemic seems to have allowed this more expansive view of OS a greater opportunity to resonate and gain traction.

The first of these represents a move away from a focus on mainly OA to OS, with OS being seen as a more integrative whole, rather than a loosely coupled set of parallel developments. This process was already happening pre-pandemic (National Academies of Sciences, Engineering, and Medicine et al. 2018) and it is certainly implicit in initiatives such as the Wellcome-coordinated statement of funders, publishers and others following the outbreak of the pandemic (Wellcome 2020). However, Besançon et al. (2021) made the point in the context of the pandemic, emphasising how different components of OS—including OA, OD, plus other open agendas, like open peer review, open metrics, open research assessment, and open preregistration—worked together synergistically (or could have done had they been used more), carrying out mutually-reinforcing, mutually-correcting roles. That view is evident in many accounts of the role of OS in the pandemic which cover a wide range of open practices and emphasise their synergistic benefits (e.g., Ala-Kyyny 2020; Gonzalez-Zapata et al. 2021; SPARC Europe 2020; Tse et al. 2020).

In terms of a broader view of the scope of OS, discussions were often focused on the UNESCO (2021) ‘Recommendation on Open Science’ which was released during the pandemic and has since prompted a good deal of commentary, particularly in our corpus from South America (e.g., Madé and Gómez-Valenzuela 2022). Alongside more familiar aspects of OS—“open scientific knowledge” and “open science infrastructures”—the UNESCO Recommendation also identifies two other “pillars” of OS: “open engagement of societal actors” and “open dialogue with other knowledge systems”. The first of these, engagement with other societal actors, is obviously relevant in the context of a global crisis, especially as the crisis disproportionately affected marginalised populations (Bhaskar et al. 2020; Jane Addams College of Social Work 2020; Kantamneni, 2020). We have already seen the case being made for OS in terms of its wider societal benefits (Capps 2021; Nelson 2022; Rijs and Fenter 2020). The pandemic was used by some contributors to the corpus to further illustrate the importance of science achieving impact beyond the academy in ways that can be facilitated by OS (Lane and Lifshitz-Assaf 2022). However, the inclusion of knowledge generated by non-academic actors into the definition of OS itself—making them a core part of OS, rather than the beneficiaries—is arguably a more radical step, at least as included in prominent policy documents.

The extension of the scope of OS to include connections with other knowledge systems is more radical still, and has received less attention, either before or during the pandemic. Chan et al. (2020) contributed to this debate with a document for UNESCO which preceded the UNESCO Recommendation, having argued for years prior to the pandemic, along with collaborators of the OCSDNet Project, for the importance of this agenda (Chan and Okune 2019; OCSDnet 2017). However, despite the work of groups such as UNESCO to bring such issues into the mainstream of OS debate, it has still often resided on the periphery. It remains to be seen whether this broader understanding of OS gains wider acceptance, and, if it does, what difference it makes to policy and practice.
Conclusion
The pandemic was a period characterised by an intense focus on science and on its links with society. It was a test of the resilience of the scientific system in responding to a major global emergency. Our findings suggest that the crisis seems to have created conditions for greater attention on openness, specifically on what its advocates see as its role in improving the scientific process and connecting science with society. It is evident that the COVID emergency gave rise to particular characterisations of OA and OS from different perspectives, with some picturing the changes being brought about as revolutionary and others more cautious. We found that one strand of the debate during the pandemic involved a focus on the scope of OS, emphasising OS as an integrative phenomenon (not just a set of loosely coupled components) and a conceptualization of OS that includes elements like engagement with actors beyond the academy and with other knowledge systems. These perspectives were significantly advanced by the influential UNESCO Recommendations on OS (UNESCO 2021). The more encompassing definition of OS used in this document, therefore, provides a useful framing with which to examine the debate associated with the pandemic. Doing so will help us to analyse where the discussion on OS was concentrated during the pandemic and what this might tell us about the direction of the debate in future.

Of the different elements of the UNESCO framing of OS, it is clear that the “pillars” of “open scientific knowledge” and “open science infrastructures” received most attention in the corpus of material we examined. There was a focus on issues such as open data sharing and preprinting, understandably, in view of the particular conditions created by the pandemic—the need for rapid, international collaboration to develop solutions to the crisis. As we have seen, in many ways the arguments advanced in favour of OS were not new, but rather reworkings of pre-existing arguments set in a new context, a context which seemed to intensify the need for OS. The fact that these arguments were not substantially new should not surprise us. If the arguments developed for OS over the last quarter of century were valid, new circumstances would not be expected to rewrite the arguments, but rather further illustrate them. That seems to have been the argument of its advocates used of OS during the pandemic and is consistent with the idea of COVID being a “test case” for OS.

During the period of intensified debate in the COVID crisis, discussions about the value of OS revealed varying perspectives. Advocates of openness deployed arguments and rhetoric around the benefits of OS to which they felt the pandemic gave a sharper edge. Key to these advocates’ arguments was the perspective that COVID-19 created clearer lines of sight between OS practices and the benefits which accrue for society. It became much easier to argue that OS was good for society in general, particularly in areas such as healthcare. The argument was also made that the pandemic illustrated that OS was good for science, with a focus on instrumental arguments, like improving the speed and efficiency of science. At the same time, the pandemic gave rise to a greater sense of urgency to calls for OS. All of this created a major opportunity for advocates, many of whom were long-standing supporters of OS, but who were now able to argue even more assertively that there should be a decisive shift to more open ways of working.

Whilst many actors advocating OS in the pandemic were pre-existing advocates, a much larger group were content to work within a more open environment during the pandemic (with, for example, all COVID-related outputs made freely available by most publishers) for pragmatic reasons—it helped them work more efficiently or effectively. Some of these pragmatists were involved in responding to the health emergency and may have become convinced of the benefits of OS, seeing it work as it did (Horby, 2022). Some policy makers also seem to have recognised the value of OS more clearly in the context of the pandemic. That was the argument explicitly made in the Nelson (2022) memo, for example, which constitutes a step change for OS policy in the USA.

However, the same issues that gave greater confidence in OS for advocates and enabled the work of pragmatists also fuelled critique of OS among its sceptics. Scepticism about OS was often concentrated on issues that became particular focuses of debate, such as data sharing and preprints, and on issues such as quality of information within and beyond the scientific community. It is in these areas that we found the most sceptical voices about OS, raising concerns during the pandemic. There were, however, very few outright opponents to OS evident in the debate.

The issue of misinformation was an important connection with the third pillar of OS as defined by UNESCO, “open engagement of societal actors”. The relationship between the latest science and the public was a major issue, as might be expected given the unprecedented features of the response to the pandemic. Concerns about the quality of information distributed in an OA form and how this might be misinterpreted or misused received attention. “Other societal actors” could include clinicians providing COVID healthcare or pharmaceutical companies developing vaccines, as well as policymakers and journalists. The pandemic arguably provided an illustration of how such engagement could work, as well as the challenges and risks it creates, not least the danger of science being misunderstood or misused.

The fourth pillar of OS in the UNESCO recommendations, “open dialogue with other knowledge systems”, including Indigenous knowledges, is perhaps an aspect of OS that was, and remains, the least discussed and accepted component of OS, despite its inclusion in this influential document. This issue seemed to prompt little discussion in our corpus, however. In fact, the relationship between the pandemic, OS and Low-Income countries received little attention. Issues such as multilingualism and Indigenous knowledges received even less. Instead, attention was predominantly focussed on the Global North and the science system which is dominated by countries and institutions in the Global North, even though the corpus included contributions written in Spanish and Portuguese, both major languages in Latin America. Solutions developed to solve the health crisis in the Global North were often assumed to be directly transferable. Perhaps by mostly focusing on familiar arguments about OS (such as the “pillars” of “open scientific knowledge” and “open science infrastructures”), any trends of asking questions about the value of OS for the Global South were stifled, even though Low-Income countries were severely affected by COVID. So, rather than the pandemic having an entirely progressive impact on the case of OS, some of the deeper questions about global equity did not feature in the debate about OS as prominently as might have been expected. This may still change, as parts of the global community that backed the UNESCO Recommendation seem to want. Like the rest of the debate on OS during the pandemic, the impact for the long-term remains to be seen. What seems clear at this stage is that the case for OS is commonly seen to have been strengthened by the pandemic and the attention OS received during this time increased. What that means for policy and practice will become increasingly apparent as we move into the post-pandemic period.