Open science in sociology. What, why and now.

WHAT

By now you’ve heard the term “open science”. Although it has no global definition, its advocates tend toward certain agreements. Most definitions focus on the practical aspects of accessibility.

“…the practice of science in such a way that others can collaborate and contribute, where research data, lab notes and other research processes are freely available, under terms that enable reuse, redistribution and reproduction of the research and its underlying data and methods.”


FORSTER, open science teaching resource

Some definitions enter the realm of ethics, feminism and social justice.

“…to imagine and design inclusive infrastructures, practices, and workflows for scientific practice that intentionally enable meaningful participation and redress (these new) forms of exclusion.


Denisse Albornoz,OCSDNet

Whatever the ontology, open science is inevitably something that challenges the status quo in science. Usage of term indicates there is something undesirable about science, otherwise advocates would simply advocate “science”.

The “open” part of the concept refers to any number of things depending on whom you ask. Commonly it means:

Open access – making the results of scientific techniques, research and theory accessible to everyone; as opposed to only in paywalled journals.

Transparency <open process> – making all methods, code, data and any biases or conflicts of interest known before and after the research is conducted. So long as doing this does not harm human subjects or violate any laws.

Open source – on the technology side of science, all programs, apps, algorithms, tools and scripts should be transparent and usable by others. This means that when a scientist develops a new technology, anyone else’s technologies can interact and interface with it. Moreover, anyone can modify the technology to better suit their own needs.

Open academia <open communication/democracy/feminism> – allowing anyone to participate in academia. That inequalities, prejudice and domination that take place in the social world are eliminated from the academic world, if not that the academic world has a goal of eliminating them in the social world. That everyone has the same place in scientific discussions, and no science is conducted by pressuring others or taking advantage of existing power structures. That no science takes place in secret, except for research that requires obfuscation for its completion.

Again, the definitions can cover a broad range. The above are just a snippet, although they strike me as the most common usages; except for ‘open academia’, this is reserved for certain justice motivated scholars.

WHY

Although I do not proclaim to be the arbiter or knower of right or wrong in academia (and life in general), the following facts seem wrong to me.

Double-work and the co-opting of journals

Scientists provide their work as editors and reviewers, because the peer review and publication process is the centerpiece of all of science. Peer reviewers and editors are the only consistent form of quality control in science. The academic journal was a functional response to previous forms of knowledge transmission that required direct scientist/practitioner to student interactions which were geographically limited and reached a very narrow audience.

The journal made it possible to transmit knowledge across the globe. Moreover, the journal reduced the simultaneous discovery and re-discovery problems of science, because no one could prove they discovered something first, and others worked on problems that were already solved unknowingly. It represents one of the first ‘open science’ movements because it was driven by the idea that science was at an impasse and could only move forward through transparent and open exchange of ideas arbitrated by being part of the public record through publishing.

Ironically, the journal format came full circle and began to undermine science. After over two centuries of journals run by non-profit academic associations, for-profit publishing houses began ‘offering’ their services to meet the growing global demand for journals and their content and the rising costs of editing and distribution. In many cases, these publishing houses were able to purchase the journals by offering the academic societies the exclusive right to determine what went in them. Within just 30 years, five conglomerates owned the titles, content or certain features of over 50% of all journal articles published globally.

The content, as always, is still a product of the scientists and the voluntary work of editors and peer reviewers. The publishing houses make large profits, but pay nothing to these workers. The editors and peer reviewers earn their income from universities mostly. The very universities that pay high fees to purchase the right to provide the journals in their libraries. This is a double tax on the universities – paying the producers of content to produce and then paying the distributors of that content to consume it. The content does not change at any point in between these two forms of payment, in other words, the publishers do not add any scientific value to this content.

Matters got even worse with the publishing houses over the past decades. As creative and deceitful capitalists, the publishing houses realized they could generate even more profit by collaborating with the private sector. For example pharmaceutical companies’ profits were directly determined by the findings of studies published in journals. Pharmaceutical companies, or any companies whose profits were determined by the outcomes of scientific experiments, would be willing to invest in shaping those outcomes if they could. Enter a novel concept pioneered by Elsevier: selling journals or journal space to private companies to boost their profits. Win-win for them.

Publish-or-perish begets questionable research practices

Thanks to the advent of the scientific journal, knowledge could be evaluated, used and further transmitted across space and time. The utility of the journal and other forms of academic publication such as books, proved so effective that they became the primary source for others to evaluate the importance of scientists and their work. This gave rise to the norm we are all familiar with, publish-or-perish.

In a survey of psychologists, John et al. (2012) found that 50% claimed they had selectively reported studies that supported their hypothesis (as in, selectively excluding those that didn’t). Moreover, 35% admitted to reporting unexpected findings as having been predicted from the start. Nearly 2% outright admitted to faking data.

Publish-or-perish and questionable research practices have a causal relationship. Except for occasional sociopathic or psychotic individuals, there is no reason for a scientist to engage in questionable research practices. No reason, except scientists’ very existence on scientists may depend on it. So many studies in reality lead to results that go in all directions, support the null or (most importantly) do not provide groundbreaking new results.

Through the peer review and editorial process, journals select studies that are path-breaking. Studies that will move knowledge forward and be of the greatest interest to readers. When faced with prospects of not getting tenured, not getting grant funding and being forced out of academia, a human’s (scientist’s) rational calculations change. Suddenly, rounding that p-value from 0.054 to < 0.05 or even adding some cases to the data becomes a cognitively defensible decision.

Like any profession, science is competitive. Those who publish more, or get more citations to their publications tend to get ahead. Those who don’t, don’t. Professional athletes use incredible tactics to gain competitive advantage. Of course steroids are well-known, but other tactics are much harder to detect. For example, endurance athletes often use blood transfusions to boost recovery and performance. This is what it means to be human, scientist or not.

One of the most radical events in the social and behavioral sciences is Diederik Stapel’s entire career faking data and results that were published in at least 54 articles that consumed millions of Euro in funding. It took almost two decades for critics and whistleblowers to finally out him. Psychology is not alone. In political science LaCour and Green published a study in Science that attitudes toward gay marriage could be changed if heterosexual people listened to a homosexual person’s story, but it turns out LaCour fabricated results of a follow up survey that never took place as uncovered by Broockman. In economics Reinhart and Rogoff published numerous studies identifying a negative impact of high debt rates on national economic growth, when in fact several points in their dataset had conspicuously missing values. When these values were added there was no longer support for their claim as identified by Herndon, Ash and Pollin.

I suspect that most questionable research practices are not intentional. The sociopathic (~psychotic) Stapel’s of the world are rare. This pressure to find a job after doing doctoral studies and then to get tenured, means a trade off between conducting science in its ideal form – so learning as much as possible about the existing literature on a subject, mastering the necessary methods to perform the research and executing the research, possibly with several iterations, and facing the prospect of null results – with science in a form that will lead to publication as fast as possible.

This ‘fast as possible’ leads to amateur science. For example, in the rush to get my first publication I attempted to use “multiple imputation”, but lacked the time to properly learn this method. Instead I simply generated several datasets and averaged them into one and re-ran the analysis on this one. This was not an intentional misuse of a method. It is a questionable research practice as a result of context. Think about matrix algebra. It is the basis of many advanced statistical techniques regularly used by social scientists. How many of us have a strong grasp of matrix mathematics? I don’t. And yet I’ve published several studies using structural equation modeling.

WHAT & WHY in SOCIOLOGY

I am aware of nothing about sociology that suggests it needs a special adaptation of open science. Most research cannot be strictly delineated as sociology or not sociology anyways. The boundaries of a discipline, especially within the social sciences, exist mostly in the institutional structure of universities. Eliason suggested that sociology is unique because it overemphasizes quantitative techniques, has needlessly long articles, lacks writing for the popular press and emphasizes research at the expense of teaching. In my experience the previous sentence perfectly describes all social and behavioral science disciplines at once. Even article length, something I thought might be peculiar to sociology, is not special. Political science and management research have very long articles. Consider that and ASR and ESR for example, limit words to 9,000 and 8,000 or less – this is relatively average if not short for social science.

Actually, I would argue the most unique thing about sociology at the moment relates to open science. Two points in particular: (A) that sociology has not had the same incredible scandals as other disciplines and (B) that sociology lags behind other social sciences in promoting open science.

A lack of scandals, not scandalousness

Could sociologists be more scientific and ethical in their research behaviors than those in other disciplines? Given identical institutional and career structures that favor productivity and innovation over replicating or checking each other’s work, I doubt it. Sociology journals and their editors, for example, rarely retract articles despite evidence of serious methodological mistakes. Carina Mood once accurately pointed out mistakes in the interpretation of odds-ratios in some American Sociological Review articles, but the editors refused to publish her comments, much less consider retractions. She shared her exchange with ASR in an email to me and discusses some of it in a working paper. An exceptional recent event was the retraction of one of Legewie’s sociological studies, but this required he himself to initiate the retraction after someone pointed out errors in his work. Until 2020, the Retraction Watch database (www.retractiondatabase.org) listed no retractions from the top sociology journals, and only two among the well-known, one in Sociology and another in Social Indicators Research.

This year, something new happened. Five articles published in Social Problems, Criminology, and Law & Society Review were retracted. These articles had the common co-author Eric Stewart. It turns out that the data he provided were faked. There is no other logical conclusion that this after exceptionally rigorous work by Pickett (a co-author of Stewart) provided evidence that the Stewart studies had consistently incorrect means and standard deviations, unverifiable surveys (sources, methods, original materials), magically changing case numbers despite identical statistical results, sometimes half the data had duplicate cases and impossible clustering structures in the data.

As an aside, one of Pickett’s findings was that the data had non-uniform terminal digit distributions. This means that the right-most digits in the reported statistics differs markedly from a uniform distribution. In particular, at the third-digit numbers should be uniformly distributed with 0-9 appearing roughly 10% of the time. In one of the papers, zeros appear less than 2% of the time. If you are considering faking data, keep in mind that it is roughly impossible to do it in a way that cannot be detected by careful investigation. Any algorithm used to generate results (even copying and pasting) leaves is statistical marks.

Perhaps we sociologists should be partly relieved, as this is just confirmation that we are as much a part of social science and its problems, as any other discipline. However, the Stewart retractions which should have been breaking news for sociology, went mostly unnoticed. The results of the investigation leading to the retractions is not published in a flagship sociology journal where it belongs. Instead it appears in Econ Journal Watch – something unlikely to be read by any sociologist. Moreover, the retraction notices from the original journals do not cite outright fraud. Stewart continues to promote his work in print claiming the main findings still hold, and several other of his studies with similar irregularities have not been retracted.

Another, extremely important event was a case of ethnomethodological research conducted by Lindsay, Boghossian, and Pluckrose in the mid 2010s. This is sociological self-examination at its best, although their backgrounds are mostly outside of the discipline of sociology. They wrote a series of 20 papers presenting fake results and making arguably unethical claims. They invented the papers to mimic the style of articles published in journals well-known for sociological research on topics of identity, hegemony and marginalization. Seven of their papers were published or had revise and resubmit recommendations before whistleblowing forced them to cancel the project. Some highlights: one paper contained sections from Hitler’s Mein Kampf. Another suggested men should be trained similar to dogs to prevent rape, and a third that white men should be forced to sit in chains on the floors of university classrooms, instead of normal desks. I am not commenting on the merit contained in these ideas, only that they all contained faked data, non-existent methods or conclusions not supported by the data. That these studies easily flew under the radar of a number of high impact journals points out how easy it is to publish without doing the necessary research work.

Lagging behind closed doors

October 6th, 2020. I entered the search terms “open science” (with quotations to search the exact phrase) and “sociology” (with quotations to only return results that contain the word) into Google Scholar. Six pages of results without a single sociology journal. On page 7, Merton’s “Priorities in scientific discovery: a chapter in the sociology of science” appears. Publication date 1957.

In 1973, Wilson, Smoke and Martin found that 80% of studies published in the top three sociology journals of that time rejected the null hypothesis, in other words they had p-values below a threshold. This suggests publication bias, if not p-hacking. Sahner (Table 5) analyzed all article submissions to the Zeitschrift für Soziologie, 1972-1980. Of those that contained significance tests, 70% were significant at p < 0.05 suggesting that authors prefer to submit significant results. More recently, Gerber and Malhotra (2008) reviewed articles published in American Journal of Sociology, American Sociological Review and The Sociological Quarterly, and specifically looked at the boundary of t = 1,96 (i.e., p<0.05) to find that as many as 4-out-of-5 studies were ‘significant’. This suggests publication bias as well. Sociology has yet to have a systematic review of p-hacking by comparing p-values within ‘significant’ results. Meanwhile psychology and political science for example are teeming with papers on “p-hacking” and “publication bias”.

Sociology is rather intransparent. An estimated 78% of the major sociology journals have long-standing transparency policies. Unfortunately, these policies are mostly artifacts on paper without much enforcement. For example, only 37% of sociology articles published in the mainstream journals between 2012-2014 include shared data and/or materials. In 2015, a small group of sociologists tried to obtain materials from the authors of 53 prominent sociological studies. They obtained these from just 19%, and only 20% of all the authors they contacted bothered to respond despite several requests. This suggests sociologists are free to hide the data and materials that led to their findings without recourse, despite such guidelines.

Other disciplines have embraced the Transparency and Openness Promotion Guidelines (TOP). The TOP guidelines with help of the Center for Open Science support journals to improve science. Journals can become signatories of TOP, and in doing so they either adopt and enforce new transparency guidelines, or certify that they already meet certain transparency standards. Most of the top psychology journals and several political science journals signed on. Other major journals such as the Journal of Applied Econometrics and later the American Economic Review adopted their own enforced transparency guidelines.

Until 2017, the only higher ranking sociology journals that signed TOP were Sociological Methods and Research and American Journal of Cultural Sociology. In 2017, Elsevier dictated that all its journals adopt guidelines and this added Social Science Research to the list. At the time of writing this, the flagship journals American Journal of Sociology and American Sociological Review neither signed TOP nor enforce their own guidelines. Of top German sociology journals, the Kölner Zeitschrift für Soziologie und Sozialpsychologie is the only signatory.

If intransparency is pervasive in sociology, then research cannot be (a) checked for errors, (b) reproduced or (c) simply critiqued. Even when exact reproducibility is not the goal, as often is the case with context-specific interpretive research, most research methods remain shrouded in mystery. This requires readers to take a giant leap to trust what others report. Part of the problem is that sociologists express little interest in reproduction or checking others’ works. There are few replications in the history of sociology, and if anything, they decreased over time until recently. For example, searching the articles in American Journal of Sociology and American Sociological Review reveals 22 replication studies from 1950-1980 and only 8 from 1981-2010.

Something telling about a lack of willingness to open sociology comes from sociology’s most ‘powerful’ society, the American Sociological Association. They collectively petitioned the US government to not make data transparency a requirement attached to grant funding in 2019.

NOW

What to do about it? Here are some simple steps to consider.

Transparency

Make all the materials – research design, methodological steps, data (when legally and ethically possible), analyses, conflict of interest and any software code – available online. The practical reason is that others can follow your work and expand it in the future. Doubly practical is that you don’t need to respond to email requests for your materials. So long as you are not a deceitful sociopath, you want others interested in your work and to replicate your work. Even if a study, seems to ‘prove you wrong’, the fact that it replicated your work is evidence of how important your work is and the topic of study. You are a piece of a much larger community of knowledge construction. Constructive exchange can lead to collaboration with critics to generate better future research without personal conflicts.

The immediate value of transparency is that being transparent forces you to be careful. Knowing everything will be public information increases the value of attention to detail. Put in its converse: not sharing your workflow publicly can indirectly foster lower quality standards, in addition to creating possibilities for misconduct. All this enables rather than hinders knowledge, and increases inter-researcher trust.

Transparency should not be much extra work. During the research process you should take high quality notes for yourself. You will often return to your data and research in the future and thus need those notes. This is a best practice with or without sharing your work. When you engage in this best practice, you have a deep familiarity with your data and can draw meaningful conclusions and easily redact identifying characteristics in your data in the case of qualitative research. In case you cannot share data, you can still reveal the design and expectations; or allow controlled access to the data. Human subjects must be protected at all costs, and yes this often means data sharing is not possible .

The ‘transparency work’ of the qualitative research process can be reduced by software platforms that provide semi-automated annotation and coding. Even if you do not share data, you can build an open workflow from the beginning that allows others to understand every step of the data generating process. However, this work can also be extremely tedious and the incentives not immediately clear. More fruitful discussion if not research assistant funding is needed in this area moving forward.

If you are using quantitative methods, immediately stop hiding your work. If you ran 100 models and 99 did not support your hypothesis, then this is your finding. If a journal does not want to publish this, point the editors and reviewers to the importance of null results and the problems of publication bias. If they still refuse, consider boycotting this journal and sharing your negative experience in public.

Preregistration

Preregistration can drastically reduce bias and hacking prior to collecting data. When you clearly outline your plans including how you will analyze the data, before conducting the research, there is little room for hacking so long as you stick to the plan. Moreover, preregistration can be done directly with a journal although sociology journals are laggards here because they generally do not offer this option. In a preregistration, even if you just put an pre-analysis plan or research design and goals online, you must think much harder about factors such as meaning, causality, inter-subjectivity and ‘how the world probably works’. You cannot hide behind results in this process and therefore you must anticipate counterarguments and explore counterfactual logic. This improves the clarity of theory and research, creating an immense gain in efficiency and effectiveness.

Regardless of the methods you use there are many opportunities to take advantage of preregistration. Some forms of qualitative research, for example those involving grounded theory and interpretivist methods, require decisions during the research process that cannot be foreseen. This uncertainty can be outlined in a preregistration stating explicitly when flexibility is and is not admissible. Moreover, simply putting a qualitative research plan online prior to conducting the research is equivalent to a pre-analysis plan. This research design need not compromise your data collection work because you can register the plan on a platform like the Open Science Framework and then embargo it, so that it is preserved but not made public until after the research concludes. Some scholars using quantitative methods might assume that preregistration is not possible because they work with secondary survey data. But the regularity and release of these survey data are known in advance, and these scholars can preregister their studies before the next round of data are collected with the knowledge of which questions and countries will be available.

Decommodify science

The central functions of the scientific publishing industry are printing and disseminating knowledge, which historically solved a problem of how to share knowledge across universities and countries. The business functions of publishing, however, come with harmful byproducts. Publishing firms extract profits from scientists twice. First, scientists provide free labor in the form of editing and peer reviewing, in addition to producing the results for the articles to be printed. Next, researchers, or their employers, must purchase the product of their own labor; labor not paid for by the publishers. The journal article as a product comes at a high cost, and often only in packages of journals meaning that universities have to pay for extra material their scholars do not use.

Sometimes publishing houses neglect science in favor of profits, but Elsevier has been particularly problematic. They sponsored weapon fairs, created and sold ‘fake’ journals to pharmaceutical companies to publish ‘results’ supporting their drugs, purchased the Social Science Research Network and created paywalls or removed legally shared working versions of articles, charge fees for open access articles, and actively lobbied against open access legislation (For a concise summary with links see Tal Yarkoni’s blog entry). This brought massive counter movements against Elsevier in the scientific community (for example, The Cost of Knowledge). You can take action and refuse to review for or publish with unethical publishers if you feel it is justified. Thus, you should inform yourself about the publishers. Your libraries are a source of information, because they deal with the business side of publishers.

If you are in Europe, check if your institution is a signatory of ProjektDEAL. A consortium of universities are collectively bargaining with publishers via ProjektDEAL demanding that publishers reduce fees and eliminate the double paying of universities. The primary objective is that publishers sign country-wide subscription agreements that enable access for all universities at once. Wiley agreed to such a model and this marks a paradigm change. It indicates how the publishing industry looks in the future, so long as the OS Movement proceeds. If you are not in Europe, consider starting a similar initiative, for example the entire University of California system of 10 universities, 5 medical centers and several research institutions that collectively produce roughly 10% of the world’s academic publications recently followed ProjektDEAL and boycotted Elsevier.

You can work around the publishing business. Prior to submitting an article or after it is published, you have the right to share a preprint – a draft of the paper you share publicly so long as it is not published elsewhere or sold for profit. Posting preprints reduces the power that publishing firms have over science, in addition to giving others immediate access to your work. But simply posting preprints on your academic website is not open enough. Use a preprint service, for example through the Open Science Framework, to ensure that your preprints appear in search engines such as Google Scholar. SocArXiv for example, is the go to location for sociology. This enables scholars to find and directly access research results based on the words they contain, uninhibited by paywalls – a crucial aspect to practicing sociology in the Global South. Preprint services are free and open access.

Die ‚Novel Coronavirus‘ Pandemie und die Grenzen von Open Science

Deutsche Übersetzung von Novel coronavirus pandemic and the limits of open science (8. April 2020).

Am 30. Januar 2019 erklärte die WHO einen ‚Global Health Emergency‘ basierend auf Hinweisen auf ein Virus, das sich schnell verbreitet. Das Coronavirus aus der SARS-Familie (Sars-CoV-2, und die Krankheit Covid-19). Die Beweise, mit denen die WHO diesen Notfall erklärte, stammten fast ausschließlich aus chinesischen Daten.

Die chinesischen Daten zeigten im Januar eine alarmierende Ausbreitungsrate, wie in Abbildung 1 dargestellt. Ohne die chinesischen Daten hätte die WHO wenig Anlass zur Sorge gehabt, da in allen anderen Ländern zusammen kaum 90 Fälle bekannt waren und kein einziger Todesfall.

Abbildung 1. Die Verbreitung von Covid-19, die zur Notstandserklärung der WHO am 30. Januar führte. Johns Hopkins Daten.

Anfang Januar ergriff die chinesische Regierung Maßnahmen, um Nachrichten und Daten1 im Zusammenhang mit dem Virus zu blockieren. Trotzdem gelang es chinesischen Wissenschaftler*innen, offenen wissenschaftlichen Praktiken zu folgen, einschließlich des Teilens partiell-genetischer Sequenzdaten mit der Welt. Dies ermöglichte der WHO, geeignete Maßnahmen zu ergreifen, und befähigte  Wissenschaftler*innen in Deutschland, Tests zur Identifizierung des ,novel Coronavirus‘ zu entwickeln. Das deutsche Team veröffentlichte seine Methoden am 13. Januar auf der WHO-Website. Technologie und globale Kommunikation haben sich zu einem Punkt entwickelt, an dem Regierungen den freien Informationsfluss verlangsamen, aber nicht stoppen können.

Die gemeinsame Nutzung aller Daten und Erkenntnisse ist die beste Form der Wissenschaft, wird aber nicht immer praktiziert. Die Open Science Movement hat das Ziel, dies zu ändern. Wenn jeder auf der Welt gleichermaßen Zugang zu Theorie, Methoden, Daten und Ergebnissen aller anderen wissenschaftlichen Forschung hat, steigen Qualität und Effizienz exponentiell an. Dies zeigt sich in den offenen wissenschaftlichen Praktiken hinter dem globalen Kampf gegen Covid-19, die Leben retten und retten werden, möglicherweise Millionen von Leben.

Abbildung 2 ist eine Simulation, die vorhersagt, wie viele Menschen in einem bestimmten Land als Ergebnis des Zeitpunkts der staatlichen Intervention an dem Virus sterben würden. Intervention heißt wann die Regierungen die von der WHO empfohlenen Vorgehensweisen befolgen, wie z. B.: Anweisungen für den Aufenthalt zu Hause, Durchführung umfassender Tests und Quarantäne für diejenigen, die positiv auf das Virus getestet wurden und die, mit denen sie in Kontakt waren. “Tag 0” in Abbildung 2 ist der Moment, in dem mindestens 3 symptomatische Fälle pro Million Menschen auftreten, normalerweise etwa 2 Monate nach dem ersten Fall in einem Land, aber natürlich viel schneller, wenn mehrere Fälle gleichzeitig auftreten.

Abbildung 2. Die Auswirkungen staatlicher Interventionen auf die Reduzierung der Todesfälle durch Covid-19. Quelle: Gabriel Goh, und eigene Berechnungen des Autors (* vorhergesagte Todesfälle)

Der Leser sollte bedenken, dass Abbildung 2 eine vereinfachte Simulation ist. Die Realität ist äußerst komplex. Insbesondere gehen die Regierungen nicht an einem Tag vom normalen Betrieb zur vollständigen Stilllegung der Gesellschaft über, dies geschieht normalerweise schrittweise. Diese Simulation basiert jedoch auf den bekanntesten Modellen der prädiktiven Epidemiologie und zeigt, wie selbst ein Tag der Unentschlossenheit Tausende von Menschenleben kosten kann.

Als Reaktion auf diesen Ausbruch in China und das rasche Auftreten von Covid-19 weltweit folgte Südkorea den standardisierten „Emergency Operating Procedures“ der WHO. Das heißt: möglichst viele Personen testen, alle Fälle isolieren, Reisen und Versammlungen beschränken, nicht notwendige Geschäfte schließen. Das Virus war eingedämmt und nur 200 Menschen starben. Natürlich haben frühere Virusausbrüche in Südkorea die Bereitschaft verbessert. Ebenso war Deutschland gut vorbereitet, weil es schnell Tests entwickelt hatte  und weil es aus den Erfahrungen Italiens als Europas „Ground Zero“ gelernt hatte.

Grob gesagt hat Italien um den 15. Februar herum die Schwelle für „Tag 0“ in Abbildung 2 überschritten. Als Land war es am wenigsten vorbereitet, weil es das erste in Europa war und ein Ort ist, zu dem Menschen aus der ganzen Welt als Touristen, wenn nicht als Fußballfans, strömen. Somit ist der Fall Italiens keine Geschichte eines großen Versagens der Regierung, auch da es Gründe gab, dem chinesischen Fall misstrauisch gegenüberzustehen.

Die Schwelle für “Tag 0” lag in Deutschland um den 2. März herum, und “Tag 0” war um den 8. März herum in New York, zumindest auf dem Papier. New York begann jedoch erst am 1. März mit dem erfolgreichen Testen von Personen, da die Anfang Februar veröffentlichten CDC-eigenen Testkits fehlschlugen. ‘Tag 0’ in New York war wahrscheinlich Mitte Februar oder früher. Dennoch hätte New York aus „pandemischer Sicht“ noch viel Zeit gehabt, Maßnahmen zu ergreifen. Der Rest der Welt hatte seit Ende Januar, dank des offenen Datenaustauschs auf der WHO-Website, genaue Tests durchgeführt. Dies geschah jedoch weder in New York noch in den USA als Ganzes. So wurde New York völlig unvorbereitet getroffen, aber nicht, weil das Virus überraschend aufgetaucht  war.

In Kombination mit den Daten aus China, Südkorea und mehreren anderen Ländern erklärte die WHO am 12. März, dass der globale Notfall nun eine „Global Pandemic“ sei. New York hatte den Ausnahmezustand verhängt, aber erst ab dem 20. März Anordnungen für den Aufenthalt zu Hause erteilt. Erst eine Woche später wurden die meisten Schulen geschlossen und die Polizei autorisiert, diese Anweisungen durchzusetzen (der blaue Pfeil um „Tag 31“ in Abbildung 2). Trotz massiver offener wissenschaftlicher Bemühungen, die durch die WHO kanalisiert wurden, haben New York und ein Großteil der USA offensichtliche wissenschaftliche Beweise und Vorhersagen einfach nicht beachtet. Dies ist umso schockierender, als Seattle und nicht New York in den USA „Ground Zero“ war. Der gesamte Bundesstaat Washington hatte frühzeitig und erfolgreich Sofortmaßnahmen ergriffen.

Die Überprüfung des Versagens von Ländern, Staaten oder Städten, vor oder am 30. Januar (globaler Notfall) oder 12. März (Pandemie) sofort drastische Notfallmaßnahmen zu ergreifen, ist nicht Gegenstand dieses Blogposts. Dank Open Science Praktiken, der WHO und mehrerer Partnerorganisationen und Websites hatte die Welt Zugang zu denselben Daten und Kenntnissen darüber, wie man auf das Virus testet.

Die Botschaft, die ich vermitteln möchte, ist, dass Open Science nicht ausreicht. Ihre Grenzen liegen in den Regierungen. In vielen Ländern hat die Wissenschaft wenig Platz in der Entscheidungsfindung der Regierung. Dies ist vielleicht in einem dysfunktionalen autoritären Regime verständlich, in dem fast alle politischen Entscheidungen getroffen werden, um die Macht aufrechtzuerhalten und zu konzentrieren. Dies ist sicherlich ein Grund dafür, dass die schlimmsten Schrecken des Virus in Afrika südlich der Sahara und in Zentralasien noch bevorstehen. Aber es ist schockierend in Demokratien, in denen es eine Schar von Wissenschaftler*innen und Agenturen gibt, die die Regierung dabei beaufsichtigen und beraten sollen, was zu tun ist, um ihre Bevölkerung zu schützen.

Die Vereinigten Staaten hatten reichlich Informationen darüber, dass sich Covid-19 in den USA befand und sich schnell verbreitete, wie man wirksame Tests entwirft und was genau zu tun ist, um die Ausbreitung des Virus und die Zahl der Todesopfer zu verringern, Monate vor dem Ergreifen größerer Maßnahmen – dieselben Informationen, die der Staat Washington zur Eindämmung der Ausbreitung nutzte. Aber diese wissenschaftlichen Informationen, die in einem Umfang und einer Geschwindigkeit geteilt wurden, die in der Weltgeschichte noch nie zuvor gesehen wurden, reichten einfach nicht aus.

Die Open Science Movement hat ethische Grundsätze, die ihrem offenen Zugang, den offenen Daten, den offenen Methoden und Empfehlungen zum Austausch von zugrunde liegen. Es ist nicht nur so, dass offene wissenschaftliche Praktiken die Wissenschaft zuverlässiger und effektiver machen. Sie fördern soziale Gerechtigkeit oder wissenschaftliche Gerechtigkeit, wenn Sie so wollen. Wenn jede*r Wissenschaftler*in auf der Welt auf alle Informationen zugreifen kann, über die jede*r andere Wissenschaftler*in auf der Welt verfügt, besteht wissenschaftliche Gleichheit. Während reiche Universitäten Elsevier boykottieren, können sich ärmere Universitäten nicht einmal ein Abonnement leisten. Open Access würde also der Welt eine globale Nord-Süd und eine dotierte vs. nicht dotierte Universitätsgleichheit bringen. Aber es kann denjenigen, die potenzielle Virusopfer sind, keine Gerechtigkeit bringen.

Im Fall der Covid-19-Pandemie schien die offene Wissenschaft zunächst das Gezänke und die Tiraden der Regierungen zu untergraben, konnte aber nur an der Tür klingeln. Einige Regierungen weigerten sich einfach, die Tür zu öffnen und Maßnahmen zu ergreifen. Dies wirft die Frage auf, ob die Open Science Bewegung politische Handlungsprinzipien verabschieden muss, die über Maßnahmen zur Förderung von Transparenz und Reproduzierbarkeit hinausgehen. Muss die Open Science Bewegung die Regierungen dazu drängen, administrative, wenn nicht verfassungsrechtliche Verfahren einzuführen, die die Regierungen bei einer Naturkatastrophe oder einem Notfall wie einem Hurrikan oder einer Pandemie den Wissenschaftler*innen gegenüber rechenschaftspflichtig machen?

Ich sage ja aus ethischer Sicht. Aber es ist nicht so einfach. Sobald wir anfangen, Dinge wie Verfahrensreformen voranzutreiben, werden tiefsitzende Sonderinteressen einbezogen und es wird hässlich. Als Wissenschaftler*innen sind wir wahrscheinlich nicht für Schlammschlachten und politisches Manövrieren geeignet. Ganz zu schweigen davon, dass wir umso weniger Zeit für die Wissenschaft haben, je mehr Zeit wir für Lobbying aufwenden. Einige von uns haben die Fähigkeit, die Bewegung zu führen und Regierungen zu beeinflussen, aber die meisten von uns sind schlecht gerüstet, um die Mächte zu bekämpfen, die hinter der Politik stehen.

Das wirft die Frage nach dem Endspiel auf: Reicht es aus, den Regierungen die richtigen Antworten zu geben, auch wenn sie sie ignorieren? Haben wir unsere Pflicht als Wissenschaftler*innen erfüllt, wenn wir nur vor der Haustür auftauchen und Regierungsbeamte entscheiden lassen, ob wir eintreten dürfen?

1 Der ursprüngliche Nachrichtenartikel wurde von der Website der chinesischen Nachrichtenagenturen gelöscht, kann aber im Internet Archive gefunden werden.

2 Quelle: Goh, Gabriel. „COVID Epidemic Calculator“. Tag 0 ist mindestens 3 symptomatische Fälle pro Million Menschen, was bedeutet, dass aufgrund der Inkubationszeit möglicherweise Hunderte infiziert sind. Für Vorhersagen verwendete Parameter: 106 mio. Bevölkerung, ein einziger Erstfall, Ansteckungsgefahr pro Person von 2,2, Übertragungsrate 0,73, Inkubationszeit 5,2 Tage und Sterblichkeitsrate 2%.

Ein Hinweis von mir: Ich habe versucht, die empirischen Beweise und den historischen Zeitplan so genau wie möglich zu erfassen, aber alle Fehler in diesem Blog-Beitrag sind meine eigenen. Ich bin dankbar für die Kommentare von Lisa Heukamp.

Novel coronavirus pandemic and the limits of open science

German version available.

On January 30th, 2019, the WHO declared a global health emergency based on scientific evidence of a rapidly spreading coronavirus from the SARS family (Sars-CoV-2, and the disease Covid-19). The evidence the WHO used to declare this emergency came almost entirely from Chinese data.

The Chinese data demonstrated an alarming spread rate in January, as shown in Figure 1. Without the Chinese data, there would have been little cause for alarm as all other countries combined had barely 90 known cases in that period, and not a single death.

Figure 1. The spread of Covid-19 leading to the WHO emergency declaration Janurary 30th. Johns Hopkins data.

In early January, the Chinese government took measures to block news and data1 related to the virus; however, Chinese scientists still managed to follow open science practices (updated news on this here) including sharing partial-gene sequence data with the world. This allowed the WHO to take appropriate measures and enabled scientists in Germany to develop tests to identify the novel coronavirus. The German team shared publicly their methods on the WHO website on January 13th. Technology and global communications have evolved to the point where governments can slow but not stop the free flow of information.

Sharing all data and findings is the best form of science, but not always practiced. The Open Science Movement has the goal of changing this. If everyone in the world has equal access to the theory, methods, data and results of all other scientific research, quality and efficiency increases exponentially. This is evidenced in the open science practices behind the global fight against Covid-19 that saved and will save lives, potentially millions of them.

Figure 2 is a simulation predicting how many people would die of the virus in any given country depending on when governments follow WHO recommended operating procedures, as in: issue stay-at-home orders, engage in widespread testing and quarantine both individuals with the virus and those they were in contact with. ‘Day 0’ in Figure 2 is the moment when there are at least 3 symptomatic cases per million people, usually about 2 months after the first case in a country but much faster if several cases arrive at once.

Figure 2. The impact of government intervention in reducing deaths from Covid-19. Source: Gabriel Goh & author’s calculations
(*indicates a predicted death toll)

The reader should keep in mind that Figure 2 is a simplified simulation. The reality of the situation is extremely complex. In particular, governments do not go from normal operations to full lockdown of society in one day, this usually proceeds in stages. Nonetheless, this simulation comes from the best known predictive epidemiology models and helps demonstrate how even one day of indecision can cost thousands of lives.

In response to this outbreak in China and rapid appearance of Covid-19 globally, South Korea followed the WHO’s standard emergency operating procedures. Meaning: Test everyone possible, isolate all cases, restrict travel and gatherings, close non-essential businesses. The virus was contained and only 200 people died. Of course, previous virus outbreaks heightened their preparedness level. Germany was also well prepared given its rapid development of tests, and because they learned from the experience of Italy as Europe’s ‘ground zero’.

Roughly speaking, Italy crossed the ‘Day 0’ threshold in Figure 2 around February 15th. It was the least prepared as a country because it was the first in Europe, and a place where people from around the globe flock as tourists if not football fans. Thus, Italy’s case is not a story of major government failure, also given that there were reasons to be suspicious of the Chinese case.

The ‘Day 0’ threshold came around March 2nd in Germany, and ‘Day 0’ was around March 8th in New York at least on paper. But New York only started testing people with success around March 1st because the CDC’s own test kits released in early February failed. This left plenty of time, in ‘pandemic terms’, to source the accurate tests being deployed in the rest of the world since January. This did not happen in New York or the US as a whole. Thus, New York was caught completely unprepared but not because the virus was a surprise arrival.

When combined with the data from China, South Korea and several other countries, the WHO upgraded the global emergency to a global pandemic on March 12th. New York had issued a state of emergency but only gave stay at home orders as of March 20th. It was not until a week later that most schools were closed and police authorized to enforce these orders (the blue arrow around ‘Day 31’ in Figure 2). Despite massive open science efforts channeled through the WHO, New York and much of the US simply failed to heed obvious scientific evidence and predictions. This is even more shocking because Seattle, not New York, was ‘ground zero’ in the US. Washington State as a whole implemented early and successful emergency measures.

Reviewing the failures of countries, states or cities to immediately take drastic emergency measures before or on January 30th (global emergency) or March 12th (pandemic) is not the subject of this blog post. The world had access to all the same data and knowledge of how to test for the virus thanks to open science practices, the WHO and several partner organizations and websites.

The message I want to convey is that open science is not enough. Its limits are found in governments. In many countries, science has little place in government decision making. This is perhaps understandable in a dysfunctional authoritarian regime where nearly all political decisions are made to maintain and concentrate power. This is certainly a reason that the worst horrors of the virus are yet to come in sub-Saharan Africa and Central Asia. But it is shocking in democracies where there are throngs of scientists and agencies tasked with monitoring and advising the government on what to do to protect its people.

The United States had ample information that Covid-19 was in the US and spreading rapidly, how to design effective tests, and exactly what to do to reduce the spread of the virus and its death toll months before any major actions were taken – the same information Washington State used to stem the spread. But this scientific information, shared at a scale and speed not seen before in world history, was simply not enough.

The Open Science Movement has ethical principles underlying its open access, data, methods and sharing recommendations. It is not just that open science practices make science more reliable and effective; they promote social justice, or scientific justice if you will. When every scientist in the world can access all the information that every other scientist in the world has, there is scientific equality. While rich universities boycott Elsevier, poorer universities cannot even afford a subscription. Thus, open access would bring a global North-South and a endowed v. not-endowed university equality to the world. But it can’t bring justice to those who are potential virus victims.

In the case of the Covid-19 pandemic, open science looked to undermine the bickering and buffonery of governments at first, but it could only ring the doorbell. Some governments simply refused to open the door, to take action. This begs the question if the Open Science Movement needs to adopt principles of political action that extend beyond policies promoting transparency and reproducibility. Does the Open Science Movement need to push governments to adopt administrative, if not constitutional procedures that make governments accountable to scientists in a natural disaster or emergency like a hurricane or pandemic?

I say yes from an ethical stand point. But its not so simple. As soon as we start pushing things like procedural reform, deep-pocketed special interests get involved and it gets ugly. As scientists we are not likely suited to mudslinging and political maneuvering. Not to mention that the more time we spend on lobbying, the less time we have for science. There are some of us with the ability to lead the Movement and influence governments, but most of us are ill-equipped to combat the powers-that-be behind politics.

That brings up the end game question: Is it enough to give the right answers to governments even if they ignore them? Have we done our duty as scientists if we just ‘show up at the doorstep’ and let government officials decide if we get to come in?

1 The original news article was deleted from the Chinese News Agencies’ website but can be found in the Internet Archive.

2 Source: Goh, Gabriel. “COVID Epidemic Calculator“. Day 0 is at least 3 symptomatic cases per million people, meaning there are potentially hundreds infected given the incubation period. Parameters used for predictions: 106 mil. Population, a single initial case, contagiousness per person of 2.2, rate of transmission 0.73, incubation period 5.2 days and mortality rate 2%.

A note from me: I have sought to capture the empirical evidence and historical timeline as accurate as possible, but any errors in this blog post are my own.

.