The Scientific Process

A dear non-scientist friend of mine took me by surprise recently as I mentioned my reviewing a manuscript for a dermatology journal. “Are you getting paid for this?” Less than 24 hours later, the same innocent question came from my own husband. Both individuals are smart, very well educated, and not particularly obsessed with money.

What struck me was not only that they would ask the question, but also that the process was so ingrained in my way of living that I had no instant reply. It was as if someone had asked me why I breathe air every day. As I thought more and more about this, I realized that their question was completely logical, but that this was a good illustration of the lack of communication between the scientific community and the public.

Although they both had heard the term “peer-review”, neither my friend nor my husband had any way to know that this small unpaid step is part of a machine designed to ensure that the scientific process is followed so that discoveries are real and beliefs are anchored in fact rather than superstition. But what is The Scientific Process?

Science follows an iterative process that starts with observation, from which a theory is constructed. Note that although the word theory is often loosely interpreted as something that is made up out of thin air, scientific theories come from observed phenomena and are far from being just made up.

After this theory is advanced, it should be tested to verify its validity. And here is a very important part of the process: whatever they are, test results should be verified and confirmed by several sources. This means data must be both verified and replicated. No single piece of data is sufficient to confirm a theory. One paper is never enough to assert a definite fact. Knowledge is the result of the accumulation of many pieces of data coming together from many different experiments. If one or just a few of these pieces of data is wrong, the entire edifice could be compromised.

When the data from one of these experiments is submitted to a journal for publication (the traditional way to disperse data), the best way to check the results is to have several scientists from the relevant industry anonymously examine the experimental setup, validate the analysis method, and confirm that the data supports the authors’ conclusions. Reviewers do not see who the authors are, and the authors do not find out who reviewed their manuscripts. Rejection is common, correction systematic, and 9 times out of 10 re-submission is requested. Each paper is one small piece of the edifice, rare are the papers that revolutionize a field in one fell swoop.

So, one paper at a time, and one reviewer at a time, validity is ensured. This can only work if reviewers are objective, and that in turn can only work if they are unpaid and blinded. I and thousands of other reviewers do not get paid to review manuscripts. We review papers so that flawed science gets intercepted, fraud gets identified before it gets published, data is the most stringent it can possibly be, and language is clear. These are our responsibilities, and we take them seriously so that our own publications can be taken seriously too.

Is this process iron-clad? Not completely, but close. Yes, the paper linking MMR vaccination and autism did get published…but scientific vigilance never stops, and when results couldn’t be replicated, the data was revealed as fraudulent. There are other similar incidents, but sooner or later, falsified data is always caught. By far today’s greatest cause of expertise denial is not bad data but premature access to publications by the untrained eye.

Take, for example, the subject of topical sunscreens. The problem with the papers showing that chemical sunscreens kill juvenile coral is not that they were published. The problem is that the data was sensationalized and publicized too early for the countering arguments to reach the awareness of the relevant authorities; there is an overwhelming amount of data showing that increased temperature and decreasing salinity due to climate change, increased turbidity due to runoff and industrial pollution, and other factors dwarf sunscreen toxicity as factors in coral bleaching.

The current controversy on the safety of chemical sunscreens is a good example of how the scientific process is supposed to work yet becomes distorted under the public eye : data is generated, both positive (chemical sunscreens are very efficient UV absorbers - there is no relevant human data that shows carcinogenicity) and negative (chemical sunscreens bind hormone receptors, can enter the blood stream and generate free radicals in the process of absorbing UV radiation). A scientific approach would take all data into account to capture a balanced safety profile for chemical sunscreens and come to a reasonable conclusion: the benefits of chemical sunscreens outweigh the risks. Yet persistent maligning has led to a shift away from chemical sunscreens, and a general underuse of all sunscreens.

To further complicate matters, the FDA opted to remove all chemical sunscreens from its GRAS (generally recognized as safe) list, which only signifies that there is not enough data to guarantee their safety but not enough to incriminate them either. This last part was overlooked, and the FDA’s action was instantly interpreted as confirmation that chemical sunscreens are dangerous, making it much more difficult to develop a new aesthetically pleasing high SPF sunscreen.

There are many more examples of premature conclusions caused by the scientific process taking place in the public eye, especially when it comes to sunscreens. It is easy to blame publication authors, and certainly pseudo-scientific consumer watchdogs bear responsibility (Hello, EWG!), but lately the worst offenders have been influencers. In the end, it is every scientist’s responsibility to get out and communicate broadly, counter pseudoscience regardless of the source, and advocate for science education everywhere. It is also our responsibility to maintain data integrity and that’s why it’s a duty and an honor to review scientific manuscripts. For free.