We’re all taught in elementary school about the scientific method:
1) Ask a question
2) Make observations and/or do some background research
3) Develop a hypothesis to explain observations
4) Test hypothesis
5) Draw conclusion
6) Report results
Seems pretty cut-and-dried, doesn’t it?
As a practicing scientist, I know that science doesn’t always follow the scientific method.
It sure seems like we do, though. Every professional publication we write is done in a particular order, with an introduction (steps 1, 2, and 3), methods (step 4), results (also step 4), and discussion and conclusions section (step 5). But if you knew how things really worked…
There have been a few opportunities for non-scientists to get a glimpse into the wacky world of ‘real’ science. The hashtag #overlyhonestmethods went around Twitter a few months ago, wherein scientists around the world gave some insight into how their research really went.
But how else does practical science differ from the scientific method? For one, steps 5 and 6 above usually generate as many questions as they answer. A person could easily never get around to reporting the results, because they’re so busy trying to answer the new questions and test the new hypotheses that arose during the original analysis.
It’s a little misleading as well, to think that a scientist always has a well-formulated question before he or she starts making observations. Sometimes we make some weird observations, and spend months trying to formulate the question and hypothesis at the same time. For example, one might be handed a huge data set and be told to ‘do something with it.’
Two summers ago I was handed a mammoth second molar. “Do something with it.” So I did. I collected data, because I could, and then looked at the data and tried to formulate a hypothesis about the results. It may or may not ever get published, but it was interesting and fun.
Recently, I’ve been working on a paper about climate change at the Paleocene-Eocene Thermal Maximum (PETM). The idea is to collect a bunch of fossil mussels and look at the carbon and oxygen isotopes from the shells as a means to get at ancient climate and weather patterns. We already know that there was rapid global warming at the PETM. We want to know what that did to the weather.
Oh hey! A question! Did weather patterns change during the PETM? Did they go back to pre-PETM patterns when the PETM was over?
This is a really great question, but it arose after I made a bunch of observations: I knew the PETM was recorded in the Hanna Formation. I knew that there were some mollusks in those same rocks. And I knew that there was some way to get climate information from shells.
That’s fine. We can still investigate this question. Let’s go get some fossils! And some rocks, too, because we need carbon isotopes from the rocks to actually find the extent of the PETM. Ok, got it.
Back to the lab. Drilling, drilling, drilling.
All right. So what does it mean?
Here’s where science isn’t quite like the scientific method. I’ve now got a ton of data, thanks to the tireless work of two students (Jen and Julia). We sampled thirteen shells from all over the Hanna Formation. We have almost 200 data points. So what do you do?
The scientific method says “test hypothesis.” I’ve got a bunch of squiggly lines. Besides, what’s our hypothesis?
Hypothesis: Weather patterns during the PETM were different than those before and after the PETM.
Secondary hypothesis: Weather patterns from before and after the PETM were the same.
OK. Do all the squiggly lines look the same?
Well, no. But some of the shells were kinda… crappy.
Can I account for that?
Sure. Use statistics.
OK. I’ll calculate averages and standard deviations. And correlation coefficients, too. Why not? Excel can do that.
OK. Are they all the same?
Well, no, but the differences might have to do with rock type and environment more than climate. Some of the mussels came from lake beds and others from river beds.
And there’s the possibility that these mussels aren’t all the same species either. Would the same species be able to live in both lake and river environments? That would make the isotopic values different, I would guess.
Well crap, there’s a whole ‘nuther paper right there. A brand new question. I don’t have time for that. This needs to get published! What can I do with these data right now?
I dunno. Plot it up and see what it looks like. (Yes folks. An important step sometimes is just to plot your data and see. Most of the time, I have no idea what I’m about to see.)
OK. Here it is. Averages for each shell plotted against stratigraphic level.
Well, there’s an interesting pattern, I think. Things go more positive for both carbon and oxygen, then go back to more negative. Then they go more positive again. Is that an important pattern? How does it compare to the rock types? Or the position of the PETM?
Rats. One thing I see it means is that I need to find more fossils from between 2500 and 2600 meters (which is in the PETM and in a river environment).
But what if I don’t find more fossils? I’ve been looking for YEARS!! This is what I have. What does all this mean?
So this is where I am right now. I’ve got to do some more research on the different species of mussel that may or may not be present in the Hanna Formation. I also need to do some more research on what kind of effect the environment really has on how a mussel records weather patterns.
The point in showing you this is to illustrate part of the inner dialog that goes on with doing science. Sometimes you just get a bunch of data and start plotting it in various ways to see what patterns arise. Once you observe a pattern then you can try to understand it. And, as is almost always the case, new questions arise.
I don’t know if the mussels are different species between the lake and river environments. I don’t know what difference that would make.
I don’t know if the same mussel species when living in lakes and rivers record weather in the same way.
I don’t know how much it matters that the shells from the lower lake unit are ‘crappy’ compared to those from the upper lake unit. I have done some spectroscopy to show that there has been not a lot of alteration of the shell material, but there is a chance that weather is recorded differently in different parts of the shell as well, due to the age of the mussel.
Well, crap. I have more questions than answers. But I’ve got to write this up. This process doesn’t feel very much like the scientific method that I learned about in elementary school.
My experience isn’t unique. Keep that in mind. A lot of science is done this way. The natural world doesn’t fit conveniently into our notions of how it should work. Opportunities arise, and we jump at a chance to collect new data. Sometimes unexpected results fall from data that we hadn’t anticipated. Sometimes results fly in the face of conventional logic.
Sometimes you spend years staring at data waiting for something to make sense. Then once day while you’re taking a shower, years after you’ve given up on it, it all makes sense!
I started this project in 2007. I’m still working on it. That’s pretty typical, too.