When scientific hypotheses don’t pan out

Research studies are often built around an educated guess. What happens when those guesses are wrong?
Illustration of a scientist holding a beaker; a rack of test tubes behind him has a cloud with question marks coming out.
Research is often driven by a scientific hypothesis, an educated guess about how studies will turn out. But sometimes the results surprise even the scientists. Illustration by Kim Carney / Fred Hutch News Service

One pair of scientists thought they’d discovered a new antiviral protein buried inside skin cells. Another research team saw early hints suggesting that the flu virus might cooperate to boost infections in humans. And a nationwide team of clinicians thought that high doses of certain vitamins might prevent cancer.

These studies don’t have much to do with each other, except that the researchers had all based their hypotheses on convincing earlier data.

And those hypotheses were all wrong.

The hypothesis is a central tenet to scientific research. Scientists ask questions, but a question on its own is often not sufficient to outline the experiments needed to answer it (nor to garner the funding needed to support those experiments).

So researchers construct a hypothesis, their best educated guess as to the answer to that question.

How a hypothesis is formed

Technically speaking, a hypothesis is only a hypothesis if it can be tested. Otherwise, it’s just an idea to discuss at the water cooler.

Researchers are always prepared for the possibility that those tests could disprove their hypotheses — that’s part of the reason they do the studies. But what happens when a beloved idea or dogma is shattered is less technical, less predictable. More human.

In some cases, a disproven hypothesis is devastating, said Swedish Cancer Institute and Fred Hutchinson Cancer Research Center public health researcher Dr. Gary Goodman, who led one of those vitamin studies. In his case, he was part of a group of cancer prevention researchers who ultimately showed that high doses of certain vitamins can increase the risk of lung cancer — an important result, but the opposite of what they thought they would prove in their trials.

But for some, finding a hypothesis to be false is exhilarating and motivating.

Herpes hypothesis leads to surprise cancer-related finding

Photo of Jia Zhu
Fred Hutch infectious disease researcher Dr. Jia Zhu was chasing down what she thought was an antiviral protein. Photo by Robert Hood / Fred Hutch News Service

Dr. Jia Zhu, a Fred Hutch infectious disease scientist, and her research partner (and husband), Fred Hutch and University of Washington infectious disease researcher Dr. Tao Peng, thought they’d found a new antiviral in herpes simplex virus type 2, or HSV-2, in part because they’ve been focused on that virus — and its interaction with human immune cells — for decades now, together with Dr. Larry Corey, virologist and president and director emeritus of Fred Hutch.

A few years ago, Zhu and Peng found that a tiny, mysterious protein called interleukin-17c is massively overproduced by HSV-infected skin cells. Maybe it was an undiscovered antiviral protein, the virologists thought, made by the skin cells in an attempt to protect themselves. They spent more than half a year pursuing that hypothesis, conducting experiment after experiment to see if IL-17c could block the herpes virus from replicating. It didn’t.

Zhu pointed to a microscopic image of a biopsy from a person with HSV, captured more than 10 years ago where she, Corey and their colleagues first discovered that certain T cells, a type of immune cell, cluster in the skin where herpes lesions form. At the top of the colorful image, a layer of skin cells stained blue is studded with orange-colored T cells. Beneath, green nerve endings stretch their branch-like fibers toward the infected skin cells.

Image of nerve endings (green) interacting with herpes-infected skin cells (blue)
Zhu's "favorite image" depicts nerve endings (green) interacting with herpes-infected skin cells (blue). Zhu et al., Journal of Experimental Medicine, 2007

“This is my favorite image, but we all focused on the top,” the skin and immune cells, Zhu said. “We never really paid attention to the nerves.”

"You take an approach and then you just have to let the science drive."
Dr. Jia Zhu, infectious disease researcher

Finally, Peng discovered that the nerve fibers themselves carry proteins that can interact with the IL-17c molecule produced in infected skin cells — and that the protein signals the nerves to grow, making it one of only a handful of nerve growth factors identified in humans.

The researchers are excited about their serendipitous finding not just because it’s another piece in the puzzle of this mysterious virus, which infects one in six teens and adults in the U.S. They also hope the protein could fuel new therapies in other settings — such as neuropathy, a type of nerve damage that is a side effect of many cancer chemotherapies.

It’s a finding they never would have anticipated, Zhu said, but that’s often the nature of research.

“You do have a big picture, you know the direction. You take an approach and then you just have to let the science drive,” she said. “If things are unexpected, maybe just explore a little bit more instead of shutting that door.”

Flu hypothesis leads to a new mindset and avenue of research

Photo of Jesse Bloom
Dr. Jesse Bloom's team studied viral infections in cells in the lab. One study sent them in a new direction. Fred Hutch file

Sometimes, a mistaken hypothesis has less to do with researchers’ preconceptions and more to do with the way basic research is conducted. Take, for example, the work of Fred Hutch evolutionary biologist Dr. Jesse Bloom, whose laboratory team studies how influenza and other viruses evolve over time. Many of their experiments involve infecting human cells in a petri dish with different strains of the flu virus and seeing what happens.

A few years ago, Bloom and University of Washington doctoral student Katherine Xue made an intriguing discovery using that system: They saw that two variants of influenza H3N2 (the virus that’s wreaking havoc in the current flu season) could cooperate to infect cells better together than either version could alone.

The researchers had only shown that viral collaboration in petri dishes in the lab, but they had reason to think it might be happening in people, too. For one, the same mix of variants was present in public databases of samples taken from infected people — but those samples had also been grown in petri dishes in the lab before their genomic information was captured.

So Xue and Bloom sequenced those variants at their source, the original nasal wash samples collected and stored by the Washington State Public Health Laboratories. They found no such mixture of variants from the samples that hadn’t been grown in the laboratory — so the flu may not cooperate after all, at least not in our bodies. The researchers published their findings last month in the journal mSphere.

Scientists have to ask themselves two questions about any discovery, Bloom said: “Are your findings correct? And are they relevant?”

Photo of Katherine Xue
Katherine Xue, a doctoral student working in Bloom's laboratory Photo by Robert Hood / Fred Hutch News Service

The team’s first study wasn’t wrong; the viruses do cooperate in cells in the lab. But the second question is usually the tougher one, the researchers said.

“There are a lot of differences, obviously, between viruses growing in a controlled setting in a petri dish versus an actual human,” Xue said.

She and Bloom aren’t too glum about their disproven hypothesis, though. That line of inquiry opened new doors in the lab, Bloom said.

Before Xue’s study, he and his colleagues exclusively studied viruses in petri dishes. Now, more members of his laboratory team are using clinical samples as well — an approach that is made possible by the closer collaborations between basic and clinical research at the Hutch, Bloom said.

Some of their findings in petri dishes aren’t holding true in the clinical samples. But they’re already making interesting findings about how flu evolves in the human body — including the discovery that how flu evolves in single people with unusually long infections can hint at how the virus will evolve globally, years later. They never would have done that study if they hadn’t already been trying to follow up their original, cooperating hypothesis.

“It opened this whole new way of trying to think about this,” Bloom said. “Our mindset has changed a lot.”

Prevention hypothesis flipped on its head

Fred Hutch and Swedish cancer prevention researcher Goodman and his epidemiology colleagues had good reason to think the vitamins they were testing in clinical trials could prevent lung cancer.

All of the data pointed to an association between the vitamins and a reduced risk of lung cancer. But the studies hadn’t shown a causative link — just a correlation. So the researchers set out to do large clinical trials comparing high doses of the vitamins to placebos.

In the CARET trial, which Goodman led and was initiated in 1985, 18,000 people at high risk of lung cancer (primarily smokers) were assigned to take either a placebo, vitamin A, beta-carotene (a vitamin A precursor) or a combination of the two supplements. Two other similar trials started in other parts of the world at around the same time also testing beta-carotene’s effect on lung cancer risk.

In a similar vein, at the same time, a small trial suggested that supplemental selenium decreased the incidence of prostate cancer. So in 2001, the SELECT trial launched through SWOG, a nationwide cancer clinical trial consortium, testing whether selenium or high-dose vitamin E or the combination could prevent prostate cancer. SELECT enrolled 35,000 men; Goodman was the study leader for the Seattle area.

Designing and conducting cancer prevention trials where participants take a drug or some other intervention is a tricky business, Goodman said.

“In prevention, most of the people you treat are healthy and will never get cancer,” he said. “So you have to make sure the agent is very safe.”

Previous studies had all pointed to the vitamins being safe — even beneficial. And the vitamins tested in the trials are all naturally occurring as part of our diets. Nobody thought they could possibly hurt.

But that’s exactly what happened. In the CARET study, participants taking the combination of vitamin A and beta-carotene had higher rates of lung cancer than those taking the placebo; other trials testing those vitamins saw similar results. And in the SELECT trial, those taking vitamin E had higher rates of prostate cancer.

All the trials had close monitoring built in and all were stopped early when the researchers saw that the cancer rates were trending the opposite way that they’d expected.

“It was just devastating when we learned the results,” Goodman said. “Everybody [who worked on the trial] was so hopeful. After all, we’re here to prevent cancer.”

When the CARET study stopped, Goodman and his team hired extra people to answer study participants’ questions and the angry phone calls they assumed they would get. But very few phone calls came in.

“They said they were involved in the study for altruistic reasons, and we got an answer,” he said. “One of the benefits of our study is that we did show that high doses of vitamins can be very harmful.”

That was an important finding, Goodman said, because the prevailing dogma at the time was that high doses of vitamins were good for you. Although these studies disproved that commonly held belief, even today not everyone in the general public buys that message.

Another benefit of that difficult experience: The bar for giving healthy people a supplement or drug with the goal of preventing cancer or other disease is much higher now, Goodman said.

“In prevention, [these studies] really changed people’s perceptions about what kind of evidence you need to have before you can invest the time, money, effort, human resources, people’s lives in an intervention study,” he said. “You really need to have good data suggesting that an intervention will be beneficial.”

rachel-tompa

Rachel Tompa is a former staff writer at Fred Hutchinson Cancer Center. She has a Ph.D. in molecular biology from the University of California, San Francisco and a certificate in science writing from the University of California, Santa Cruz. Follow her on Twitter @Rachel_Tompa.

Related News

All news
Fantastic failures: Scientific setbacks fuel big gains Whether solving the mysteries of outer space or cancer cells, many major advances followed messy moments August 18, 2015
Stealing a secret from an unwelcome virus An unexpected discovery in a herpes lab might bring relief to cancer patients June 29, 2017
When flu viruses join forces Certain influenza variants infect cells better together than apart, new study shows March 15, 2016

Help Us Eliminate Cancer

Every dollar counts. Please support lifesaving research today.