THIS ARTICLE WILL MAKE YOU 10X SMARTER, ACCORDING TO SCIENCE
We often see articles touting headlines such as, “Drinking a Cup of Coffee Makes Your Workout 10x Better,” “Dark Chocolate Cures the Flu,” or “Wine Drinkers Live Longer, According to Science.”
The above titles are fictional; these claims are purposefully extreme to represent the eye-catching headlines used to attract readers through many news outlets. In doing so, the original data can be misused, the message misinterpreted, and the claims misunderstood to support behaviors that may not actually be helpful or healthful.
Misuse, misinterpretation, and misunderstanding: problems that Joshua Wallach confronts through his work as a PhD Candidate in Epidemiology and Clinical Research and Graduate Teaching Affiliate in Human Biology at Stanford University. Joshua’s most recent research focuses on the use of statistical significance in academic literature. For example, he recently published a paper in the Journal of the American Medical Association (JAMA) titled, “Evolution of Reporting P-Values in Biomedical Literature.” In its simplest description, a p-value gives the probability of obtaining an effect at least as extreme as your sample data, when the null hypothesis is true. Convention says p-values less than 0.05 represent a “statistically significant” result. However, the paper concluded that p-value reporting has increased over time, and almost all abstracts and articles studied had a statistically significant p-value.
Why is this important? “We are taught to use p-values. We use p-values and report them in our papers. Journals have come to expect them. Readers have come to expect p-values, and If you don’t have a statistically significant finding, you are unlikely to get published. P-values are not necessarily bad on their own, but in their use there is selective reporting,” says Joshua.
In essence, The reporting of statistically significant p-values has become a vicious cycle;
A cycle that leads some to splice, dice, and manipulate their data for a statistically significant result.
Joshua states that a step in the right direction would include a higher acceptance of null findings in the scientific community. Outcomes with insignificant results are indeed important because one-time findings do not always mean much on their own. One study on its own might have a spurious finding. Good scientific experiments should be repeated to see if the original results can be validated.
This field—“research on research”—is of particular interest to Joshua. He is a member of METRICS, the Meta-Research Innovation Center at Stanford. Its mission is to improve the quality of scientific studies in biomedicine and beyond. “At the very top of the research pyramid is a meta-analysis: a synthesis of many studies. You want a synthesis of multiple experiments that have looked at the exact same question, each with its own significant or null findings. Bringing together evidence in this way makes a more powerful claim,” says Joshua. “In the work I’m doing, we want studies that can be replicated and completed in the same way by independent investigators.”
I asked Joshua about his path to meta-research. He lived in the Netherlands until his junior year of high school with approximately 80 kids in his class, and moved back to the U.S. for senior year graduating with 1000 kids in his class. He stayed in California for college at UC Davis; having studied economics during his undergraduate years, he discovered that he always liked the statistical aspects of finance and mathematics. He enjoyed the methodology of health research and thought he would apply it to economics after graduation, but he discovered that he loved the health aspects more.
Joshua’s realization led him to study Epidemiology at Stanford. One of his first classes was taught by his current advisor and the Co-Director of METRICS, Dr. John Ioannidis. “It was a class about meta-research and all types of biases in scientific literature,” says Joshua. From then on, his work has combined epidemiology, statistics, and meta-research that allows him to transcend the traditional divisions of specific medical fields. “I can study across the entire biomedical literature and I get to think about the methodology in epidemiology and statistics that are used across all fields. I don’t have to be locked into any one field, and I am able to understand and think about the problems that influence all of them.”
His aforementioned paper published in JAMA on the problem of reported p-values is one such example.
“That involved a lot of reading.”
“How much reading?”
“I read 1000 abstracts and 100 full text articles over and over again to extract all the necessary information we needed!”
Indeed, he is very dedicated to his research. However, when he’s not researching, he enjoys teaching. “It’s a dream to be learning and teaching at Stanford,” says Joshua, “and I am very happy to have the opportunity to teach. Everyone is engaged and interested and asks questions that keep you challenged. If you asked me ten years ago to picture where I am right now, researching and teaching a class at Stanford, I would not have thought it was possible.” I took a class taught by Joshua called “Statistics in the Health Sciences,” and he was always open to talk about his students’ research endeavors outside the classroom. He encourages students to reach out to him about their unique interests and learn how to apply statistics to their work and everyday life.
Between teaching, researching, and commuting 3 hours daily from Oakland (NPR and KQED being the entertainment of choice), Joshua likes to hike, travel, and explore. More than just a way to clear his mind and recharge through exercise, when he visits a new city he also prefers to walk everywhere as a way to learn about the area. When he was younger he learned to play classical guitar and then picked up flamenco guitar, although his current schedule has limited his playing time.
In my final question, I asked Joshua what advice he would give to undergraduate students. He highlighted the importance of finding something that makes you happy everyday; for him, that’s exercising and weekend hikes because it’s a refreshing change of pace from his usual work schedule.
“Work hard- but don’t let work itself be your entire life!”
Joshua makes a great point, and sometimes it is difficult to balance school with other passions.
Nonetheless, it is important to be mindful of how you allocate your time for your overall health and wellbeing: statistically speaking.