Science on trial
A close friend sent this linked article to me this morning. As an active member of a scientific non-profit group and co-author of the Metabolic Disease white paper, it got me thinking...
The credibility crisis in science
https://www.nakedcapitalism.com/2026/05/the-credibility-crisis-in-science.html
A novel perspective on scientific fraud—how undisclosed “tweaks” to research designs and model specifications fuel the credibility crisis in science.
In The Credibility Crisis in Science, leading social scientists Thomas Plümper and Eric Neumayer argue that the most impactful fraud is crucially under-recognized. While data fabrication and manipulation are widely recognized as fraudulent, “tweaks”—the intentional selection of research designs and model specifications based on the results they give—are not. As a consequence, the credibility crisis in science is even more severe than both scientists and the public believe.
The authors show how easily observational data analyses, experimental designs, and causal models are tweaked in ways that are extremely difficult, often impossible, to detect. They also argue that conventional strategies to deter, prevent, and detect fraud will not work for tweaks. They put forth two potential solutions: first, a classification system that categorizes data based on its susceptibility to manipulation and the probability of such manipulation being identified; and second, the proposal that journal editors and reviewers, rather than authors, select robustness tests.
Excerpts.
The credibility of science is not, as such, about scientific ideas, theories, or models that turn out to be false. Rather, the crisis has been caused by scientists who deliberately publish overconfident, misleading, and often simply false empirical results based on research designs or model specifications they have intentionally specified to give the desired results. We call this practice “tweaking.” In extreme cases, published results rely on manipulated or outright fabricated data. Whether tweaked, manipulated, or fabricated, the results often cannot be replicated – not even of replication analysts use identical research designs.
…
Tweaking is potentially more damaging to science in the long run than data manipulation and fabrication. Any particular tweaked empirical result is likely to have a smaller effect on the fabric of science than cases of data fabrication and manipulation, but the cumulative effect can still be can still be larger than the cumulative effect of data fabrication and manipulation because these strategies are rare, while tweaking is common.
My comments.
I have come to view science the same way I view the results I get from LLM prompts…
I never expect absolute truth to come from those sources. Informed perception is the best I can glean, and it’s up to me and my personal “BS meter” to discern what is useful and even valuable.
Absolute Truth is the rarest of all artifacts of consciousness.
Relative Truth is the perspective of individuals and their unique life lens.
Publishing houses — the institutions that specialize in approving and distributing scientific papers — collectively offer a vast universe of perspectives from specialists with academic and professional credentials in a domain of knowledge that is inevitably narrow in scope but deep in penetration. The credential Ph.D. has been characterized by some euphemistically as the earned academic designation for someone who has acquired deep knowledge of a study domain that is “a mile deep and an inch wide”.
Specialization and Consilience.
Consilience is a philosophical term popularized by E. O. Wilson describes the “unity of knowledge,” where insights from different fields converge into a coherent understanding.
Looking back on my career over 36 years as an IT Recruitment Professional, I witnesses the process of consilience as it continually reinvented the entire landscape of computer and communications infrastructures than provided the needed scaffold s to support all segments of our economy and the human knowledge domains that it served.
The term “systems integration” became common as the early days of my career as stand-alone mainframes and minicomputers were rapidly superseded by complex networks comprising “stacks” of computing layers and components. Each offered a piece of the puzzle needed to fulfill increasingly complex operations and decisions.
Likewise, for human knowledge “integration”.
Gone are the days when one “superstar” could competently run a business or lead an institution. Modern ‘complexity in all things’ is the result of our building upon vast stacks of knowledge domains over time — the knowledge scaffold.
This reality has necessitated new management approaches. Today, teams of specialists are needed to participate in planning and decision-making to address organizational and project challenges. The best teams include people carefully selected for their specific knowledge and experience which, when interacting with one another, often precipitates consilience — new ideas that lead to the best possible outcomes.
I fondly recall my days as Director of the Talent Supply Chain of an international SAP systems integration consulting firm. In project planning/execution meetings, we leveraged the knowledge of participating experts using SWOT analysis discussions to draw in diverse perspectives and suggestions. We considered the participation of those experts vital to making decisions and plans with which we had confidence.
Looking ahead.
I reference AI often in these pages.
While I depended on a small group of human experts at the table for advice and crucial information in the past, today I think of my LLM “co-workers” as offering a vastly broader and richer source of advice.
I am currently preparing to set up at AI Assistant on my Claude.ai PRO subscription and its tools Claude Code, Claude Design, and Claude Cowork on my new Apple Mac Mini. I face a significant learning curve to create this authors “project desktop”. I look forward to the day when I can write my next book with the productivity gains I expect from my very own Claude AI co-worker.
Stay tuned.
Tweaking is part of my process.
The authors of the book referenced above portrayed tweaking in a suspicious, derogatory light. Do they expect absolute truth from scientific investigations - a standard of moral and ethical purity which few can match? Or do scientists publish to generate perspectives which may trigger novel thinking and more Consilience?
After all, science is always “becoming truth” rather than reaching absolute truth.
Science has continually led humanity into battle with the many sources of decay and harm that our species faces daily. Without science and its relentless pursuit of better “mousetraps” and superior ways to live, would I be authoring my essays on wax tablets?
Personally, I write to stimulate curiosity and contemplation in myself and others. Ideas have always motivated me to explore my mindscape.
As my father said to me years ago,
“Life should surprise you”
I took this as dad’s way of telling me to be open the adventures that life offers and to always enjoy the ride.