Current application of P-value: challenges and optimization strategies

doi:10.12092/j.issn.1009-2501.2026.02.011

Abstract

Abstract:

The P-value (based on the frequentist school of thought) is an important tool in statistics for testing the null hypothesis, representing the probability of observing the current or more extreme results when the null hypothesis is true. Although P-values are widely used to determine the statistical significance of research findings (e.g., P<0.05, P<0.01), their abuse and misuse have sparked numerous controversies. This article reviews the importance and limitations of P-values in scientific research and clinical trials, including their sensitivity to sample size, the misleading nature of binary classification, and their inability to objectively reflect clinical significance, among other issues. Additionally, it explores supplementary and optimized methods, such as confidence intervals, effect sizes, and Bayesian statistics. Based on existing consensus and controversies, recommendations for the rational use of P-values are proposed, emphasizing the integration of multiple statistical indicators to enhance the rigor and reproducibility of scientific inferences, while recognizing the core role of P-values in controlling error rates and making group comparisons. Looking ahead, it advocates repositioning P-values as an indicator of the "compatibility" with the null hypothesis and applying them in conjunction with other methods to avoid simplistic binary classification.

Key words: statistics, P-value, research progress, statistical abuse

CLC Number:

C8
O212.1

Zhiwei LIU, Shuqing CHEN, Youjun CHEN, Yiming LI, Fangrong YAN, Yang ZHAO, Hua SUN, Haitang XIE, Ling WANG. Current application of P-value: challenges and optimization strategies[J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2026, 31(2): 240-246.

References 50

1	Anonymous. Taking the P: why the founder of P-values would be turning in his grave [M]. London: Unknown Publisher, 2020: 1-50.
2	Fisher RA. Statistical methods for research workers [M]//Kotz S, Johnson NL. Breakthroughs in Statistics. New York: Springer, 1992: 66-70.
3	Dahiru T. P-Value, a true test of statistical significance? A cautionary note[J]. Ann Ibadan Postgrad Med, 2011, 6 (1): 21- 26. doi: 10.4314/aipm.v6i1.64038
4	McShane BB, Gal D. The ASA statement-related article: statistical significance and the dichotomization of evidence[J]. J Am Stat Assoc, 2017, 112 (519): 885- 895.
5	Hodzic-Santor E, Fralick M. Treating patients, not P values[J]. NEJM Evid, 2025, 4 (1): 1- 5.
6	Sharma H. Statistical significance or clinical significance? A researcher's dilemma for appropriate interpretation of research results[J]. Saudi J Anaesth, 2021, 15 (4): 431- 434. doi: 10.4103/sja.sja_158_21
7	Thiese MS, Ronna B, Ott U. P value interpretations and considerations[J]. J Thorac Dis, 2016, 8 (9): E928- E931. doi: 10.21037/jtd.2016.08.16
8	Rothman J. A show of confidence[J]. N Engl J Med, 1978, 299 (24): 1362- 1363.
9	Trafimow D. Editorial[J]. Basic Appl Soc Psychol, 2014, 36 (1): 1- 2. doi: 10.1080/01973533.2014.865505
10	Trafimow D, Marks M. Editorial[J]. Basic Appl Soc Psychol, 2015, 37 (1): 1- 2. doi: 10.1080/01973533.2015.1012991
11	Wasserstein RL, Lazar NA. The ASA statement on p-values: context, process, and purpose[J]. Am Stat, 2016, 70 (2): 129- 133. doi: 10.1080/00031305.2016.1154108
12	Harrington D, D'Agostino RB, Gatsonis C, et al. New guidelines for statistical reporting in the journal[J]. N Engl J Med, 2019, 381 (3): 285- 286. doi: 10.1056/NEJMe1906559
13	Young PJ, Nickson CP, Perner A. When should clinicians act on non-statistically significant results from clinical trials?[J]. JAMA, 2020, 323 (22): 2256- 2257. doi: 10.1001/jama.2020.3508
14	Ioannidis JPA. What have we (not) learnt from millions of scientific papers with P values?[J]. Am Stat, 2019, 73 (S1): 20- 25. doi: 10.1080/00031305.2018.1447512
15	Van Zwet E, Gelman A, Greenland S, et al. A new look at P values for randomized clinical trials[J]. NEJM Evid, 2023, 3 (1): 1- 8. doi: 10.1056/evidx2400007
16	Wasserstein RL, Schirm AL, Lazar NA. Moving to a world beyond "P < 0.05"[J]. Am Stat, 2019, 73 (S1): 1- 19.
17	Begg CB. In defense of P values[J]. JNCI Cancer Spectr, 2020, 4 (2): pkaa012. doi: 10.1093/jncics/pkaa012
18	Mayo DG, Hand D. Statistical significance and its critics: practicing damaging science, or damaging scientific practice?[J]. Synthese, 2022, 200 (3): 220. doi: 10.1007/s11229-022-03692-0
19	Lytsy P. P in the right place: revisiting the evidential value of P-values[J]. J Evid Based Med, 2018, 11 (4): 288- 291. doi: 10.1111/jebm.12319
20	Aguinis H, Vassar M, Wayant C. On reporting and interpreting statistical significance and P values in medical research[J]. BMJ Evid Based Med, 2021, 26 (2): 39- 42. doi: 10.1136/bmjebm-2019-111264
21	Ahmed ES, Butt MN. The misunderstood P-value: why statistical significance is not enough in clinical practice[J]. Br J Anaesth, 2025, 134 (4): 909- 913. doi: 10.1016/j.bja.2025.01.008
22	Stahel WA. New relevance and significance measures to replace P-values[J]. PLoS One, 2021, 16 (6): e0252991. doi: 10.1371/journal.pone.0252991
23	Gao J. P-values – a chronic conundrum[J]. BMC Med Res Methodol, 2020, 20 (1): 167. doi: 10.1186/s12874-020-01051-6
24	Polack FP, Thomas SJ, Kitchin N, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine[J]. N Engl J Med, 2020, 383 (27): 2603- 2615. doi: 10.1056/NEJMoa2034577
25	Lammers D, Richman J, Holcomb JB, et al. Use of Bayesian statistics to reanalyze data from the pragmatic randomized optimal platelet and plasma ratios trial[J]. JAMA Netw Open, 2023, 6 (2): e230421. doi: 10.1001/jamanetworkopen.2023.0421
26	Tran NA, McGrory A, Poonai N, et al. A comparison of alternative ranking methods in two-stage clinical trials with multiple interventions: an application to the anxiolysis for laceration repair in children trial [J]. Clin Trials, 2024, 21(5): 17407745241251812.
27	Manson JE, Cook NR, Lee IM, et al. Vitamin D supplements and prevention of cancer and cardiovascular disease[J]. N Engl J Med, 2019, 380 (1): 33- 44. doi: 10.1056/NEJMoa1809944
28	Manson JE, Cook NR, Lee IM, et al. Marine n−3 fatty acids and prevention of cardiovascular disease and cancer[J]. N Engl J Med, 2019, 380 (1): 23- 32. doi: 10.1056/NEJMoa1811403
29	Patel S, Green A. Death by p-value: the overreliance on p-values in critical care research[J]. Crit Care, 2025, 29 (1): 73. doi: 10.1186/s13054-025-05307-9
30	Marschner IC. Confidence distributions for treatment effects in clinical trials: posteriors without priors[J]. Stat Med, 2024, 43 (6): 1271- 1289. doi: 10.1002/sim.10000
31	Wilcox RR, Serang S. Hypothesis testing, P values, confidence intervals, measures of effect size, and Bayesian methods in light of modern robust techniques[J]. Educ Psychol Meas, 2017, 77 (4): 673- 689. doi: 10.1177/0013164416667983
32	Murtaugh PA. In defense of P values[J]. Ecology, 2014, 95 (3): 611- 617. doi: 10.1890/13-0590.1
33	Halsey LG. The reign of the P-value is over: what alternative analyses could we employ to fill the power vacuum?[J]. Biol Lett, 2019, 15 (5): 20190174. doi: 10.1098/rsbl.2019.0174
34	Homa-Bonell JK. Interpreting P values in 2023[J]. J Patient Cent Res Rev, 2023, 10 (3): 102- 103.
35	Amrhein V, Greenland S. Discuss practical importance of results based on interval estimates and P-value functions, not only on point estimates and null P-values[J]. J Inf Technol, 2022, 37 (3): 316- 320. doi: 10.1177/02683962221105904
36	Rovetta A, Piretta L, Mansournia MA. P-values and confidence intervals as compatibility measures: guidelines for interpreting statistical studies in clinical research[J]. Lancet Reg Health Southeast Asia, 2025, 33, 100534. doi: 10.1016/j.lansea.2025.100534
37	Freire APCF, Elkins MR, Ramos EMC, et al. Use of 95% confidence intervals in the reporting of between-group differences in randomized controlled trials: analysis of a representative sample of 200 physical therapy trials[J]. Braz J Phys Ther, 2019, 23 (4): 302- 310. doi: 10.1016/j.bjpt.2018.10.004
38	Fernandes AD, Vu MTHQ, Edward LM, et al. A reproducible effect size is more useful than an irreproducible hypothesis test to analyze high throughput sequencing datasets [EB/OL]. Cornell University: arXiv, 2019, arXiv: 1909.05208v1.
39	Valladares-Neto J. Effect size: a statistical basis for clinical practice[J]. Rev Odontoc Sci, 2018, 33 (1): 84. doi: 10.15448/1980-6523.2018.1.29437
40	Citrome L. Quantifying clinical relevance[J]. J Clin Psychopharmacol, 2020, 40 (2): 115- 121.
41	Storey JD. The positive false discovery rate: a Bayesian interpretation and the q-value[J]. Ann Stat, 2003, 31 (6): 2013- 2035. doi: 10.1214/aos/1074290335
42	Fithian W, Lei L. Conditional calibration for false discovery rate control under dependence[J]. Ann Stat, 2022, 50 (6): 3139- 3164.
43	Rafi Z, Greenland S. Semantic and cognitive tools to aid statistical science: replace confidence and significance by compatibility and surprise[J]. BMC Med Res Methodol, 2020, 20 (1): 244. doi: 10.1186/s12874-020-01105-9
44	Bai Z, Fujikoshi Y, Hu J. Strong consistency of the AIC, BIC, Cp and KOO methods in high-dimensional multivariate linear regression [EB/OL]. Cornell University: arXiv, 2020, arXiv: 2006.06152v1.
45	Rezaei M, Malekjani M. Comparison between different methods of model selection in cosmology[J]. Eur Phys J Plus, 2021, 136 (2): 219. doi: 10.1140/epjp/s13360-021-01200-w
46	Soukup P. Může Bayesův faktor nahradit P hodnotu?[J]. Sociální Stud, 2021, 18 (1): 73- 99. doi: 10.5817/SOC2021-1-73
47	Lu S, Liu K, Chen XY, et al. Bayesian methods as a complementary tool: balancing innovation and rigor in critical care research[J]. Crit Care, 2025, 29 (1): 135. doi: 10.1186/s13054-025-05380-0
48	Malone HE, Coyne I. Complementing the P-value from null-hypothesis significance testing with a Bayes factor from null-hypothesis Bayesian testing[J]. Nurse Res, 2020, 28 (4): 41- 48. doi: 10.7748/nr.2020.e1756
49	Kennedy-Shaffer L. When the alpha is the omega: P-values, "substantial evidence, " and the 0.05 standard at FDA [J]. Food Drug Law J, 2018, 73(4): 589-616.
50	Mayo D. P-values on trial: selective reporting of (best practice guides against) selective reporting[J]. Harv Data Sci Rev, 2020, 2 (3): 1- 18. doi: 10.1162/99608f92.e2473f6a

[1]	JING Jiawen, MENG Qingbo, BI Zheng, WANG Fanjing, LI Yufan, FANG Zhaohui. Advances in animal models of diabetic erectile dysfunction based on therapeutic approaches [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2025, 30(9): 1224-1232.
[2]	WU Yiwen, SUN Yue, LU Zixuan, PAN Jiahe, YU Er, WO Hongmei, TANG Shaowen, ZHAO Yang, DAI Juncheng, YI Honggang. Application of the Bayesian mixture model based on a principal stratum strategy in clinical trials [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2025, 30(7): 942-949.
[3]	ZHANG Yidan, SUN Xinghua, QU Yang, HU Xiaoyang, ZHANG Miao. Research progress on the regulation of PI3K/Akt signaling pathway by active ingredients of traditional Chinese medicine in the treatment of cerebral ischemia-reperfusion injury [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2025, 30(6): 820-827.
[4]	DING Ding, WU Shengnan, WANG Ancai, WANG Deguo. Research progress on the inhibition of cardiac remodeling by vericiguat [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2025, 30(6): 858-863.
[5]	LIU Yeyuan, QI Yafeng, ZHANG Maofu, LI Xinyu, SHEN Yanyun, LIU Yu, ZHANG Shangzu, LI Yangyang, ZHANG Liying, ZHANG Zhiming. Research progress of traditional Chinese medicine intervention in chemotherapy renal injury [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2025, 30(4): 556-569.
[6]	HUANG Shumin, XIE Baocheng, LIU Guohui. Research progress on the mechanism of GLP-1 receptor agonists in the treatment of diabetic nephropathy [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2025, 30(12): 1701-1710.
[7]	TIAN Jiexiang, QI Wenxia, WANG Gang, YAN Yanfeng, WANG Zhandong, WEI Yong, LIU Hailong, ZHANG Yuanyuan. Advances in the study of the pathological role of salivary gland epithelial cells in Sj?gren's syndrome [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2025, 30(11): 1550-1558.
[8]	LUO Jiawei, CHEN Tianwang, WANG Xinyu, LIU Juan, HUANG Bo, LI Lisheng, XU Shangfu. Research progress on the effect and mechanism of natural products of traditional Chinese medicine in preventing vascular restenosis [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2025, 30(10): 1408-1416.
[9]	LIU Xuan, FENG Cuijuan, WANG Yiqiang, LI Fang. Research progress of miRNA in ulcerative colitis [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2024, 29(8): 917-929.
[10]	ZHAO Quanming, YANG Mandou, HU Yibo, SU Youtong, PU Li, ZHANG Yu, LI Wenliang. Research progress on drug treatment and drug resistance mechanism of gastrointestinal stromal tumors [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2024, 29(1): 82-89.
[11]	XU Jinhui, YUE Hongmei, LI Yating, LIU Miaomiao, WU Xingdong, ZHU Haobin. Advances in the study of mesenchymal stem cells in obstructive sleep apnea hypoventilation syndrome [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2024, 29(1): 114-120.
[12]	LUO Shiping, ZHANG Jie, YU Yushuai, SONG Chuangui. Advances in targeted therapy for HER2-positive breast cancer [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2023, 28(8): 876-886.
[13]	ZHANG Li, LIN Xingyao, SHANG Yun, WANG Qiang. Progress on the pathological mechanism and treatment of frostbite [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2023, 28(3): 347-354.
[14]	LI Chenhao, YU Xihua, YAN Wenhao, CHAI Yuxia, LI Yanyan, LI Kun, LI Zeyun. Research progress on the relationship between pediatric epilepsy and vitamin D [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2023, 28(11): 1299-1306.
[15]	LIU Dan, LIU Ming, JIN Liangyou, PAN Juan, XIN Haoru, LIU Mengyuan, LI Xin, ZHENG Kun, FENG Xiaoling. Research progress on the antitumor activity of costunolide [J]. Chinese Journal of Clinical Pharmacology and Therapeutics, 2023, 28(10): 1168-1176.