pages tagged statistics

Analysis of short-term performance improvement of top swimmers in 400m individual medley

Tue, 16 May 2017 23:59:39 +0000

Meng Lu <lumeng3@gmail.com>
Aug. 7, 2012

16-year-old Chinese swimmer Ye Shiwen swam more than 5 seconds faster than her personal best in women's 400m individual medley (IM) in London 2012 Olympics, winning her a gold medal. This performance received high exposure to media and mixed comments, even controversy. For example, the prestigious scientific magazine Nature published a news report titled "Why great Olympic feats raise suspicions -- 'Performance profiling' could help to dispel doubts" [1] and immediately raised great attention and a great number of reader comments. In the following, I analyze short-term performance increase of top swimmers in 400m IM during 2011 and 2012, and attempt to contrast that to Ye Shiwen's, using Fédération Internationale de Natation (http://www.fina.org) data [2]. I hope to bring some data perspective to the issue.

Data

The FINA 40m IM data sets contains the 300 fastest race times recorded in 2011 and 2012 respectively, and for male and female swimmers, respectively. It turns out there are 195 female and 191 male swimmers who have both a 2011 time and a 2012 time, which will be used in the following analysis.

FINA data has two pool course sizes: long course (LC) and short course (SC) however the latter is different from the Olympic standard pool size. We'll include SC times in the analysis for comparison, but will keep it separate from LC times.

Here is the first 10 rows from the data sets for "women's 400m IM, LC":

time	name	country	event date	event	event city	event country	qualify for
4:28.43	Ye, Shiwen	CHN	28/07/2012	2012 Olympic Games	London	GBR	OQT
4:31.27	Beisel, Elizabeth	USA	28/07/2012	2012 Olympic Games	London	GBR	OQT
4:32.20	Zheng, Rongrong	CHN	08/01/2012	National Spring Championships2 2012	Nanning	CHN	OQT
4:32.67	Miley, Hannah	GBR	03/03/2012	British Gas Swimming Champs 2012	London	GBR	OQT
4:32.83	Hosszu, Katinka	HUN	29/03/2012	2012 Indianapolis Grand Prix	Indianapolis	USA	OQT
4:32.91	Li, Xuanxu	CHN	28/07/2012	2012 Olympic Games	London	GBR	OQT
4:33.45	Rice, Stephanie	AUS	15/03/2012	2012 EnergyAustralia Swimming	Adelaide	AUS	OQT
4:33.91	Belmonte Garcia, Mireia	ESP	29/03/2012	Spanish Open Spring Swimming Championships	Malaga	ESP	OQT
4:34.48	Leverenz, Caitlin	USA	25/06/2012	USA Olympic Team Trials	Omaha	USA	OQT
4:35.68	Jakabos, Zsuzsanna	HUN	21/05/2012	European Swimming Championships	Debrecen	HUN	OQT
4:36.21	Evans, Blair	AUS	27/01/2012	South Australian State Open Championships	Adelaide	AUS	OQT
4:36.64	Otsuka, Miyu	JPN	02/04/2012	2012 Japan National Championships	Tokyo	JPN	OQT
4:37.48	Willmott, Aimee	GBR	03/03/2012	British Gas Swimming Champs 2012	London	GBR	OQT

Direct data download

(I'll zip up the cleaned data and host it on Google Drive soon.)

Variables

We define the following variables to simplify our discussion of the analysis:

: swimmer's best 400m IM time in 2011
: swimmer's best 400m IM time in 2012
: date when is achieved
: date when is achieved

and compute several derived quantities that can be used as swimmer's performance improvement indicators:

: absolute improvement (unit: second); it should be positive if 2012 performance is faster than that of 2011;
: relative improvement (unit: %)
: time rate of relative improvement (unit: %/day)

Ye's performance improvement in 400m IM during 2011-2012 is

Analysis

Distribution of performance improvement indicators

We examine the distributions of three performance improvement indicators , , and in histograms and quantile plots:

The distribution of is closest to a normal distribution, which has standard deviation 0.379 sec and mean 3.45 sec. Ye Shiwen's sec is at 1.41 sigma. She is in the top 8% highest relative performance improvement.

One can compare Ye to all 400m IM swimmers in the space:

A few comments:

The sample points are approximately distributed along the diagonal, which likely suggests that for the majority top swimmers, there is no systematic improvement within a short time window < 2 years. So dividing the performance improvment ( or ) by time () is not terribly relevant as data doesn't suggest such trend.
The sample also obviously seperates into two categories: male and female. Although, the distribution of the two categories seem similar up to a shift of their center. (More quantitative study is due on this.) So it is intriguing to shift and combine the two categories and study the shape characteristics of the distribution for the combined sample.
Interestingly but not surprisingly, Ye is at the relatively unique place that her 2011 and 2012 best performance sec and sec are both in between majority male and female swimmers. Therefore it is non-trivial to compare her with either the majority of male swimmers or the female ones. But this might not be exactly surprising, given that she's the world's No. 1 female in this event.
The SC times enjoy a greater performance gain and lies below the diagonal. A short course pool might mean improvement on the pool-end technique can significantly contribute to the overall time improvement. (Just a guess though!)
It seems the more of the top female and male swimmers tend to have a performance increase as compared to others within same sex category, as the lower left corner of each sex category slightly bends below the diagonal.
Similarly, the swimmers with worst 2011 best performance tend to have performance improvement. This is probably simply because they have more room/potential for improvement compared to the best of the best.
There seem to be a higher fraction of male swimmers who gain a better best performance in 2012 than that of female swimmers. (Visually, there are more squares under the diagnol than dots.)

One can also plot the performance improvement indicators against 2011 best performance and the period of time in between 2011 and 2012 best performances to see some of these trends:

Sub-sample of great performance improvers

To select a subsample of performance improvers to compare with Ye's significant performance improvement in London 2012 Olympics, one can select out swimmers who has at least one of the three performance improvement indicator better than 85% of that of Ye's. We also restrict to "LC" which is the pool size same as or closer to the Olympics swimming pool size. It turns out there are as many as 68 swimmers satisfying this condition. (See table in Section 4). Furthermore, there are actually 21 of them who beat Ye Shiwen in all the three improvement indicators. (See table in Section 5).

Performance improvement v.s. age

It is also noticable that younger swimmers are more likely to gain performance improvement. Most great performance improvement are done by swimmers under 22. And Ye doesn't look terribly anomalous in gaining a high performance improvement among many other top teenage swimmers:

Swimmers who achieved good performance improvement in 400m IM over the 2011-2012 period

name	sex	country						age	height	weight		2011 best performance		2012 best performance
Trombetti, Luisa	F	ITA	81	293.7	5.9		0.0248	19.6			4:53.70	Italian Open Winter Nationals 2011	4:47.80	Italian National Spring Championships
Suggitt, Chris	M	GBR	78	269.81	4.71	1.74	0.0223				4:29.81	British Gas Swim Wales Winter	4:25.10	British Gas Swimming Champs 2012
Rousseau, Sebastien	M	RSA	157	266.95	9.12	3.41	0.0217	21.9	1.9	80	4:26.95	Minneapolis Grand Prix	4:17.83	Sa National Aquatic Championships
Meyer, Michael	M	RSA	136	263.87	7.45	2.82	0.0207	24.66			4:23.87	Open Dutch Championships	4:16.42	Sa National Aquatic Championships
Caldwell, Katy	F	CAN	49	291.09	2.83	0.972	0.0198	18.6			4:51.09	2011 Canada Cup	4:48.26	2012 Austin Grand Prix
Chen, Xinyi	F	CHN	167	289.95	8.94	3.08	0.0184	14.6			4:49.95	7th National City Games 2011	4:41.01	National Swimming Championships
Cuoghi, Alessandro	M	ITA	81	262.94	3.58	1.36	0.0168	21.6			4:22.94	Italian Open Winter Nationals 2011	4:19.36	Italian National Spring Championships
Maestri, Riccardo	M	ITA	81	270.16	2.92	1.08	0.0133	18.3	1.77	74	4:30.16	Italian Open Winter Nationals 2011	4:27.24	Italian National Spring Championships
Mann, Becca	F	USA	227	290.17	8.56	2.94	0.0129	14.6			4:50.17	Minneapolis Grand Prix	4:41.61	USA Olympic Team Trials
Ventrone, Stefano	M	ITA	81	271.73	2.8	1.03	0.0127	21.6			4:31.73	Italian Open Winter Nationals 2011	4:28.93	Italian National Spring Championships
McMaster, Keryn	F	AUS	121	290.8	4.43	1.52	0.0125				4:50.80	2011 McDonald's Queenslands Championships	4:46.37	2012 Australian Age Champs
Mckeown, Taylor	F	AUS	121	289.32	4.29	1.48	0.0122				4:49.32	2011 McDonald's Queenslands Championships	4:45.03	2012 Australian Age Champs
Seltenreich-Hodgson, Erika	F	CAN	311	294.3	10.5	3.56	0.0114	17.6			4:54.30	2011 Canadian Age Group Championships	4:43.80	Mare Nostrum
Berrada, Morad	M	MAR	92	268.27	2.69		0.0108	21.12			4:28.27	Arab Games 2011	4:25.58	Elite French Championships
Coton, Quentin	M	FRA	241	268.85	6.63	2.46	0.0102	21.04			4:28.85	Summer Nationals	4:22.22	Elite French Championships
Beraud, Romain	M	FRA	241	270.91	6.53	2.41	0.01	23.98			4:30.91	Youth Senior Open Championships	4:24.38	Elite French Championships
Tutumi, Takahiro	M	JPN	213	265.35	5.47	2.06	0.00967	19.9			4:25.35	Japan Inter-College Swimming Competition	4:19.88	2012 Japan National Championships
Siwicki, Breanne	F	CAN	250	293.79	7.01	2.38	0.00954	17.6			4:53.79	Summer Nationals	4:46.78	2012 Canadian Olympic and Paralympic Trials
Geni, Simone	M	ITA	81	269.98	2.04	0.755	0.00932				4:29.98	Italian Open Winter Nationals 2011	4:27.94	Italian National Spring Championships
Gilmour, Tianni	F	AUS	121	295.65	3.31	1.11	0.00925				4:55.65	2011 McDonald's Queenslands Championships	4:52.34	2012 Australian Age Champs
Wedel, Kevin	M	GER	309	268.84	7.64	2.84	0.00919	18.6			4:28.84	38th European Junior Championships 2011	4:21.20	124th German Championships
Hosszu, Katinka	F	HUN	116	275.76	2.93	1.06	0.00915				4:35.76	UT Invite Long Course Time Tri	4:32.83	2012 Indianapolis Grand Prix
Soerensen, Katrine	F	DEN	245	293.96	6.58	2.23	0.00913				4:53.96	Summer Nationals	4:47.38	Danish Open
Donaldson, Mitchell	M	NZL	107	270.86	2.62	0.967	0.00904				4:30.86	2011 Auckland Age Groups Invit	4:28.24	State 2012 NZ Open Championships
Mann, Karley	F	GBR	356	294.94	9.3	3.15	0.00885	18.6			4:54.94	British Gas ASA Youth Champs 2011	4:45.64	British Gas Nat Youth MC Chmps 2012
Glavnyk, Iryna	F	UKR	238	294.7	6.21	2.1	0.00885	16.24	1.74	58	4:54.70	38th European Junior Championships 2011	4:48.49	Open Ukrainian National Championships
Pinto, Andreina	F	VEN	181	292.6	4.58	1.56	0.00864				4:52.60	XVIII National Games	4:48.02	2012 US Santa Clara International
Lowe, Danielle	F	GBR	356	295.9	9.1	3.07	0.00863	17.6			4:55.90	British Gas ASA Youth Champs 2011	4:46.80	British Gas Nat Youth MC Chmps 2012
McKee, Anton Sveinn	M	ISL	281	270.15	6.51	2.4	0.00857				4:30.15	38th European Junior Championships 2011	4:23.64	Icelandic National Championships
Persson, Erik	M	SWE	260	271.82	5.86	2.15	0.00829	18.6			4:31.82	38th European Junior Championships 2011	4:25.96	Danish Open
Quah, Zheng Wen	M	SIN	122	264.33	2.63	0.994	0.00815	15.85	1.75	57	4:24.33	XXVI Sea Games 2011	4:21.70	Singapore National Age Group Championships
Proud, Stephanie	F	GBR	223	288.31	5.21	1.8	0.0081	23.99	1.68	65	4:48.31	2011 FINA World Championships	4:43.10	British Gas Swimming Champs 2012
Kochetkova, Kristina	F	RUS	261	294.48	6.08	2.06	0.00791	19.52			4:54.48	Russian National Swimming Cup	4:48.40	International Meeting
Kuhn, Sierra	F	USA	231	293.88	5.17	1.75	0.00761				4:53.88	2011 Conoco Phillips National Championships	4:48.71	2012 NCSA Junior National Swim
Smiddy, Sam	M	USA	234	268.74	4.58	1.7	0.00728	18.6			4:28.74	2011 Speedo Junior National Championships	4:24.16	2012 Indianapolis Grand Prix
Gyurta, Gergely	M	HUN	281	261.9	5.34	2.03	0.00725	20.903	1.74	68	4:21.90	CXIII Hungarian Swimming Championship	4:16.56	Open National Championships
Aoki, Takehiro	M	JPN	213	268.36	4.02	1.49	0.00703				4:28.36	Japan Inter-College Swimming Competition	4:24.34	2012 Japan National Championships
Schepers, Jan David	M	GER	282	264.71	5.24	1.97	0.00701				4:24.71	German Championships	4:19.47	Qualification for European Championships
Nguyen, Thi Anh Vien	F	VIE	214	294.65	4.38	1.48	0.00694	15.74	1.72	53	4:54.65	XXVI Sea Games 2011	4:50.27	1st SEA Swim Championships
Tezuka, Yuki	M	JPN	114	261.93	2.04	0.778	0.00683	17.6			4:21.93	2011 JPN Open	4:19.89	2012 Victorian Championships
Mori, Yousuke	M	JPN	213	261.22	3.79	1.45	0.00681				4:21.22	Japan Inter-College Swimming Competition	4:17.43	2012 Japan National Championships
Vanderkaay, Peter	M	USA	398	263.28	7.06	2.68	0.00673	28.48	1.93	90	4:23.28	2011 Michigan Grand Prix	4:16.22	2012 NC Charlotte UltraSwim
Ye, Shiwen	F	CHN	285	273.66	5.23	1.91	0.0067	16.27	1.6	45	4:33.66	7th National City Games 2011	4:28.43	2012 Olympic Games
Zhou, Min	F	CHN	365	284.69	6.86	2.4	0.0066				4:44.69	National Swimming Championships	4:37.83	National Swimming Championships
Hersey, Kathleen	F	USA	139	286.03	2.61	0.912	0.00656	22.46	1.8	64	4:46.03	Minneapolis Grand Prix	4:43.42	2012 Indianapolis Grand Prix
Kuramatsu, Mako	F	JPN	213	292.16	4.05	1.38	0.0065				4:52.16	Japan Inter-College Swimming Competition	4:48.11	2012 Japan National Championships
Tanaka, Mizuho	F	JPN	259	291.18	4.76	1.63	0.00631	18.6			4:51.18	National Games - Swimming Competition	4:46.42	Japan Open 2012
Suvorau, Yury	M	BLR	390	267.71	6.36	2.37	0.00609	21.36			4:27.71	2011 Open Champ of Belarus	4:21.35	European Swimming Championships
Satou, Gou	M	JPN	213	271.41	3.46	1.27	0.00598				4:31.41	Japan Inter-College Swimming Competition	4:27.95	2012 Japan National Championships
Weiss, Michael	M	USA	328	261.35	5.04	1.92	0.00587				4:21.35	2011 Conoco Phillips National Championships	4:16.31	USA Olympic Team Trials
Gilliland, Jared	M	AUS	298	270.15	4.66	1.72	0.00578				4:30.15	XLIV Santa Clara International	4:25.49	2012 Australian Age Champs
Kron, Kristian	M	SWE	364	271.52	5.47	2.01	0.00553				4:31.52	Swedish Championships	4:26.05	Swedish Championships 2012
Le Clos, Chad	M	RSA	327	256.88	4.64	1.8	0.00552				4:16.88	10th All African Games - Maputo 2011	4:12.24	2012 Olympic Games
Fraser-Holmes, Thomas	M	AUS	349	256.75	4.94	1.92	0.00551				4:16.75	2011 Telstra Australian Swimming Championships	4:11.81	2012 EnergyAustralia Swimming
Evans, Blair	F	AUS	301	280.86	4.65	1.65	0.0055				4:40.86	2011 Telstra Australian Swimming Championships	4:36.21	South Australian State Open Championships
Adams, Cammile	F	USA	304	282.99	4.72	1.66	0.00548	20.906			4:42.99	2011 Conoco Phillips National Championships	4:38.27	2012 ST TXLA Elite Invite
Tikhonov, Alexander	M	RUS	318	259.73	4.49	1.72	0.00543				4:19.73	2011 Barcelona Mare Nostrum	4:15.24	Russian National Swimming Championships
Fujino, Maiko	F	JPN	359	283.97	5.42	1.9	0.00531	29.2	1.61	57	4:43.97	2011 International Competition Selection Trial	4:38.55	2012 Japan National Championships
Hagino, Kosuke	M	JPN	435	254.58	5.64	2.21	0.00509	17.98	1.75	70	4:14.58	2011 Japan Open	4:08.94	2012 Olympic Games
Muir, Ross	M	GBR	399	264.23	5.13	1.94	0.00486				4:24.23	British Gas ASA National Championships	4:19.10	British Gas Nat Youth MC Chmps 2012
Oldershaw, Marni	F	CAN	363	291.56	5.07	1.73	0.00479				4:51.56	World Trials	4:46.49	2012 Canadian Olympic and Paralympic Trials
Stevens, Leah	F	USA	405	295.04	5.62	1.9	0.0047	15.6			4:55.04	2011 SE NAC Summer Sizzler	4:49.42	2012 ST TXLA American LC Champ
Stitski, Monika	F	CAN	323	295.23	4.45	1.5	0.00466				4:55.23	2011 IA Speedo Champions Series - CZ Section 1	4:50.78	2012 Mutual of Omaha Swimvitational
Marin, Luca	M	ITA	428	257.04		1.94	0.00454				4:17.04	Italian Spring National Championships	4:12.04	49th Sette Colli Trophy
Brandon, Bonnie	F	USA	347	293.81	4.61	1.56	0.00452				4:53.81	2011 CA Los Angeles Invitational	4:49.20	USA Olympic Team Trials
Wallburger, Tim	M	GER	388	264.05	4.63	1.75	0.00451				4:24.05	2011 German open	4:19.42	Qualification for European Championships
Martynova, Yana	F	RUS	365	283.24	4.55	1.6	0.0044				4:43.24	Russian National Championships	4:38.69	Russian National Swimming Championships
Tranter, Daniel	M	AUS	398	260.84	4.46	1.7	0.00429	20.6	1.93	80	4:20.84	2011 New South Wales Open Champs	4:16.38	2012 EnergyAustralia Swimming

Direct data download

Swimmers who gained better performance improvement than Ye Shiwen over the 2011-2012 period

name	sex	country						age	height	weight		2011 best performance		2012 best performance
Trombetti, Luisa	F	ITA	81	293.7	5.9		0.0248	19.6			4:53.70	Italian Open Winter Nationals 2011	4:47.80	Italian National Spring Championships
Rousseau, Sebastien	M	RSA	157	266.95	9.12	3.41	0.0217	21.9	1.9	80	4:26.95	Minneapolis Grand Prix	4:17.83	Sa National Aquatic Championships
Meyer, Michael	M	RSA	136	263.87	7.45	2.82	0.0207	24.66			4:23.87	Open Dutch Championships	4:16.42	Sa National Aquatic Championships
Chen, Xinyi	F	CHN	167	289.95	8.94	3.08	0.0184	14.6			4:49.95	7th National City Games 2011	4:41.01	National Swimming Championships
Mann, Becca	F	USA	227	290.17	8.56	2.94	0.0129	14.6			4:50.17	Minneapolis Grand Prix	4:41.61	USA Olympic Team Trials
Seltenreich-Hodgson, Erika	F	CAN	311	294.3	10.5	3.56	0.0114	17.6			4:54.30	2011 Canadian Age Group Championships	4:43.80	Mare Nostrum
Coton, Quentin	M	FRA	241	268.85	6.63	2.46	0.0102	21.04			4:28.85	Summer Nationals	4:22.22	Elite French Championships
Beraud, Romain	M	FRA	241	270.91	6.53	2.41	0.01	23.98			4:30.91	Youth Senior Open Championships	4:24.38	Elite French Championships
Tutumi, Takahiro	M	JPN	213	265.35	5.47	2.06	0.00967	19.9			4:25.35	Japan Inter-College Swimming Competition	4:19.88	2012 Japan National Championships
Siwicki, Breanne	F	CAN	250	293.79	7.01	2.38	0.00954	17.6			4:53.79	Summer Nationals	4:46.78	2012 Canadian Olympic and Paralympic Trials
Wedel, Kevin	M	GER	309	268.84	7.64	2.84	0.00919	18.6			4:28.84	38th European Junior Championships 2011	4:21.20	124th German Championships
Soerensen, Katrine	F	DEN	245	293.96	6.58	2.23	0.00913				4:53.96	Summer Nationals	4:47.38	Danish Open
Mann, Karley	F	GBR	356	294.94	9.3	3.15	0.00885	18.6			4:54.94	British Gas ASA Youth Champs 2011	4:45.64	British Gas Nat Youth MC Chmps 2012
Glavnyk, Iryna	F	UKR	238	294.7	6.21	2.1	0.00885	16.24	1.74	58	4:54.70	38th European Junior Championships 2011	4:48.49	Open Ukrainian National Championships
Lowe, Danielle	F	GBR	356	295.9	9.1	3.07	0.00863	17.6			4:55.90	British Gas ASA Youth Champs 2011	4:46.80	British Gas Nat Youth MC Chmps 2012
McKee, Anton Sveinn	M	ISL	281	270.15	6.51	2.4	0.00857				4:30.15	38th European Junior Championships 2011	4:23.64	Icelandic National Championships
Persson, Erik	M	SWE	260	271.82	5.86	2.15	0.00829	18.6			4:31.82	38th European Junior Championships 2011	4:25.96	Danish Open
Kochetkova, Kristina	F	RUS	261	294.48	6.08	2.06	0.00791	19.52			4:54.48	Russian National Swimming Cup	4:48.40	International Meeting
Gyurta, Gergely	M	HUN	281	261.9	5.34	2.03	0.00725	20.903	1.74	68	4:21.90	CXIII Hungarian Swimming Championship	4:16.56	Open National Championships
Schepers, Jan David	M	GER	282	264.71	5.24	1.97	0.00701				4:24.71	German Championships	4:19.47	Qualification for European Championships
Vanderkaay, Peter	M	USA	398	263.28	7.06	2.68	0.00673	28.48	1.93	90	4:23.28	2011 Michigan Grand Prix	4:16.22	2012 NC Charlotte UltraSwim
Ye, Shiwen	F	CHN	285	273.66	5.23	1.91	0.0067	16.27	1.6	45	4:33.66	7th National City Games 2011	4:28.43	2012 Olympic Games

Direct data download

References

[1] Ewen Callaway, Why great Olympic feats raise suspicions, http://www.nature.com/news/why-great-olympic-feats-raise-suspicions-1.11109
[2] FINA, "Swimming World Rankings" data base, http://www.fina.org/H2O/index.php?option=com_wrapper&view=wrapper&Itemid=805

Relearning p-value

Tue, 16 May 2017 23:59:39 +0000

After reading "The ASA's statement on p-values: context, process, and purpose", and some other related references, here are some excerpts and notes I took on p-value and null-hypothesis significance testing.

American Statistical Association (ASA) has stated the following five principles about p-values and null hypothesis significance testing:
1. "P-values can indicate how incompatible the data are with a specified statistical model."
2. "P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone."
3. " … It is a statement about data in relation to a specified hypothetical explanation, and is not a statement about the explanation itself."
4. "Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold."
5. "… Practices that reduce data analysis or scientific inference to mechanical “bright-line” rules (such as “p < 0.05”) for justifying scientific claims or conclusions can lead to erroneous beliefs and poor decision-making. …"
6. "Proper inference requires full reporting and transparency."
7. "A p-value, or statistical significance, does not measure the size of an effect or the importance of a result."
8. "… Smaller p-values do not necessarily imply the presence of larger or more important effects, and larger p-values do not imply a lack of importance or even lack of effect. Any effect, no matter how tiny, can produce a small p-value if the sample size or measurement precision is high enough, and large effects may produce unimpressive p-values if the sample size is small or measurements are imprecise. …"
Null hypothesis is usually a hypothesis that assumes that observed data and its distribution is a result of random chances rather than that of effects caused by some intrinsic mechanisms. It is usually what is to disapprove or to reject in order to establish evidence to or belief in that there is some real effect due to underlying intrinsic mechanism. In turn, the details of the statistical model used in this evaluation can be used to make quantitative estimations on properties of the underlying mechanism.
The p-value is the probability that one has falsely rejected the null hypothesis.
- The smaller is, the smaller the chance is that one has falsely rejected the null hypothesis.
- Being able to reject or not being able to reject the null hypothesis may tells one if the observed data suggests that there is an effect, however, it does not tell one how much an effect there is and if the effect is true. See effect size.
- "a p-value near 0.05 taken by itself offers only weak evidence against the null hypothesis".
- UK statistician and geneticist Sir Ronald Fisher introduced the p-value in the 1920s. "The p-value was never meant to be used the way it's used today."
As ASA p-value principle No. 3 states, the decision to reject the null hypothesis should not be based solely on if p-value passes a "bright-line" threshold. Rather, in order to reject the null hypothesis, one must make a subjective judgment involving the degree of risk acceptable for being wrong. The degree of risk of being wrong may be specified in terms of confidence levels which characterizes the sampling variability.
Alternative ways used for referring to data cherry-picking include data dredging, significance chasing, significance questing, selective inference, p-hacking, snooping, fishing, and double-dipping.
"The difference between statistically significant and statistically insignificant is not, itself, statistically significant."
"According to one widely used calculation [¹], a p-value of 0.01 corresponds to a false-alarm probability of at least 11%, depending on the underlying probability that there is a true effect; a p-value of 0.05 raises that chance to at least 29%." See the following figure:

p-value and probable cause.png

Some related concepts

The standard score, or z-score is the deviation from the mean in units of standard deviation. A small p-value corresponds to a large positive z-score.
68-95-99.7 rule
MAGIC criteria.
- Magnitude - How big is the effect? Large effects are more compelling than small ones.
- Articulation - How specific is it? Precise statements are more compelling than imprecise ones.
- Generality - How generally does it apply?
- Interestingness - interesting effects are those that "have the potential, through empirical analysis, to change what people believe about an important issue".
- Credibility - Credible claims are more compelling than incredible ones. The researcher must show that the claims made are credible.

References

"The problem with p-values: how significant are they, really?", phys.org Science News Wire, 2013, http://phys.org/wire-news/145707973/the-problem-with-p-values-how-significant-are-they-really.html
Regina Nuzzo, "Scientific method: statistical errors," 2014, http://folk.ntnu.no/slyderse/Nuzzo%20and%20Editorial%20-%20p-values.pdf
Tom Siegfried, "Odds Are, It's Wrong - Science fails to face the shortcomings of statistics," 2010, https://www.sciencenews.org/article/odds-are-its-wrong
Gelman, A., and Loken, E., "The Statistical Crisis in Science," American Scientist, 102., 2014, http://www.americanscientist.org/issues/feature/2014/6/thestatistical-crisis-in-science
"The vast majority of statistical analysis is not performed by statisticians," simplystatistics.org, 2013, http://simplystatistics.org/2013/06/14/the-vast-majority-of-statistical-analysis-is-not-performed-by-statisticians/
"On the scalability of statistical procedures: why the p-value bashers just don't get it," simplystatistics.org, 2014, http://simplystatistics.org/2014/02/14/on-the-scalability-of-statistical-procedures-why-the-p-value-bashers-just-dont-get-it/
Andrew Gelmana and Hal Sterna, The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant, The American Statistician, Volume 60, Issue 4, 2006, http://www.tandfonline.com/doi/abs/10.1198/000313006X152649

Goodman, "Of P-Values and Bayes: A Modest Proposal," S. N. Epidemiology 12, 295–297 (2001), http://journals.lww.com/epidem/fulltext/2001/05000/of_p_values_and_bayes__a_modest_proposal.6.aspx ↩

Compute sample variance of data stream

Tue, 06 Oct 2015 19:39:59 +0000

Sample variance of a data stream $\left \{ X_1, X_2, \cdots, X_N, X_{N+1}, \cdots \right \}$ can be computed without saving the data points individually, but only sample mean and sample variance values for the current sample size $N$:

$$ \begin{align} s² N &= \frac{1}{N} \sum {i=1}^N \left( X i - \bar{X} N \right)² \\ &= \frac{1}{N} \sum {i=1}^N \left( X i² - \bar{X} _N² \right)~, \end{align} $$

where

$$ \begin{equation} {\bar{X}} N = \frac{1}{N} \sum {i=1}^N X _i~. \end{equation} $$

Store $N$, $\bar{X} N$, and $s N²$. When datum $X _{N+1}$ is obtained, the values can be updated as the following:

$$ \begin{equation} \bar{X} {N+1} = \frac{1}{N+1} \left( N \bar{X} N + X _{N+1} \right)~, \end{equation} $$

$$ \begin{align} s² {N+1} &= \frac{1}{N+1} \sum {i=1}^{N+1} \left( X i² - \bar{X} {N+1}^2\right) \\ &= \frac{1}{N+1} \sum {i=1}^{N+1}X i² - \bar{X} {N+1}^2 \\ &= \frac{1}{N+1} \left(N s N² + \bar{X} N² + X {N+1}^2 \right)- \bar{X} _{N+1}^2~. \end{align} $$