Statistical Calculator

Ranked Data

This calculator can be used to analyse data from parapsychological experiments in which data represent rank order judgments of the match between a target and a response.

This procedure is widely used in free response ESP experiments in which recorded "impressions" about a possible target are later ranked on the basis of their similarity to a judging set of several possible targets (the actual target plus one or more decoys).

The ranking may be done either by the person whose impressions are being examined, or by one or more independent judges. Of course it is vital that when the ranking is done, the actual target is not known by the person making the judgments. If using more than one judge, the rankings from the different judges should be averaged. The average ranks may then be used to determine the majority rank position. The following examples illustrate the procedure.

Ranking Procedure

First, select the procedure used to rank the data. If a rank of 1 was used to indicate the preferred target, choose "Hit = low rank". If the preferred target was given the top rank (e.g., 4 in the above example), choose "Hit = high rank".

Number of Targets in Judging Set

Next, select the number of targets used in the judging set (this must be the same for all trials). In the above example, this = 4.

Trials and Sum of Ranks

Next enter the total number of trials and the Sum of Ranks (across all trials) given to the actual target. Then click "Calculate".

In the above example, trials = 3, Sum of Ranks = 5 (for one judge), Sum of Ranks = 3 (for several judges).

If there are sufficient data (at least 10 trials), statistical analysis will be performed.

The statistical procedure used is the z-test performed on the Sum of Ranks (sometimes called the "Sum of Ranks Test"). This compares the obtained Sum of Ranks with the Sum of Ranks expected by chance. The latter is known as the Mean Chance Expectation (MCE) and is equal to (number of trials) x (average of the ranks in the judging set). For example, if there are 4 targets in the judging set, the average rank = (1+2+3+4)/4 = 2.5. If 50 trials are completed, MCE = 50 x 2.5 = 125.

The z test calculates the probability (p) of obtaining results which differ from the MCE to the extent observed. If this probability is low (generally 0.05 or less), the results are said to be statistically significant and are therefore possible evidence of psi, or paranormal ability. Depending on the direction of the results, the analysis reports the results for either psi-hitting or psi-missing, as appropriate.

If there is a strong prior expectation of either psi-hitting or psi-missing, the statistical analysis may be carried out on the basis of a one-tailed test. A one-tailed test does not examine the possibility that the non-expected outcome may result. Although the probabilities obtained will be halved, a one-tailed test is not generally recommended and, in most cases, statistical analysis should be two-tailed (which tests for both psi-hitting and psi-missing).