Crawford-Howell Test

To place the score of an individual relative to the score of a sample, one can simply rank the scores and compute the percentile in which the individual score falls.

If one is ready to assume that the scores are distributed normally in the population where the samples comes from --- and that the distribution in the populations can be fitted from the sample mean and standard deviation --- then , one can then compute a z-score, dividing the distance of the individual's score to the group average by the sample's standard deviation

This last approach is not very appropriate for very small samples, where the estimation of the standard deviation can be underestimated.

Crawford & Howell (1998) proposed an improved score based on the Student distribution. This is can be seen as a t-test where the individual is treated as a sample of size 1. With x the individual score and n, μ and σ thesize, mean and standard deviation of the sample, the observed score is :

t = (x - μ) / (σ * (n - 1 / n))

which can be placed on the cumulative probability density of the Student t-distribution with n-1 degrees of freedom.

Python code

from numpy import mean, std
from scipy.stats import t

def CrawfordTest(case, controls):
    """ Implementation of Crawford and Howell (1998; The Clinical Neuropsychologist) test to compare an individual to a sample.

    Args:
      case : score of the individual
      controls : scores of from a sample group

    Return:
      the one-tail probability associated to the score 'case' compared to the scores in the list 'controls'
    """

    tobs = (case - mean(controls)) / std(controls, ddof=1)
    return t.cdf(tobs, len(controls) - 1)

R code

For an R implementation, see https://www.r-bloggers.com/2012/08/crawford-howell-1998-t-test-for-case-control-comparisons/ (beware: it is two tails)

Reference:

• Crawford, John R., and David C. Howell. 1998. “Comparing an Individual’s Test Score against Norms Derived from Small Samples.” The Clinical Neuropsychologist 12 (4): 482–86.