Snap’s Statistics
March 2003
Snap Survey Software boxThis month we look at the statistics features in Snap; what they are and how they are used. Snap provides Summary Statistics and Significance Testing. Summary statistics reduce large amounts of information to a single figure, thereby allowing comparisons between two or more sets of data. Significance Testing provides a measure of how confident we are that the results obtained for the survey reflect the true pattern of response in the population at large.
Summary Statistics
Summary Statistics include averages, measurements of the spread of data values and the divergence of the data values from normal distribution patterns.
Mean, Mode and Median
All three are types of average and are measures of the location (size) of the data values. The mean is calculated by adding up all the values and dividing by the number of them. The mode is the most commonly occurring value. The median is the value in the middle when all values are put in order from smallest to largest. If there is an even number of values (and hence two 'middle' values) then the median is calculated as the mean of the two middle values.
For example, 10 parties of visitors to a visitor attraction were interviewed and asked the question "How many people are in your party today?". The following answers were given:
1, 2, 3, 4, 3, 4, 5, 4, 6, 2
The mean, mode and median would then be
Mean = 3.4 (the sum of the answers divided by 10)
Mode = 4 (since 4 is the most frequently occurring answer)
Median = 3.5 (when arranged in order, the middle values are 3 and 4)
Minimum, Maximum, Range and Quartiles
The minimum, maximum, range and quartiles are used to measure the spread of the data values. Range is the difference between the minimum and maximum values. The quartiles are one-quarter and three-quarters of the way through the data. Range and inter-quartile range (the difference between the upper- and lower-quartiles) are crude measures of the spread of the data.
Variance, Standard Deviation and Standard Error
The Variance, Standard Deviation and Standard Error of the Mean measure the spread of the data in a more sophisticated way. They are used in significance testing and calculated taking every single data value into account. Variance and Standard Deviation measure how much each individual response differs from the mean of all responses. If the data are closely grouped, this difference will be relatively small. If the data are widely spread, the difference will be larger. Standard Deviation is the square route of Variance. Standard Error of the Mean is the Standard Deviation divided by the square root of the sample size.
Normal Distribution
Normal Distribution is also known as the bell-shaped curve. This is a pattern of data that fits many naturally occurring scenarios, e.g. height and weight of adult humans. Many statistical tests assume the data are distributed normally. Statistical tests exist to examine whether or not data are normally distributed.
Kurtosis and Skewness
Kurtosis and Skewness measure the degree to which a set of data values differs from the Normal Distribution. . It is very rare that either Kurtosis or Skewness are used or quoted.
Where they are required, Kurtosis measures the spread of the distribution curve – a positive value indicates that the distribution curve is more pointed than normal (that is, the data is clustered around the central mean); a negative value indicates that the distribution is flatter and more spread out than normal.
Skewness measures the extent to which the tails of the data may be more drawn out on one side than the other. A value is 0 or close to it indicates that the left and right-hand sides of the distribution are balanced. If the skewness is positive then the peak of the graph (the mode) is closer to the left hand side. In that instance, the mean value is greater than the median. If skewness is negative then the opposite is true: the mean is less than the median and the peak of the graph is moved to the right.
Significance Testing
Significance Testing is necessary where data is gathered from a sample and not from the entire population. Significance Testing tells us how confident we can be that the survey sample accurately reflects the views of the entire population. A Significance level is the probability that the result is true and not just a random variation. T-Tests, Confidence Intervals and the Chi-Squared Test are all forms of Significance Testing.
t-Test
The t-Test measures the likelihood that two results being compared could have been found purely by chance. It does this by comparing the mean value of two (and only two) sets of data. The t-Test is reported as a confidence level that the two scores being compared are related in the same way – for example, that one is smaller than the other. It says nothing about the actual mean values themselves.
The higher the confidence level, the more certain you can be that there really is a difference in the two groups being tested. For example, 95% confidence means that there is only a 5% chance that such a difference in scores could have been found purely through the effects of sampling. In Snap, simply check the t-test box on the Statistics tab of the Tailoring dialogue box.
Confidence Intervals
Confidence Intervals report two values – the confidence level and the confidence interval. The confidence level is the percentage likelihood at which the test was carried out (see t-Test above).
The confidence interval is the range of values within which it can be confidently said that the true value (for the entire population) is likely to lie, based on the value observed in the sample. For example, a survey result quoted as 48% +/-5% at the 95% confidence level means that we are 95% confident that the true result is between 43% and 53%. In Snap, check the Confidence box in the Results Definition dialogue box.
Chi-Squared Test for Independence
The Chi-Squared Test tests whether there appears to be a relationship between two variables or not. In Snap where you have a table with at least two rows and two columns, you can run a Chi-Squared Test by clicking on the upper case sigma.
The Test reports a confidence level if there is sufficient evidence of a relationship existing, or of no relationship existing. A high Chi-Squared value implies a relationship exists, conversely a low value indicates that no relationship is apparent. Its main use is with data where the different categories are in no particular order, e.g. gender, ethnicity.
The test cannot provide any indication of what the relationship is; this is left to the researcher to interpret. To this end, use can be made of Indexed values (click on the appropriate check-box in the results definition dialogue box) – a value greater than 100 indicates that there are more respondents in the cell than would be expected, a value of less than 100 indicates that there are less respondents than expected.
Summary
Snap provides many operations to calculate Summary Statistics and to perform Significance Testing. Applications of these methods enables researchers to understand the underlying trends behind their data and whether or not they are a true reflection of the entire population not just the survey sample.