Quantitative Analysis in Program Evaluation

Quantitative data analysis is an important part of the program evaluation process. Basic statistics help us understand and communicate educational program results.

There are several ways an evaluator can approach statistically analyzing their data, and this approach depends on (1) the research question and (2) the data at hand. For example, the type of statistical test an evaluator may use can depend on if they are interested in testing mean differences or predicting an outcome. Further, the statistical test an evaluator may use will also depend on the variable type (e.g., continuous, categorical, interval, etc.).

For most evaluation and outcome analysis, the most basic and useful analysis is to compare measurements taken at two points in time. A

meaningful measure will be a mean, or average, of the values collected. For example, for a survey administered at the beginning and end of an educational program that collects data on college students’ STEM efficacy, we could use a t-test to see if there is any significant difference between the two average values. An ANOVA will compare two or more means simultaneously and also takes into consideration the variance within a sample and between samples.

Basic statistics that an evaluator may use are:

1. Descriptive statistics: Descriptive statistics are useful for when an evaluator has data that they would like to describe. Tests within this category include means (averages), percentages, or range.

2. Inferential statistics: Inferential statistics allow the evaluator to make inferences (predictions) about the data.

a. Comparison tests: Comparison tests are useful for when an evaluator would like to test the group mean differences between or within participants. Common tests include: t-test, paired t-test (for pre- and post- test data), and ANOVA.

b. Correlational tests: Correlational statistics are useful for when an evaluator would like to examine the relationship between variables or predict an outcome. Common tests include: correlation, regression, and hierarchical linear modeling.