What is evaluation? Perspectives of how evaluation differs (or not) from research

Results 
Demographics 
A total of 233 AEA members (out of 985; 23.7%) and 499 AERA members (out of 1,563; 
31.9%) participated in this study for a response rate of 28.7%; furthermore, an additional 55 
participants were recruited from social media. Of those who participated in the study, 165 AEA 
members, 330 AERA members, and 25 people from social media completed the entire survey (n 
= 520), for an average completion rate of 66.1%, with completion rates of 70.8% (AEA), 66.1% 
(AERA), and 45.5% (social media). All subsequent analyses are conducted with the valid sample 
for the variables in question, meaning sample sizes will vary depending on the variables used as 
the ns are not the same across variables. 
There were roughly equal numbers of participants who considered themselves primarily a 
researcher (n = 278, 51.0%) versus an evaluator (n = 263, 49.0%). As hypothesized, AEA 
members were more likely to consider themselves primarily evaluators (n = 134, 81.2%) than 
researchers (n = 31, 18.8%), and AERA members were more likely to consider themselves 
researchers (n = 232, 66.5%) than evaluators (n = 117, 33.5%), 
𝜒𝜒
2
(1) = 100.07, p < .001. Those 
who considered themselves primarily evaluators were also more likely to be a member of an 
evaluation association such as AEA (n = 207, 78.7%) compared to researchers (n = 101, 36.3%), 
𝜒𝜒
2
(1) = 97.26, p < .001. Furthermore, evaluators were more likely to have taken evaluation-
specific courses in post-secondary education, 
𝜒𝜒
2
(3) = 18.34, p < .001; professional development 
courses, 
𝜒𝜒
2
(3) = 56.74, p < .001; and conduct more evaluation than research work, 
𝜒𝜒
2
(5) = 
323.95, p < .001. Most analyses were performed comparing participants based on whether they 

---

considered themselves primarily an evaluator or researcher; however, analyses were also 
performed comparing participants who were AERA, AEA members in one of the youth TIGs 
(i.e., PreK-12 Educational Evaluation TIG or Youth-Focused Evaluation TIG), or AEA members 
not in one of the youth TIGs. When discrepencies existed between these two analyses, both are 
discussed.