Maxine Roberts, Ph.D., Director, Strong Start to Finish
Early in my career as an educator, I worked in after-school programs that provided supplemental support for middle and high school students so they could be academically successful in school. Some of my students were referred to our program because their teachers labeled them “disruptive,” “low achieving,” or “troublemakers” as compared to their peers. But these labels didn’t accurately describe the students we got to know during those hours after school.
To support our students’ success, we knew we needed to change how we worked with and evaluated them. In the program, we focused on learning more about their lives as we engaged with them and used what we learned to design programs that fostered their growth and helped them cultivate their own successes. We learned to ask questions that ultimately helped us change our practices to support them effectively.
We knew that comparing our students to their school peers did not serve them nor the program well. These comparisons framed our students negatively, and at the same time, held those they were being compared to as the norm, and even as superior. In fact, we knew such comparisons were made without much understanding about students’ lives and their experiences.
Observing outcomes within student groups
Addressing what we know about certain student groups, rather than solely focusing on comparing their outcomes drives my thinking of a broader approach that we can use to measure student outcomes.
Standard practices for identifying who is successful, who is not, and measuring student achievement typically involves the use of normative measurement practices. Often, these measurements involve comparing outcomes across student groups — for instance, comparing pass rates between students from different racial/ethnic groups or students with different socioeconomic statuses — to provide information about their achievements.
Such comparisons are often the way that we think about (in)equity, where measurements are used to answer questions such as: “Who is performing the best relative to others and who is doing worse?” These questions result in current policy prescriptions that focus on raising the “worst” to the level of the “best.” While such comparisons provide measures for student progress, exclusively using these types of observations encourages gap gazing — a process of focusing primarily on the differences in outcomes between student groups — says little about growth within student groups over the period, and does not provide nuanced information about students that can be useful for their success.
If we’re seeking ways to understand student outcomes and to determine whether or not we are removing barriers to student success, one practice that is promoted by education scholars is to focus on advancement by calculating and observing outcomes within student groups.
Figure 1 shows two different ways of presenting outcomes data, demonstrating the difference between comparisons made between groups and within groups.
1: Comparison Between Groups
2: Comparisons Within Groups
The visualized charts in this blog provide helpful information about students while prompting different points for discussion about their academic outcomes. For instance, Chart #1 shows a four-percentage point difference in outcomes between Pell and Non-Pell recipients during Semester 1 and variation in outcome differences over semesters 2 and 3. Those who focus on identifying gaps between groups will likely point out that the difference between outcomes shrinks to two-percentage points by Semester 4. The data on pass rates for the student groups in Chart #2 encourages readers to observe different outcomes, including gains for Latinx Pell grant recipients during the period and results for Latinx Pell grant recipients in their first year by their entering GPAs. Exploring the characteristics of student groups provides information that extends beyond differences in outcomes and prompts us to deepen our understanding of students who are often the focus of expanding or closing differences in academic outcomes.
Combining between- and within-group comparisons to address inequities.
The widespread use of between-group comparisons in education indicates that this method of collecting and comparing data will continue into the foreseeable future. Therefore, it is important to change how we use between-group comparisons by pairing these with within-group comparisons to provide a view of students that is holistic, asset based, and helps us understand what we need to advance success for learners who are marginalized in education. In combination, between-group and within-group comparisons can advance efforts to address inequities in education. For instance, between-group comparisons can reveal which student groups are disproportionately impacted by a reform or practice; and within-group comparisons can help us learn about these students’ characteristics, their outcomes, and particular achievement patterns that would be useful for supporting their success. Key in this practice is determining what one wants to learn about student groups to advance their success, then collecting appropriate data to answer those questions.
A popular sentiment emphasizes that when we achieve educational equity, we will not be able to determine students’ academic outcomes by their personal characteristics or zip codes. Within-group comparisons prompt thinking about how to support the achievement of student groups who are marginalized by identifying ways to measure these learners’ advancements, expand our knowledge about them, and identify the institutional barriers that positively and negatively shape their achievement.
