A Shift in Measurement Mindset

When I think about mindful measurement, as the structure of the word ‘mindful’ suggests, I think of what fills the mind with respect to an assessment. What fills students’ minds when they take an assessment? When they see their scores? What fills teachers’ minds when they administer an assessment? When they see their students’ scores? What fills parents’ and guardians’ minds as their children prepare for assessments? When they see their children’s scores? What do we want to fill their minds with?

To a large degree, I believe that we, as measurement specialists, dictate what fills their minds. Consider the student: what fills their mind is determined by the content of assessment, but also their preparation for the assessment, the design of the assessment, the purpose and stakes of the assessment, and the perceived utility of the assessment, to name a few. I think most would agree that we want their minds to be filled with positive and freeing thoughts that allow them to perform their best on the assessment as well as grow from the assessment experience while recognizing the utility of the assessment. But we aren’t there yet. So how do move in the direction of this assessment utopia?

It’s hard to say for sure, but I have an idea. This is not my personal idea, but one that I support fully and have devoted my research to, and one that has many measurement folks excited. I believe we need a paradigm shift: a shift away from unidimensional assessments designed to locate student ability on a continuum and towards multidimensional assessments that provide diagnostic student profiles. Instead of an overall ability level in a particular area of study (e.g., 4th grade mathematics), these profiles would indicate student levels of understanding on targeted concepts or skills (e.g., standards). There are of course complexities involved with this sort of paradigm shift, but I want to focus on what fills the minds of those involved under each paradigm. Let’s consider a student who scored at the 20th percentile on the ability continuum. What fills their mind? Feelings of being less than, other than? Hopelessness, as there is not a clear path forward? Was the assessment useful for them on their learning journey?

Now consider the same student in a parallel universe where they have mastered 3 out of 8 concepts. What fills their mind? Some happiness for the 3 concepts mastered? Motivation to master the other 5 standards? Growth and hope, as they have a clear path forward, with time and means to master the other 5 concepts? Was the assessment useful for them on their learning journey? I would argue “yes”, as the assessment helped identify the student’s strengths and areas for improvement. Similar thoughts would fill the minds of their teachers and parents. Imagine how, if students took diagnostic profile-based assessments all year long, how their mindsets toward assessment, and general educational outlook, could change.

If this were the case, because of the assessment utility and clarity, I don’t think parents would be boycotting and revolting against large-scale assessments. I don’t think teachers would dread assessment days. To be clear, I’m not suggesting abolishing the current measurement paradigm. I’m suggesting that infusing some profile-based measurement into the current framework would better support teaching and learning, and potentially foster student confidence, self-worth, ownership, investment, and engagement in their education. For the past couple of years, I have been working with Navvy Education, a diagnostic and standards-level assessment system operating in several school districts in Georgia. Though its early, we are already seeing evidence of positive educational and socio-emotional changes from students, teachers, and school leaders.

It’s vital that measurement specialists not only consider the psychometric quality of assessments, but that we are mindful of the social and emotional impact of assessments on students’ educational journeys. This is especially the case when we think about fairness and equity in testing; we not only have to consider substantive and semantic properties of items and validity of inferences; we must consider how assessments can differentially impact students. How can an assessment, its implementation, score interpretations, and usages differentially impact students at different levels of performance? Does the assessment engage and challenge students with advanced understandings? Does it motivate or deflate students whose skills are still emerging? If we desire a truly fair and equitable assessment system, we must consider its socio-emotional impact on students’ learning journeys.

While this post focuses on educational measurement, this measurement paradigm shift is also relevant to daily life in how we measure and think about ourselves in relation to others. In 2013, I attended my first National Council on Measurement in Education conference in San Francisco; the preeminent educational measurement organization in the United States. I was a first-year graduate student, just learning about the field so I went to every session and meeting I could. One of the meetings I attended was an awards reception, where attendees just walked around and mingled. I remember walking in and seeing so many measurement legends. I was star-struck! I met a few of them and they were kind and super down to earth. But I found myself thinking, “what am I doing here? I will never be as smart or knowledgeable as they are, I will never achieve what they have.” I left the reception feeling dejected. It took me years before I started to look at things from a different perspective, a profile-based perspective, which changed my outlook.

Every person in that room has a unique profile of infinite length that makes them who they are. Visually, this profile looks like a bar chart with an infinite x-axis. The profile starts with their genes, but then builds with every life experience. When I think about things from this profile-based perspective, my mind is no longer filled with thoughts of being “less than”; instead, my mind is filled with thoughts of being “different”, and different is good. My measurement knowledge bar may never reach the level of the measurement legends that were in that room, and that’s ok. Because there are so many other bars that I possess that are unique to me, and those bars are just as valuable as the measurement knowledge bar. So now, instead of being upset about the bars I do not have, I’m thankful for the bars I do have, and I’m excited about the bars I can potentially acquire or increase through life-long learning and experiences.

I believe this mindset change can be useful for everybody, but particularly useful for Black, Indigenous, and other people of color who are underrepresented and may sometimes feel out of place. You matter, your opinions and experiences matter, you are valued, your unique profile is necessary for the field of educational measurement to grow, and your unique profile is vital for the diversification of the next generation of educational measurement scholars.

I encourage us all to think about how a change in measurement mindset can change our outlook and the outlook of others, whether we are developing a targeted educational assessment, reviewing applications, or self-assessing. I think it’s clear, both formally and informally, in education and outside of education, that the way we measure ourselves and others undoubtedly alters what thoughts, values, and visions fill our minds. This reality means our power as measurement specialists is greater than ever. We have an obligation to use this power to change the world around us for the better.