Mindful Measurement – A Blog about Thoughtful Evidentiary Reasoning
Seemingly everywhere in the educational landscape, writ large, changes are afoot. These changes often seem to come at us at breakneck speeds – switching to online or blended learning for sharper diagnosis, integrating new EdTech products into our ecosystem of resources, using artificial intelligence to parse through oceans of data and information, and so on.
Similarly, the global job market is in the middle of a relentless transformation that requires many of us to consider new career trajectories, new pathways for upskilling, and new professional identities. Being a data scientist, learning designer, or UI/UX specialist is an incredibly attractive moniker right now and quite a few jobs can be found across industries for people who assign themselves such labels. This pattern holds even as broader job markets are caving in or are only beginning to reconstitute themselves.
Countless job ads ask future employees to be “ready to hit the ground running” while working with “enthusiastic, highly motivated colleagues” in a “fast-paced, quickly changing environment” that uses “agile development cycles” with “frequently changing objectives”. This seems to beg the question then: why can’t we all just be faster, more efficient, more reliable, and become more like machines, ideally yesterday?
In this blog, I am interested in the seemingly opposite – the islands of calm and quiet amidst a sea of change while thunderstorms are brewing. That is, I am interested in how human beings, with all of their complexities, fallacies, and beautiful characteristics, take the time to “slow down” and reflect on the way they are going about the work in order to be more principled, thoughtful, mindful.
Rather than any particular framework for educational assessment, EdTech, or other related fields, I want to use a mindset to guide the conversation – the mindset of “mindfulness”. While mindfulness can have varied nuanced definitions and operationalizations, the simple perspective of “being thoughtful, aware, and focused on the present moment” is a good starting point. This is why this blog and my website are entitled “mindful measurement” – I want to call out the practice of how individuals and teams connect with one another, structure their workflows, leverage their technologies, and create shared understandings and practices from which to enhance their principled work in educational measurement.
Of course, calling out mindfulness is not to suggest an either-or distinction between two states with mindfulness being always preferable. To be clear: we need ambitious, passionate, and infinitely energetic entrepreneurs who drive our disciplines and fields forward and show us possible roadmaps, exist points, and strategic landmarks. However, we also need team members, including leaders, who are not afraid to embrace holistic, humanistic, and mindful practices in order to help us navigate the resulting landscape and, I would argue, help us discover alternative routes, vantage points, and treasures along the way.
To me, there is a visceral beauty in taking the time to connect with other human beings mindfully while doing technologically-centered work in learning and assessment spaces. For me, this has always been a part of how I engaged with disciplines. My own research interests have historically centered around synthesizing, integrating, and translating complex ideas into guiding frameworks and essays that may help others as some critical questions about the work that they are doing.
For example, work in diagnostic measurement – in this case, how to use modern latent-variable methods and related tools to classify learners into states that have resonance with stakeholders – is critically about how more nuanced theories get operationalized, evaluated, and refined through measurement models and surrounding practices. Work in automated scoring also challenges us to use emerging tools to better understand what learners are doing, why they are doing it, and how we can help them and teach them to help each other. There are of obviously other uses to these toolkits but what has always struck me is their potential for deeper, richer explanations – a potential that only gets realized in certain moments, when mindful measurement is part of the equation.
As the instruments that we create and deploy in educational spaces change and become more sophisticated, multifaceted, and data-rich, our opportunities for understanding situated engagement in the associated ecosystems become larger and more challenging. Theories become more complex, systems of assumptions and design decisions become more complex, and technological solutions become more complex – thus making it intellectually more complex to create convincing, situated, and differentiated evidence-based stories in this world. Being successful at this work requires breakneck-speed technological innovation – yes, certainly – but it also requires that thoughtful, mindful human beings are in the loop who are able to separate noisy patterns from plausible explanatory narratives and champion such mindsets and practices. Ideally, with mission-driven principles grounded in compassion, diversity, and humanity. Whenever we unlock these mutual “aha-moments” and build one another up through this work, infinite creative potential can be released.
Often this work is not particularly glamorous, obvious, or quantifiable. It is certainly not singularly oriented towards immediate returns-on-investment, but when done well there is a true beauty in the mutual discoveries that await. Moreover, in well-functioning interdisciplinary teams the work can be principled, data-driven and colorful, creative without one dominating or suffocating the other. In other words, it can be a truly holistic human experience. In order to do this work well, then, we need to collectively develop and fine-tune the same kinds of complex competencies that we say we want to teach to and assess in our learners through the systems we create.
I have had many conversations with friends and colleagues over the years around these issues – individually over coffee, publicly in panels, or semi-publicly through disseminations. There is frequent acknowledgement of how hard it can be to do this principled, deeply humanistic work in cross-functional teams amidst all kinds of operational pressures from organizations. I could cite many individuals or frameworks that I respect and admire in this regard, but this blog is not about proving that I have read certain literatures. Instead, it is about creating a safe space and small microcommunity where colleagues can share their experiences, lessons learned, and recommendations for how to do mindful measurement together.
To this end, I have invited several colleagues to contribute to this blog and will post their entries here and on LinkedIn as they come in. Every few entries I will take the opportunity to look across past contributions and share some reflections. First and foremost, I would love this effort to encourage us all to slow down for a second, appreciate the humanistic side of our work, engage in metacognition and reflection, and share our unique colors with others in our field.
I hope you will be able to embrace this space and make it a small part of your personal ecosystem of inspirational resources. I would love to hear from you about how you are approaching mindful measurement in your own practice – via a comment, an invited blog entry, or just a chat over the phone!
All the very best to you and your loved ones from Princeton.
André A. Rupp
Founder, Mindful Measurement
