What makes good research? New Concordia speaker series examines inclusivity and the shifting landscape of evaluation

For decades, academia has relied on quantitative metrics like publication counts and journal rankings to assess the value of research.

But do these figures paint a full and accurate picture? Or do universities need to rethink academic evaluation, recognizing the diverse outputs, practices and activities that contribute to research quality and impact?

Concordia’s Office of the Vice-President, Research and Graduate Studies and Graduate Studies, in collaboration with the Office of the Provost and Vice-President, Academic, have recently launched a speaker series exploring these questions.

The series, Evaluating Excellence: Rethinking Academic Assessment, kicked off on November 16, 2023, at Concordia’s 4TH Space. Giovanna Lima, research impact officer at the Erasmus University of Rotterdam in the Netherlands, presented a talk and workshop titled “Demonstrating Scholarly Impact.”

Moderated by Tanja Tajmel, Concordia University Research Chair in Equity, Diversity and Inclusion in Science Technology Engineering and Mathematics, the series continues throughout the winter term with five more events.

It will incorporate different formats to address key questions, such as the challenges with traditional metric-based assessment, the relationship between open science practices and assessment reform, and how assessment reform can support decolonization, indigenization and the well-being of academics.

An international reform movement

“This series is intended as a platform for meaningful discussions on the evolving landscape of inclusive excellence and academic assessment,” says Anne Whitelaw, provost and vice-president, academic.

The speaker series is also a response to recommendations for reform arising from the Working Group on the Impact of the COVID-19 Pandemic on Faculty and Librarians and the Concordia Open Science Working Group. However, a broader reform movement is also underway, underpinned by several international manifestos and agreements.

One such agreement is the Declaration on Research Assessment (DORA). Core commitments include broader recognition of the diverse practices, activities and careers in research, and a shift from relying primarily on quantitative indicators. Rather than using traditional indicators such as the “journal impact factor” and “h-index,” supporters agree to move toward more holistic peer evaluations of research quality and impact.

Signatories to DORA include the Canadian Institutes of Health Research (CIHR), Natural Sciences and Engineering Research Council of Canada (NSERC) and Social Sciences and Humanities Research Council of Canada (SSHRC), alongside several universities.

Reimagining evaluation at Concordia

For Monica Mulrennan, associate vice president of research, development and outreach, the time has come for the Concordia research community to discuss academic assessment.

“Increasing evidence suggests that the use of quantitative metrics alone creates a variety of skewed incentives for researchers. These incentives almost surely do not provide a complete picture of the true impact and value of academic work,” Mulrennan explains.

The series’ objectives are ambitious yet essential:

• Investigate how other institutions, within and beyond Canada, are reimagining evaluation methods
• Learn from institutional initiatives adopted elsewhere, embracing best practices and lessons learned
• Bring attention to concerns and challenges associated with traditional metric-based assessment practices
• Champion equity and open science as keystones of reform
• Prepare Concordia to be at the vanguard of evaluation reform, ready to embrace changes in national and international norms

Mark your calendars and RSVP to the next installment of Evaluating Excellence on January 31.

Learn more about Concordia's equity, diversity and inclusion efforts.