University rankings: Should we be concerned?
The Times Higher Education World University Rankings, The Globe and Mail Canadian University Report, Webometrics, QS World University Rankings, The Aspen Institute’s Beyond Grey Pinstripes, Maclean’s…
University rankings—national, regional, global—proliferate. Some can even be downloaded as apps for mobile devices. Each claims to provide significant insight into post-secondary institutions and the quality of education they offer. But do they really?
Brad Tucker, Concordia’s director of Institutional Planning, has looked at the issue closely. “There is no generally agreed-upon measure of institutional quality, nationally or internationally,” Tucker explains.
Concordia does very well in some rankings and not so well in others. Why is that?
Each is based on a different system of measurement. Some use publicly available or institutionally provided empirical statistics, such as research funding, student awards, graduate employment rates and library holdings. Some use surveys of educational stakeholders, including prospective students and parents. Some use a combination of the above.
In other cases, the methodology is unclear and there is little way of determining how results were compiled.
David Naylor, president of the usually highly ranked University of Toronto, has written of the difficulty with rankings: they do not fully account for the differences in mission and disciplinary mix that make each institution unique. Concordia President and Vice-Chancellor Judith Woodsworth agrees. “We want Concordia to be recognized for what it is—a responsive, dynamic institution with close ties to our community. Our innovative teaching, research and program offerings, our commitment to society and the opportunities we offer our students contribute to the university’s distinct place in Montreal, Quebec and Canada,” Woodsworth says. “This is what some rankings fail to capture.”
Part of Concordia’s distinctive approach and long-standing mission are its admission policies and flexible scheduling that enable people who might not initially look qualified on paper, or who may have other barriers, to have access to post-secondary education. Concordia also allows part-time study in all its programs. “We are committed to providing all students with the means to thrive and realize their full potential,” adds Woodsworth.
And students do flourish at Concordia. The university’s retention and graduation rates are as good as—or better—than other institutions, as are its post-graduation employment rate.
Six months after graduation, 92 percent of graduates are employed; two years after graduation, the rate rises to 96 percent.
Concordia leads all other Canadian universities in the number of graduates who hold Chief Executive Officer positions in Global Fortune 500 companies, according to a survey conducted by L’École des Mines de Paris. These measures are rarely included in standard university rankings.
With respect to student satisfaction, there is no difference between Concordia students’ evaluations of their experiences and those of students attending other comprehensive universities during their first or final years. (See the accompanying charts.)
If they had the decision to make again, Concordia students’ responses indicate they would be as likely to choose Concordia again as students elsewhere would choose their schools. Concordia holds its own in comparison to its peers across Canada and does so with considerably fewer resources. For instance, the university’s downtown Webster Library, open 24/7, is the envy of other Canadian institutions and offers students textbook-sharing and laptop-lending services.
Concordia also offers classes during the day and evening, with no distinction in student status. “All these measures enhance the student experience and ensure access,” says Woodsworth.
“Our students care deeply about these types of difficult-to-measure, value-added services and they have even volunteered funds, through their student union, to support some of these initiatives.”
Comparing apples with oranges
There are clear difficulties with measuring the quality of university education, says Tucker.
“Rankings tend to assume all students want or need the same thing in all places, and that there is one clear definition of university quality,” he says.
Tucker cites the example of the consistently poor showing of Concordia and most Quebec universities in Canada’s best-known ranking, the Maclean’s Guide to Canadian Universities.
At the outset, the ranking is heavily skewed toward universities with medical and health science schools. For instance, a variety of prizes and awards in scientific fields are recognized but Governor General Awards, which some of Concordia’s stellar Fine Arts faculty members have earned, are not. The magazine also assesses quality based on 13 factors directly related to funding levels. Since Quebec universities are considered to be underfunded in relation to those in other provinces—to the tune of an estimated $500 million annually for the 2007-08 academic year—this means that most Quebec universities are at a disadvantage when it comes to the Maclean’s rankings.
McGill University is the one exception. But the latest figures (from 2008) show McGill’s endowment to be 10 times the size of Concordia’s and seven times that of the Université de Montréal.
“The bigger the endowment, the bigger the potential to invest more in an institution, its research and its people,” says Tucker. “Because this type of funding is masked in rankings, Maclean’s and other surveys are essentially comparing apples with oranges.”
Despite these drawbacks, Concordia does do well in several Maclean’s indicators. Maclean’s places Concordia among 11 comprehensive universities, which the magazine defines as universities with a “significant” amount of research and wide range of undergraduate and graduate programs. Concordia is ranked in the top six in three categories: Social Sciences and Humanities Research Council Grants (third); Library Holdings per FTE (full-time equivalent) Student (sixth); and Reputational Survey (sixth). These account for 33 percent of the total score. Maclean’s placed Concordia eighth in reputation in 2006 and sixth since 2008.
How valid are the rankings? “It depends,” Tucker responds.
Some of the indicators reported on are measures of institutional quality that universities track themselves. The Maclean’s ranking includes some indicators that are generally accepted drivers of institutional quality. Several of these, such as student and faculty awards, research funding and student support, are part of Concordia’s own core indicator set.
However, rankings are also based on publicly available information that can be two to three years out of date. Some universities may already be taking action on indicators that are not at desired levels and bringing about changes in policy and programs to reflect institutional priorities but it takes time to achieve improved rankings. “At Concordia, we’ve made a significant commitment to supporting professors who apply for external funding, to fundraising for student awards and to providing incentives to attract high-calibre students,” says Tucker. These efforts are already bearing fruit, he adds. “For instance, we had a record increase in PhD enrolment last year.”
Woodsworth expands on the point. “Guided by our strategic framework, we are investing in student financial aid, particularly for graduate students, along with a wide range of student services and support for our libraries,” she says. “We have made great strides in the area of research and in the quality of our programs and we make an increasingly important contribution to local and regional economic, social and cultural development. We remain focused on moving forward as an institution, continually striving to better ourselves.”
Tucker suggests it would be possible to develop a more equitable tool for comparing institution but it would be challenging to achieve.
So, where does that leave Concordia?
“Committed to our mission, dedicated to our students, engaged in our community. This is where we excel,” says Woodsworth.
“It is gratifying to hear from our students, faculty, staff and alumni just how much they value their experience at Concordia. I know they will spread the message across Canada and around the world, and I am confident that the word will get out.”