Mapping nonlinear research impact pathways

Universities have always performed work that benefits the broader community. But in recent years there has been much more emphasis on demonstrating and measuring the impact that this work has outside academia, with impact assessment now embedded in many national and international research rating systems, which in turn feed into funding.

Most impact assessments use the Research Impact Pathway, which tracks inputs, activities, outputs and outcomes of an invention or initiative to assess its impact beyond scholarly contributions to an academic research field. This can be benefits to the environment, society, economy and culture. But not all research impacts are linear, and the pathway can be difficult to negotiate without a map. In an article recently published in Nature Communications, Macquarie University's Professor Kirstie Fryirs and National Research Assessments Leader Thom Dixon, together with the University of Auckland's Gary Brierley, provide a three-part approach to produce such a map. The approach can be used to capture all the complexity of long-term projects with benefits and costs that might not be clear for many years, and with impacts that may not be directly or quantitatively measurable.

The initial phases of the Research Impact Pathway—inputs, activities and outputs—are relatively straightforward and consistent across projects. Researchers, their collaborators and their institutions can plan these factors, and have some control over them, leading to an intended outcome—commercial products and licenses, job creation, new contracts, or programs, citations of work, new companies, and new joint ventures.

Impacts are not necessarily simple to identify. First, researchers have little control over whether their work is adopted and used. Second, in the four main areas outside academia: environmental, social, economic and cultural spaces, there can be multiple intended and unintended consequences of uptake. Third, impacts are intrinsically difficult to measure. And this is what Fryirs and her colleagues have addressed.

Thom Dixon says, "Reporting on research impact is here to stay. This impact mapping toolbox will help researchers identify and capture evidence in a rational way, it will help researchers explain how their work changes the world." Kirstie Fryirs adds, "We believe the approach we have developed can be readily applied within many research impact reporting and assessment systems around the world."

The River Styles Framework, which Fryirs and her colleagues work on, provides a case in point. The Framework has been adopted internationally, because of its innovative approach to geomorphic analysis of rivers. It is tailored for the landscape and institutional context of any given place to produce scaffolded, coherent and consistent datasets for catchment-specific decision making. But while assessing an outcome like adoption is simple, impacts occur on a timescale that can be measured in decades. The standard impact pathway isn't designed for that. Kirstie Fryirs says, "Using the River Styles Framework as the case study has enabled us to test the approach as well as provide a worked example that others can use for their own ex-ante or ex-post assessments and reporting."

What Fryirs and her colleagues propose for these cases is an interactive map. This three-part approach contains a context strip, an impact map, and soft impact intensity strips to capture the scope of the impact and—crucially—the conditions under which it has been realized.

Interactivity is vital to the map. It can change when factors the developers had forgotten, or thought peripheral, later re-appear as an influence on a stakeholder, community or network not originally considered as an end-user.

The map also enables users to differentiate between what the authors call hard and soft impacts, those that can be direct attributed to an invention or initiative, and those that are less easy to attribute, yet are often a dominant part of the impact mix. The inclusion of soft impacts in the assessment and mapping process provides a new and valuable tool for the kind of multi-faceted and long-term projects that provide great value but don't fit the standard, often quantitative or economic means of measuring that value.

Gary Brierley explains, "In this fast changing, increasingly crazy world, we're forever looking for easy measures of performance and impact, yet often it's the masked elements of research that really matter—those quiet, behind the scenes activities that make a difference, on-the-ground."

"It's not all about widgets, citations and grand-standing. Ways of being impactful vary massively, and this paper provides guidance on how we can start to think about such matters. It encourages more inclusive approaches to appraisal of 'soft' impact—something that takes quite different forms."

Kirstie Fryirs adds, "One of our aims in developing this new approach to research impact mapping was to provide a more inclusive means for identifying and recognizing impact, particularly for research that does not produce an invention as such, but has really important and long-lasting impact on, for example, philosophy, practice, citizen knowledge and professional development, and for us as river scientists, environmental health. We hope our method will encourage others to profile some of their marvelous, world-leading, world-changing and impactful research."