The perspective of research funding agencies

Research funding agencies increasingly want to see evidence of the broader impact of science in grant proposals.

Over the past two decades there has been an evolution among major research funding agencies worldwide in the merit review criteria used to evaluate research proposals to allocated funds. While all maintain a strong focus on the quality of the science proposed in advancing knowledge within a given field, there is now a much greater emphasis on the potential benefits the research will have on society. For example, since 1997, the U.S. National Science Foundation (NSF) has employed two overarching criteria in the evaluation of proposals: 'intellectual merit' and 'broader impact'. The European Union's largest Research and Innovation programme, known as Horizon 2020, now employs three overarching criteria: 'excellence', 'impact' and 'quality and efficiency of the implementation'. Other agencies refer to this as 'knowledge transfer'.

While the definition of these impact criteria and their importance varies among funding agencies, programmes and even individual calls for proposals, the increasing demand for innovation and effectiveness in ensuring scientific discovery has the potential to benefit society is clear. Achieving broader impact from scientific discovery or technological innovation, whether it be basic science or a marketable technology, depends on how involved people beyond the research team become in what is proposed. These people are stakeholders, and this wiki demonstrates proven tools exist to effectively engage them in order to achieve desired impacts. In the past, proposals that emphasized concepts like 'awareness raising', 'educational outreach' and 'science communication' without a complementary and fully integrated strategy for participation and engagement generally could not demonstrate the kinds of changes that constitute broader impact. ** The concepts and tools in this wiki are central to overcoming this shortcoming.**

One way to demonstrate the growing relevance of broader impact and knowledge transfer criteria on grant proposals is to provide specific examples from research agencies around the world. This list is not comprehensive, but it illustrates how important participatory research and stakeholder engagement are for Earth and environmental scientists as they work to meet these criteria in their proposals. (This list is illustrative only and is not intended to provide information about any specific call.)


U.S. National Science Foundation
All NSF proposals are evaluated through use of two National Science Board approved merit review criteria.

1. Intellectual Merit: The Intellectual Merit criterion encompasses the potential to advance knowledge; and

2. Broader Impacts: The Broader Impacts criterion encompasses the potential to benefit society and contribute to the achievement of specific, desired societal outcomes.

According to NSF's Grant Proposal Guide (Section II.C.2.d), broader impacts may be accomplished through:

* the research itself,
* activities directly related to specific research projects, or
* activities supported by, but complementary to the project.

The Guide goes on to explain that the broader impacts criterion promotes societally relevant outcomes beyond scientific knowledge.


European Commission Horizon 2020

In the Work Programme 2016-2017, for all Research and innovation actions (RIA); Innovation actions (IA); SME instrument actions…Experts will evaluate on the basis of the criteria 'excellence', 'impact' and 'quality and efficiency of the implementation'.

Impacts are defined as:

Any substantial impacts…that would enhance innovation capacity, create new market opportunities, strengthen competitiveness and growth of companies, address issues related to climate change or the environment, or bring other important benefits for society.

Scoring and Weighting

  • For most proposals, each of the three criteria will be equally weighted. (For Innovation actions and the SME instrument, the score for the criterion 'impact' will be given a weight of 1.5.)
  • For the evaluation of first-stage proposals under a two-stage submission procedure, only the criteria 'excellence' and 'impact' will be evaluated.

Science with and for Society

This European Commission programme is instrumental in addressing the European societal challenges tackled by Horizon 2020, building capacities and developing innovative ways of connecting science to society. It aims to make science more attractive (notably to young people), increase society's appetite for innovation, and open up further research and innovation activities. It is promotive of Responsible Research and Innovation (RRI), and approach which allows all societal actors (researchers, citizens, policy makers, business, third sector organisations etc.) to work together during the whole research and innovation process in order to better align both the process and its outcomes with the values, needs and expectations of European society.


Research Councils of the United Kingdom (RCUK)
The following principles define, in broad terms, the approach and expectations of the Research Councils:

* Excellent research with impact is central to Research Council activities,
* The onus rests with research applicants to consider and demonstrate how they would achieve excellence with impact ,
* Research Council guidance and assessment procedures aim to:
* maximise both excellence and impact , and
* ensure user perspectives are strongly represented.

The RCUK defines research impact as 'the demonstrable contribution that excellent research makes to society and the economy'.

Economic and societal impacts embraces all the extremely diverse ways in which research-related knowledge and skills benefit individuals, organisations and nations by:

* Fostering global economic performance, and specifically the economic competitiveness of the United Kingdom
* Increasing the effectiveness of public services and policy
* Enhancing quality of life, health and creative output.


L'Agence Nationale de la Recherché (ANR) – France

The Work Programme 2016 describes the ANR funding instruments and both societal and knowledge challenges to be addressed by research. The Generic Call for Proposals 2016 lists the proposal evaluation criteria (see page 10-11), which include 'Quality and originality of research proposed', 'Project organisation and resources implemented', 'Overall impact of the project'.

The overall impact of the project includes

* The societal impact in terms of the project's capacity to address issues related to the challenge and the instrument
* Scientific impact and strategy for disseminating results
* Economic impact and valorisation strategy


Deutsche Forschungsgemeinschaft (DFG) – Germany

The DFG advises applicants to address six principles of in the development of a proposal. They include 'Good Scientific Practice', 'Promoting Equal Opportunity', 'Diversity in the Research System', 'Legal Aspects', 'Programmes and Modules', and Knowledge Transfer.

The term 'knowledge transfer' refers to a transfer of knowledge between research and industry or the general public.

The idea: From technology transfer to knowledge transfer

As a major research funding organisation, the DFG has the unique potential to facilitate the application of a wide range of knowledge generated through basic research. This knowledge could be very useful to industry and society. Knowledge transfer is therefore an important activity for the DFG, and the organisation funds the transfer of knowledge between research and practice across all disciplines.

The goal: Innovation and mutual benefit

The DFG hopes that transfer funding will have a double effect, encouraging more economic and social innovation while enabling collaborative projects to stimulate new research questions and therefore further basic research.


Research for Sustainable Development – FONA-3 – Germany

Framework program of the Federal Ministry of Education and Research (BMBF)

FONA-3 focuses the majority of the funded sustainability research on implementing research findings in specific technical and social innovations. The goal is to maximize the impact of sustainability research on society and for research and innovation funding to be more closely linked in the future. This transcends the boundaries of conventional knowledge transfer methods. A central aspect of this approach is encouraging companies to adopt new research findings and apply them as quickly as possible for innovative products, techniques and services.

Research is now being carried out through broader engagement with partners from the fields of politics, administration, industry and civil society, as well as across disciplinary boundaries…This framework programme…is the product of extensive dialogue with science, industry, civil society, local communities and politics…to help researchers find answers, ensure that their solutions are adapted for transfer

FONA-3 continues to set great store in… collaborative research : the next step is to integrate participants from civil society to a larger extent in the research projects…


Ministry of Environment, Energy & Climate Change - Greece
Πληροφόρηση και Επικοινωνία με στόχο τη διάχυση της πληροφόρησης και την αύξηση της ευαισθητοποίησης σε επιστημονικα / περιβαλλοντικά θέματα
(Information and Communication actions aimed at disseminating information and raising awareness in scientific / environmental issues)
Ενδυνάμωση της προσαρμοστικής ικανότητας: Έρευνα, εκπαίδευση και ευαισθητοποίηση
(Strengthening the adaptive capacity: Research, education and awareness – see section 5.6, page 84)

The specificity and relevance of climatic phenomena requires building the capacity of institutions and informed citizens and able to plan their future several decades. Our existing capacity to respond to extreme climatic conditions and the gradual, irreversible changes that are expected to bring to our way of life is limited.

The success of this program lies in strengthening the adaptive capacity so as to avoid as much as possible the consequences to society and the economy of our country. The enhancement of adaptive capacity is a multidimensional concept and includes developing ways to strengthen the adaptive capacity of professional teams, state bodies and interest groups such as:

* Early warning systems
* Information dissemination and accessibility of climate data
* Connection between the scientific community and stakeholders including partnerships with the private sector
* Training and lifelong learning


Australian Research Council (ARC)

Research Impact Principles and Framework

There is an increasing focus on showcasing or measuring the societal benefits from research , and a need for better coordination in reporting and promoting the impact of these research outcomes. This will become increasingly important in a tight fiscal government environment where returns on investment in research will need to be demonstrated in terms of environmental, economic and social impact. For these reasons and others, key stakeholders including government, industry and the community require more information on the benefits derived from investment in Australian research activities.

The Definition of Research Impact: Research impact is the demonstrable contribution that research makes to the economy, society, culture, national security, public policy or services, health, the environment, or quality of life, beyond contributions to academia.

In developing your 'Impact Statement' in an ARC grant application form you should consider examples (or illustrations) from your research plan representing all columns of the Research Impact Pathway Table. Please ensure your chosen examples that demonstrate both the expected Outcomes and Benefits over time are included in the Statement.


Environmental Research Grants – New South Wales (NSW), Australia

The aim of the Environmental Research program is to support applied research projects that help address environmental problems in NSW.

The Environmental Research Program Guidelines for 2016 include four assessment criteria for stage 2 proposals: 'Significance for the NSW environment';, 'Approach and methodology', 'Communication and dissemination' , and 'Lead applicant and collaborators' (see page 12).

For 'Communication and dissemination', three of five key questions to be addressed are focused on impact and stakeholder engagement:

* Will the communication strategy enable meaningful application/implementation of research findings?
* Will dissemination activities build capacity of groups/organisations?
* Will implementation result in tangible environmental benefits for the environment of NSW?

For 'Lead applicant and collaborators' two of four key questions to be addressed are focused on impact and stakeholder engagement:

* Is there meaningful engagement with the end-user(s)?
* Are the people delivering in the communication strategy appropriately qualified and experienced?


Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) – Argentina

Argentina's National Council of Scientific and Technical Research (CONICET) promotes between their members (researchers of universities and research centers) the importance of transfer of research results to society. The evaluation criteria to enter Research Career at universities or CONICET is based mainly on academic production (number of articles, book chapters, books and conference presentations published), and the formation of human resources (direction of work and graduate thesis). But in recent times transfer activities to society have been valued. These actions generally involve contact with actors outside the academic field, and adaptation of the knowledge produced unprofitable commercial or social use.

Indeed, the 2015 call for access to, or promotions for members pursuing a career in research specifies that:

"Article 54 •. The resolution of the Advisory Committees and Promotion Board and Qualification shall include, as appropriate to each class of the Career of Scientific Research and Technological, all the elements to assess the academic quality of scientific production, the theoretical and methodological consistency of the research plan, the assessment of human resources trained by the research, the impact and /or transfer to environment , management activities in the scientific and technical area, as well as other elements of an academic nature to endorse the recommendation and / or the order of merit".


Gobierno de la Provincia de Córdoba – Argentina

Programa Transferencia de Conocimientos 2015

This program seeks to establish a closer link between society, science, technology and innovation, guiding the dissemination of knowledge and practices towards technical and social needs of our region.

The Call involves three modalities with the purpose of:

* Identify the results, experiences or transferable knowledge generated by research groups
* Support events of science and technology
* Reward those research teams with strong links with the regional socio-economic environment.


Secretaría de Estado de Investigación, Desarrollo e Innovación de España – Spain
Formación e Investigación** (Training and Research)
Estrategia Española de Ciencia y Tecnología y de Innovación 2013-2020**
(Spanish Strategy of Science, Technology and Innovation 2013-2020)

'Knowledge transfer and management' is one of the six priorities of the Spanish Strategy for Science, Technology and Innovation…In this context, an important part of the activities arising from the objectives of the strategy are aimed at promoting dialogue and communication between agents in the system , so that confidence is generated between themselves by creating tools to improve the quality and legal safety. In this way strategic management of intellectual rights, knowledge transfer , commercialization of research results property is assured and the basis for the implementation of open innovation models are established. The knowledge transfer and management will include four key dimensions: inter-sector-transfer, inter-territorial transfer, international transfer and transfer between the different system agents.

According to the modification in 2015 of the directive regulating the procedures aimed at obtaining the necessary accreditation to access to professor position at Spanish university, experience in knowledge transfer is an evaluation criterion ( BOE 17/06/2015 Anexo II, Sec. I. Pág. 50334 )

Transfer of knowledge and professional experience

* Patents and products with intellectual property registration, knowledge transfer to the productive sector, involvement in technology-based companies, etc.
* Quality and dedication in professional activities in companies, institutions, public research organizations or hospitals other than teaching or research activities.
* Contracts for the transfer or provision of professional services to companies, public administrations and other institutions.
* Other merits of transferring knowledge and professional experience.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License