On December 10th, the Horizon 2020 programme will be launched in Ireland, and Europe-wide the following day. This European programme will make more than €70 billion available for research and innovation between 2014 and 2020. In Europe, the only country that has been increasing its science budget is Germany. So Horizon 2020 will be highly competitive and researchers will need to write outstanding proposals.
Proposals are evaluated through international peer review. Lobbying and other means will have no influence on the outcome. So it is critical for researchers to be well prepared and to understand in great detail the evaluation process.
One of the aims in developing Horizon 2020 was for simplification. This resulted from the common criticism by researchers that European funding is highly bureaucratic and heavy on administration. The response of the European Commission was to introduce the concept of simplification where every aspect of the programme is looked at from the perspective of the applicants and contractors. Every effort is then made to simplify the process. This has led to a single set of evaluation criteria for proposals regardless of whether they are for a fellowship in the humanities or a large multinational project in nanotechnology.
Proposals submitted are evaluated on the basis of three criteria: excellence; impact; and the quality and efficiency of the implementation. Excellence refers to the quality of project, the people and the partner organisations. Impact depends on the type of proposal being funded. For example, in the case of a Marie Curie individual fellowship, impact is measured in relation to how the fellowship enables the researcher to develop his or her career prospects. A clear implementation plan is necessary to cover consortium decisions, management structures, participant description, resources and budgets.
Wide-ranging evaluation
The evaluation of proposals is carried out by mail reviews and panel meetings. After submission, a proposal is sent to at least three experts in the area, who independently provide written reviews according to a fixed template. The Commission staff collate these reviews, and a panel meeting of evaluators is convened to discuss the proposals. Each panel member is assigned the task of leading the evaluation and discussion of a number of proposals, taking into account the online reviews. There are checks and balances to deal with instances where reviewers may have significantly different opinions and further expert opinion can be requested.
Finally, a ranked list of proposals is drawn up. The number put forward for funding then depends on budget available. Proposals that lie just below the funding threshold are placed on a reserve list. This is because funds can become available if a funded proposal does not go ahead or the final negotiated amount is less than the requested amount. Applicants receive an evaluation summary report that provides the scores and the reviewers’ comments.
In order to carry out evaluations, the Commission needs a large pool of researchers who are willing to give their time to review proposals. More than 500 researchers in Ireland participated as evaluators or experts over the period of the Seventh Framework Programme 2007-2013, out of a total of more than 40,000 worldwide. Most were from universities, but there were also experts from institutes of technology, research centres, funding agencies and industry. The list of evaluators is published annually in alphabetical order, and gives no details that could associate a reviewer with a specific proposal. In that sense the peer-review process is anonymous but also very transparent.
Anyone can sign up to be an evaluator on the Commission’s database and this means it can be difficult for Commission staff to easily identify high-quality evaluators. One way is for national delegates to recommend lists of researchers to act as evaluators, providing a level of guarantee as to the quality of the people.
Researchers planning to participate in Horizon 2020 should put their details on the Commission’s expert database and contact the relevant national delegate. This will allow them to gain a deeper understanding of the evaluation and selection process, and to meet potential future collaborators. It will be invaluable for younger researchers to gain experience, meet peers and get an overview of new directions in their disciplines. Ultimately it can improve their chances of success in Horizon 2020.
Conor O’Carroll is research director in the Irish Universities’ Association, iua.ie
