Some critical reflections on the role of Work Programmes in Horizon 2020  

Download PDF

This paper aims at analyzing the role of Work Programmes for the implementation of Horizon 2020. Given the long history and the paramount importance of this implementation tool it might seem rather strange to put their role into question. However, imagining a European Research Programme without annual Work Programmes illustrates that the merits of Work Programmes come at a surprisingly high price … and that at the end the European Research System might be better-off with no or substantially “lighter” Work Programmes. 

0. Intro

The European Commission has published a first set of data on the implementation of Horizon 2020 in 2014. These data seem to indicate a rather sharp drop in the ratio between proposals funded and proposals received. This so-called “success rate” fell from around 20% in FP7 to around 14% for the first year of Horizon 2020. 

Although a lower “success rate” for the applicants (in the sense defined above) does not mean a less successful programme, there seems to be quite some unease about this development. And although for any meaningful selection process a certain level of “oversubscription” is necessary, the term now stands for an allegedly non sustainable situation. Apparently, the European Commission plans to react for the next round of calls by further tightening the evaluation criteria and the work programme specifications. 

Against this background, it might be useful to have a closer look at the role of Work Programmes in Horizon 2020 – and to argue for a much broader reflection on how to maximize the impact of Horizon 2020.

1. About Work Programmes

Organising a research funding programme with public money means operating at the interface between two completely different worlds: On the one side the research world is – at least ideally – spontaneous and in a sense erratic, whereas on the other side public administration aims at solid long-term planning and exact implementation. 

Within the European Research Programmes, Work Programmes are exactly positioned on this borderline. The classical “Framework Programme” as such is a broad political document describing the overall space for potential applications, and the proposals submitted describe essentially the research agenda of an envisaged project. In between these two are the Work Programmes, and depending upon the circumstances these could be rather restrictive administrative documents (as is currently the case for most thematic areas) or quite open documents (as is currently the case for the European Research Council). 

Historically, Work Programmes became greatly important in European Research Programmes as the extremely small budgets in the early years required a very detailed description of those activities, for which funding would be provided. Over the years, Work Programmes became the main steering documents, as the overall Framework Programme decision making procedure is extremely cumbersome and hence limited to fixing the essentials, such as overall budget, funding modalities and broad research areas covered, for a period of (initially) four to (nowadays) seven years. All the details concerning specific research activities are laid down in the mostly annual Work Programmes, for which the formal adoption process is much lighter. 

It is somewhat amazing to observe that these fundamentals have remained unchanged from the last 25 years, and more importantly over the long way from a small targeted activity to the largest multi-national research programme world-wide.

As a result, Work Programmes have now become extremely complex, and it is an significant (albeit probably unintentional...) achievement of the Horizon 2020 Participants Portal that this complexity is no longer directly visible to potential applicants, as they will see only the (small) part of the overall documentation relevant for a given research topic. 

However, looking at it from a governance perspective, it is important to grasp the bigger picture – and in this case, it is really big:

The Work Programmes valid for (early) 2015 listed in the “Documents” part of the Participants Portal add up to some 1.800 pages – and these do not include “specific” Work Programmes for Joint Technology Initiatives and similar activities. A rough estimate for the total volume of Work Programme documents valid during the seven years of Horizon 2020 would therefore lead to a figure between 15.000 and 20.000 pages. 

At this point, it might be useful to note that these 20.000 pages are just the tip of the iceberg (visible from the distance). In practice, the preparation of these Work Programmes is a very long Commission internal process, involving several rounds of consultations with external experts, Member States, and other Commission services. The “final” version of an annual Work Programme emerges only after numerous revisions, so that in the process of preparing Horizon 2020 Work Programmes there might be well some 200.000 pages drafted. 

2. The functions of Work Programmes

Following the official documents, the main “raison d’être” for the Work Programmes is their crucial role in providing specific and strategic focus. Since the Framework Programme itself keeps the research agenda wide open, Work Programmes specify for every year a distinctive selection of topics to be addressed in the dedicated calls. 

In its extreme form, every theme mentioned in the Framework Programme is at least addressed once over seven years in an annual Work Programme. Sometimes (notably in FP7) the Work Programmes stipulate that only one project per topic is funded, so that at the end every aspect of a given research domain as specified in the Framework Programme is “covered” by exactly one project. 

Although this is obviously the extreme form of a bureaucratic planning approach, the official documents are full of a far more elaborate rhetoric, using terms like “strategic focus," “developing research agendas," “avoid duplication," “priority setting as every euro counts” and so on. 

In essence, the main function of Work Programmes is to reduce the demand for EU funding by limiting the access to the Framework Programme in a given year to a limited number of research topics. Any application for a theme other than those mentioned in the Work Programmes is not permitted. Thanks to this massive restriction, the so called “success rate," the ratio between successful proposals and the total number of proposals received, remained relatively steady in the order of 20%, before it has fallen to 14% for the first year of Horizon 2020. A stable success rate in such orders of magnitude is considered by the programme management as a key success indicator – and somewhat surprisingly this view is shared by the vast majority of the research community. 

In economic terms, Work Programmes are a tool for rationing the demand for research funds, so that this demand corresponds with the average success rate to the overall budget available. The number of proposals submitted does not represent the real demand for EU funding, but represents only the small fraction of those fitting – more or (very frequently) less – to the Work Programme. This is also a gentle reminder to all of us, and notably those dealing with programme statistics, that the data available represent the situation after this rationing process, so they are useful to analyse the programme implementation process, but they can’t tell us anything about the full picture in the European research landscape. 

3.      The drawbacks of Work Programmes

The current system of implementing Horizon 2020 through annual Work Programmes is in line with the historical heritage (and as such clearly not “a break from the past”) and they basically do their job in terms of smooth programme management.

Unfortunately though, a closer look reveals a number of important deficiencies: 

•  Enormous bureaucracy

With several thousands of pages drafted every year, the preparation of these Work Programmes is a major bureaucratic endeavour. Whether these documents provide – beyond the described rationing of demand - anything more substantial in terms of analysis and strategy seems rather doubtful. They clearly fail to provide a clear focus, so that it is impossible to see whether Horizon 2020 develops any clear funding priorities. 

• Lack of transparency

The drafting of Work Programmes takes roughly one year and involves many different actors: While the Commission keeps the pen, comments and contributions come from Advisory Groups and Programme Committees, and obviously from many other sources. The final text reflects thus the views of a very loosely defined group of people, without any clear transparency rules. For a normal researcher, it remains completely opaque why certain topics are in, and others are out. 

• Counterproductive streamlining

The drafting process is dominated by a culture of “consensus building” – and avoiding conflicts. This means that any topics “off the beaten track” have a very slim chance to survive the long drafting process. Work Programmes tend rather to reflect the established scientific mainstream, as this gets broad support both from the expert groups and the Member States. The very nature of the Work Programme drafting process might thus prevent what Horizon 2020 is aiming at – innovation. 

• Open for massive lobbying

The most popular advise given at any workshop for future applicants – organised by private consultants or National Contact Points – is not to start the preparation process with drafting a proposal, but rather at a much earlier stage with efforts to put “your” topic on the next Work Programme. A specific hint is to mobilise also all potential partners for lobbying the case in their respective national contexts. The emergence of an ever-growing number of “Brussels offices” for all kinds of organisations interested in Horizon 2020 is also very openly justified by the growing importance of being present during the entire Work Programme life cycle. This anecdotal evidence illustrates that it is generally understood and in a sense accepted, that the drafting of Work Programmes is subject to external influences. 

While Work Programmes are officially supposed to be published only once they are finally adopted, a simple search in Google shows that many of these more or less advanced drafts are circulating freely. It does not require a lot of fantasy to imagine that as a result of systematic leaking the most current versions are openly available at least for the more “connected” candidates. 

Although the described strategies are probably appropriate for individual applicants, their general use is clearly detrimental to the efficiency of Horizon 2020. In reality, the competition for the best proposal at the evaluation and selection phase is partially sidelined by a competition for the most efficient “positioning” of themes and key words in the Work Programme texts. The current practice might thus systematically favour the “haves” (established organisations familiar with the tacit rules of the game) over the “have not’s” (relative newcomers).     

• Long delays

The complexity of these processes leads to long time lags, as it might take one year to get access and persuade actors involved into the Work Programme preparation process, one year for the actual drafting, and another year for the call and the proposal selection. The total time between the start of the endeavour and the actual project start might thus add up to up to three years.

• Erratic changes in funding opportunities

Since Work Programmes are published on a yearly basis, and since in most parts of Horizon 2020 their content differs almost entirely from one year to another, there are de facto no continuous funding opportunities for the research community, but rather erratic changes in the call agendas. In the worst case, a specific topic might be open for applications at one single point in time over the total lifetime of seven years for Horizon 2020. 

A quick analysis of the activities of National Contact Points shows that they spend actually most of their time and efforts in presenting every year the “forthcoming” Work Programmes to their respective research communities – before they return the year after in presenting a completely different Work Programme content, and two years later with yet another headline story... 

4. Supporters of Work Programmes

Against the background of these serious problems, it is somewhat amazing that there is surprisingly little – or, to be blunt, virtually no – discussion about this implementing tool, although it is not difficult to imagine alternative approaches. 

A simple explanation could be a certain conservatism of the system – and the fact that Work Programmes have been there since the very beginning of Framework Programme activities. They are considered part of the Framework Programme DNA and thus not subject to further questions. 

A more subtle reason for this amazing level of public acceptance might be a phenomenon well-known in political economy, where it explains, for instance, why tax systems tend to be more complicated and unfair than they ideally should be, simply because the main actors all benefit from the status quo. 

This seems also to be true for the Horizon 2020 Work Programmes: 

·         The Commission – key activity for Programme Management

In times where notably DG Research and Innovation is about to be transformed into a “Policy DG”, the preparation of annual Work Programmes for Horizon 2020 is regarded as a core activity to illustrate the strategic importance of this DG and the Commission in general, in the sense of supporting overall Commission priorities, developing strategic research agendas and safeguarding an efficient programme implementation. 

After the launch of “New Management Modes” (basically the transfer of the project management to separate agencies), the Programme Management and most importantly the responsibilities for the preparation of the Work Programmes are now regarded as key competencies. 

·         Member States – Having a say

Once a Framework Programme is adopted, Member States have basically no influence on the implementation process, except for the adoption of Work Programmes. The in-depth discussion on draft texts for the Work Programme has become the central task of the Programme Committees. Even if it is questionable to what extent an individual country can really influence the Work Programme, Member States as a whole seem to be convinced of their importance in this process. 

·         European Parliament – Possibilities for lobbying

Although the European Parliament has formally no role in the Framework Programme implementation process, MEPs seem to be quite attracted by Work Programmes as a way to mirror their preoccupations and requests. Obviously, the long and complex drafting process allows MEPs to become proactive and opens possibilities for all kinds of informal lobbying. 

·         Big players – Keeping the game predictable

Somewhat surprisingly, the most prominent European research organisations seem to “love” the Horizon 2020 Work Programmes. Whereas these organisations campaigned during FP6 and FP7 for a substantial “simplification” of the application procedures, there is currently complete silence regarding the Work Programmes. 

A potential explanation is that the established “big players” know perfectly well how to use the preparation process of Work Programmes for their own interests – and thus assume that they are better off with the current system. 

Another possible explanation is that big research organisations tend nowadays to be less driven by “research” and more by “organisation”: As they are themselves characterised by huge bureaucracies, they might find the bureaucratic nature of Work Programmes very appealing – especially as this prevents the system from sudden surprises. 

5.   Horizon 2020 without Work Programmes?!

This critical look at the Horizon 2020 Work Programmes would remain incomplete and unfair without developing some thoughts on a possible alternative approach.

The simplest and at the same time most radical substitute scenario is to imagine Horizon 2020 without Work Programmes. This is not necessarily meant as a completely realistic implementation approach, but rather for didactical purposes, in order to highlight the contrast between the two implementation styles. 

In a Horizon 2020 without Work Programmes, proposals could be submitted at any point in time for any topic covered by the Framework Programme. This might sound very strange, but it describes actually a reality which is already in place at the ERC since over eight years... 

Abolishing the somewhat artificial reduction of the demand for Horizon funding through annual Work Programmes will lead to a massive increase in the number of applications – and thus to a sharp fell in the success rates. It is open, however, whether this would become a lasting situation, or whether learning processes within the research community would lead to a gradual reduction in the application numbers. 

The evaluation of proposals submitted under such an “open call” system will require substantially more efforts than within the present system with predetermined calls, in order to make sure that proposals in all areas have the same chances for success. This might notably lead to a system with both area-specific evaluation panels and overarching strategic panels to ensure a certain calibration of marks. The increased requirements in such an evaluation system might favour a more stringent selection of evaluators and the need for a kind of “standing evaluation committees” for a period of several months. 

These reflections were meant to illustrate that also an alternative comes at a high price, as there is obviously no miracle solution. 

There are, however, also a number of extremely important arguments in favour of such an alternative approach: 

• Radical simplification

Getting rid of annual work programmes would represent the single most important simplification in European Research Policy since its very beginning. Such a step would free an enormous amount of intellectual resources, which are currently occupied with drafting, adopting, disseminating, interpreting and understanding the 20.000 pages of Work Programme texts.

• Level playing field

Horizon 2020 without Work Programmes would put the competition for scarce resources back at the only place where it belongs - the rigorous evaluation of all research proposals for their scientific quality and their societal relevance. It would stop all the distorting practices of lobbying and pre-determining Work Programme texts, and make sure that there is no possibility for a systematic bias in the project selection.

• Expect the unexpected

Under the current system, there are almost no surprises possible, as the drafting of the Work Programme texts predetermines the nature of the proposals to be received. In a system without such restrictions, there is room for surprising and truly innovative proposals, based on the creativity of the research community and not on the consensus among the insiders drafting the Work Programmes.

• Focus on better proposals

Within the current system, an enormous amount of time and effort of the research community is absorbed by following (and ultimately influencing) the development of Work Programmes, in order to make sure that a consortium is ready to address (more or less) one of the topics listed at the point in time specified in the call. Moving away all this red tape would allow researchers to focus entirely on the preparation of their proposal, and should thus lead to a higher average quality.

• Europe supports the best ideas

Most importantly, in the current “top down” logic of Horizon 2020 researchers get support if they manage to address an issue foreseen in the Work Programme, or if they at least manage to make their own ideas fit in somehow. In a “bottom up” approach Europe could send a strong message that it supports the best research teams in realising their ideas, at the point in time they would like to launch their endeavours. 

 6.   Some final reflections

This paper does not present any revolutionary new insights – it rather puts generally known aspects in a new perspective, and it challenges the current political mainstream. 

Over recent years, FP7 and Horizon 2020 have gone extremely far in implementing a very detailed top-down approach, in taking specific topics out of the general competition into ring fenced budgets, and in using the economic crisis as an argument to interfere directly into the research agenda. 

The reflections in this paper illustrate that there are in fact alternative ways of organising European Research Programmes, and it remains an open question whether the extensive use of restrictive Work Programmes is really the best way forward.   

Version 1.0 - 07.07.2015 - Thanks for your feedback