Twostage calls –
A real improvement in times of low success rates?
The relatively low success rates for applicants in most parts of Horizon 2020 have been heavily and unanimously criticized by the stakeholders. In response, the European Commission introduced more generally a proposal evaluation in two stages, in order to ease the burden of unsuccessful applicants during the first stage. This approach received a very positive feedback from the scientific communities.
This paper presents a simple model to analyse the likely consequences of such a change and shows that, despite the general enthusiasm about the new approach, the net effects are rather nuanced. To make this approach a success, a very careful implementation will be required.
0. Intro
Success rates, defined as the ratio between proposals submitted and proposals funded, have dropped quite substantially across Horizon 2020: While success rates in FP7 were around 20%, the figure for Horizon 2020 is so far around 15%.
Since apparently most other features of Horizon 2020 run reasonably well, the issue of success rates has become the focal point of public criticism. Lobby groups and policy makers complain about the high amount of effort (and notably money...) that is finally wasted in the 85% of unsuccessful proposals. While the most popular “solution” to this problem – an increase in the number of funded proposals – remains wishful thinking in times of tight budgets, the European Commission has reacted to the growing unease by announcing the more general use of twostage calls.
This approach is generally welcomed by the scientific community, as it promises to cut the effort required by the applicants considerably through much simpler and easier proposals at the first stage.
The following analysis based on a simple model will show that things are a bit more complicated as was reflected so far in the public debate – and that a careful implementation is needed to safeguard the expected efficiency gains and to avoid unwarranted distributional effects.
1. A simple model
The rather trivial baseline scenario for a singlestage call reads
TE = AE * NA
TE Total Effort
AE Average Effort per application
NA Number of applications
The “total effort” in terms of time, money and other resources for all applications in a given call is determined by the number of applications multiplied with the average effort for each application. Throughout this paper the concept of “effort” is used, as the more tangible concept of “costs” might hint at concrete figures, which this paper is unable to present.
For the singlestage call, the starting assumption is that there were 100 applications, requiring on average an effort of 10, so that the total effort required for all applications under this call is 1000.
Box 1: Baseline scenario for a single stage evaluation
Box 1 illustrates the scenario for a 15% success rate. The total coloured surface in the box indicates the overall effort invested by all applicants.
Things get slightly more complex for the twostages call, where the formula used reads as follows:
TE = (AE1*NA1) + (AE2*NF*SR)
TE Total Effort
AE1 Average Effort per first round application
NA1 Number of first round applications
AE2 Average Effort for a second round application
NF Number of proposals to be funded
AR Admission Ratio (Applications admitted to the second round in relation to the number of proposals to be funded)
The total effort for such a call is calculated as the sum of efforts required for the first round and for the second round:
The baseline scenario for the twostages call is using the following assumptions:
· AE1 = 2
The average effort for a proposal outline is estimated at 20% of the effort required for a full proposal.
· NA1 = 100
The number of applications in the first round is unchanged compared to a singlestage call
· AE2 = 10
The average effort for preparing a full proposal at the second stage is equal to the one for preparing a proposal in a singlestage call
· NF = 15
This assumption is in line with current success rates in Horizon 2020.
· AR = 3
The number of proposals admitted to the second stage should be three times higher than the number of projects likely to be funded.
Table 1 in Annex provides some (fictive) calculations for illustration purposes. The figures used are mere assumptions and not based on any empirical evidence. The aim here is not to come up with any kind of measurement, but just to provide some ideas about the impact of changes in the different variables.
Box 2 illustrates the data from the corresponding column in Table 1, showing that the Total Effort for a twostage call under the assumptions made above would be 650, so roughly one third less than for a single stage call. This would represent a real progress in terms of saving monetary and intellectual resources and justify a clear recommendation for the implementation of twostage calls.
Box 2: Baseline scenario for a twostages evaluation
In Box 2 and all following boxes the rectangular “black box” represents the total effort required in a singlestage call. The coloured surface represents the efforts required by participants in a twostage call. For as long as the “white” surface inside the box is greater than coloured surfaces outside that box, the effort required in a twostage calls is lower.
But as always in mathematics – and in real life ... , everything depends on the assumptions made. The following four chapters present for each of the variables in the model a detailed analysis on plausible variations and likely consequences.
2. Three pages, five pages, twenty pages? –
Variations in the average effort for a first round application
The main reason for the attractiveness of the twostage calls is the massive reduction in the effort needed to prepare a proposal outline in the first round as compared to the efforts required for a fully developed proposal in a singlestage call.
Obviously the savings are most significant if first round applications consist of short texts of several pages only. On the other hand, the need to carry out a substantial and well grounded proposal evaluation puts a natural limit to ideas for radical simplification, as it seems not feasible to base such an evaluation process on very short abstracts.
If no stringent boundaries are implemented, the tendency among applicants might be to come up with more substantial documents at the first round, just to underline the seriousness of the application and to increase their chances with the evaluation panel. Similarly, if the Commission comes up with a too detailed catalogue of issues to be addressed by applicants in the first round already, the average effort required is also likely to increase.
Box 3: Higher effort required for first stage proposals
Box 3 (based on the corresponding column in Table1) illustrates the impact of such a tendency: If the average effort for a first round application is not 20% of a full proposal, but rather 40%, the net savings in terms of efforts required for a twostages call would be reduced by more than half, and the net effect would be in the order of 15% only, which can no longer be regarded as a decisive progress.
3. Elasticity of demand? – Variations in the number of first round applications
The massive or at least considerable reduction of the average effort required at the first stage means that applicants have to pay a much lower price for the participation in a call. Contrary to the assumption made in the baseline scenario, where the number of applications remains stable, economic theory would suggest that with a reduced price one would observe an increase in demand. The degree of induced change is described by the concept of “elasticity of demand”, which varies on classical markets and might also vary across different parts of the Framework Programme.
One could expect that in areas with a limited number of potential applicants, such as probably Fusion research or aeronautics, a massive cut in the application costs will have no dramatic effect on the number of applications, as there are high barriers to entry for new candidates and almost all members of the relevant scientific communities are already involved. But there are also major fields in Horizon 2020, such as Social Sciences and Humanities, but probably also Life Sciences, where a supposed reduction in the effort required by the factor five will mobilise additional applicants from a large pool of experts so far not involved.
Box 4: Increased number of proposals in the first round
Box 4 (based on the corresponding column in Table 1) illustrates the impact of this elasticity of demand: If the number of applications in the first round is increasing by just 50% (after a “price cut” by the factor five...), the net savings in terms of efforts required for a twostage call would be reduced, so that the remaining net effect would be in the order of 25% only.
4. Nice evaluators? – Variations in the admission ratio for the second round
There seems to be some general consensus that for a solid evaluation at the second stage one should admit three times as many applications as projects could be funded. This is regarded as a good compromise between the need to have a broad range of good applications to select the truly outstanding ones, and the wish to limit the burden linked to a second round proposal for a too great number of applicants.
In the baseline scenario, this means that for 15 proposals to be funded 45 out of the 100 proposals submitted at the first stage should be admitted to the second round. This is a situation where first stage evaluators operate so to speak “in the comfort zone”, as they could give a green light for the second round to 45% of the proposals.
Things change, however, quite drastically if the number of proposals submitted increases, as argued in the previous chapter. With 150 proposals at the first stage, evaluators were forced to work with a success rate of just 30% for the admission to the second stage. Many evaluators will feel uncomfortable with this situation, and there will be a kind of natural tendency to lower the pressure by increasing the number of proposals admitted to the final round. Unfortunately though, “nice” evaluators will result in an increase in the overall effort required for the call.
Box 5: Increased number of proposal admitted to the second stage
Box 5 (based on the corresponding column in Table 1) illustrates the impact of “nice” evaluators as a consequence of a high elasticity of demand: If the number of applications in the first round is increasing by 50%, and evaluators in the first round give a green light for 60 proposals (instead of 45 in the baseline scenario), the net effect of the twostages evaluation would shrink to just 10% of the initial total effort.
5. Quit or double? – Variations in the average effort for a second round application
The final variable to be analysed is the average effort for a proposal at the second stage. In the baseline scenario it is assumed that this is equal to the effort for a proposal in a single stage procedure.
Here again, basic economic theory suggests that applicants might again change their behaviour, as the willingness to invest in a proposal might be substantially higher, if the chance of success is no longer 15% (as in single stage calls), but rather 33% (as in the final round of a twostages evaluation).
Box 6: Increased efforts at the final stage
Box 6 (based on the corresponding column in Table 1) illustrates the impact of such increased efforts at the second stage: Just by an increase of 20% in the average effort for a proposal in the second round, the overall net effect of introducing a two stages evaluation would be reduced by more than 25%.
6. Addingup of unwarranted effects
Whereas in the previous chapters for didactic purposes changes in behaviour were analysed in isolation for each of the variables, a more realistic scenario is the one where the different effects do actually cumulate.
It is surprising to see, how relatively minor changes at the different parameters do add up in the end to put the overall efficiency at risk.
Box 7: Addingup of unwarranted effects
Box 7 (based on the corresponding column in Table 1) illustrates such a scenario where the different effects addup:
All these relatively small changes taken together lead at the end to a situation where the total effort required by a twostage call is almost identical to the efforts required for a single stage call.
7. Good for the bad ones, bad for the good ones? – Distribution of efforts among participants
The simple model presented here does not only allow to analyse possible effects on the total effort required by all applicants, it permits also to analyse the distribution of these efforts across different groups of participants.
Not surprisingly, participants who are not successful in the first round are substantially better off than in a single stage call. On the other hand, all applicants admitted to the second round will need to invest more efforts.
In the numerical examples presented above, the “savings” for participants not successful in the first round (in red colour in the different boxes above) are in the order of 70 to 80%, whereas the relative “loss” for the participants in the second stage ranges from 20 to 40% (always compared to the scenario in a single call).
The result of this analysis is somewhat at odds with the political priority to stimulate excellence: Excellent and good consortia will have to invest (maybe even considerably) more, whereas the lower basic effort required might attract even more applicants which are “far from excellence”.
8. Less effort, more happiness?
The examples presented above illustrated that there is after all a fair chance to cut the overall effort required from call participants through twostage calls.
It is a somewhat different question, whether this would also lead to higher degree of “satisfaction” among the participants. Unfortunately, the picture is not as a clear and positive as one would have hoped for:
9. Some simple recommendations
The purpose of this paper is not to argue against twostage calls. The analysis carried out confirms that there are good chances to reach substantial cuts in the overall effort required.
The analysis also highlighted, however, that the success of this implementation will depend on a number of crucial factors, which would need a very careful exante analysis and close monitoring.
The following recommendations seem to be of particular relevance:
Annex:
Table 1: Data used for analyzing the effects of different variables


Box 2 
Box 3 
Box 4 
Box 5 
Box 6 
Box 7 








Number of applications submitted first round 
NA1 
100 
100 
150 
150 
100 
130 
Average effort per application first round 
AE1 
2 
4 
2 
2 
2 
3 
Admission ratio for second round 
AR 
3 
3 
3 
4 
3 
3,5 
Average effort per application second round 
AE2 
10 
10 
10 
10 
12 
11 
Number of projects to be funded 
NF 
15 
15 
15 
15 
15 
15 








Total effort 
TE 
650 
850 
750 
900 
740 
967,5 
















Average effort not successful first round 

2 
4 
2 
2 
2 
3 
Number of applications not successful first round 

55 
55 
105 
90 
55 
78 
Average effort not successful second 

12 
14 
12 
12 
14 
14 
Number of applications not successful second round 

30 
30 
30 
45 
30 
37 
Average effort successful 

12 
14 
12 
12 
14 
14 
Number of successful applications 

15 
15 
15 
15 
15 
15 
Number of applications not successful (total) 

85 
85 
135 
135 
85 
115 
Version 1.0  08.11.2016  Thanks for your feedback
Unbedingt erforderliche Cookies ermöglichen grundlegende Funktionen und sind für die einwandfreie Funktion der Website erforderlich. Daher kann man sie nicht deaktivieren. Diese Art von Cookies wird ausschließlich von dem Betreiber der Website verwendet (FirstPartyCookie) und sämtliche Informationen, die in den Cookies gespeichert sind, werden nur an diese Website gesendet.
Jimdocartv1 Erforderlicher lokaler Speicher, der Informationen über den Warenkorb des WebsiteBesuchers speichert, um Einkäufe über diesen OnlineShop zu ermöglichen. Anbieter: Jimdo GmbH, Stresemannstraße 375, 22761 Hamburg Deutschland. Cookie/Lokaler Speichername: Jimdocartv1. Laufzeit: Der lokale Speicher läuft nicht ab. CookieRichtlinie: https://www.jimdo.com/de/info/cookies/policy/ Datenschutzerklärung: https://www.jimdo.com/de/info/datenschutzerklaerung/
shd Cookie Dieses Cookie identifiziert wiederkehrende Besucher dieser Website und verwaltet den Status des Benutzers bei allen Seitenanfragen. Anbieter: Jimdo GmbH, Stresemannstrasse 375, 22761 Hamburg, Deutschland. CookieName: shd CookieLaufzeit: 1 Jahr. CookieRichtlinie: https://www.jimdo.com/de/info/cookies/policy/ Datenschutzerklärung: https://www.jimdo.com/de/info/datenschutzerklaerung/
Cookielaw Dieses Cookie zeigt das CookieBanner an und speichert die CookieEinstellungen des Besuchers. Anbieter: Jimdo GmbH, Stresemannstrasse 375, 22761 Hamburg, Deutschland. CookieName: ckies_cookielaw CookieLaufzeit: 1 Jahr CookieRichtlinie: https://www.jimdo.com/de/info/cookies/policy/ Datenschutzerklärung: https://www.jimdo.com/de/info/datenschutzerklaerung/
PHPSESSIONID Lokaler Speicher, der für das korrekte Funktionieren dieses Shops zwingend erforderlich ist. Anbieter: Jimdo GmbH, Stresemannstraße 375, 22761 Hamburg Deutschland Cookie /Lokaler Speichername: PHPSESSIONID Laufzeit: 1 Sitzung CookieRichtlinie: https://www.jimdo.com/de/info/cookies/policy/ Datenschutzerklärung: https://www.jimdo.com/de/info/datenschutzerklaerung/
Stripe Dies ist zwingend erforderlich, um den von Stripe betriebenen Zahlungsverkehr über diesen Shop zu ermöglichen. Anbieter: Stripe Inc., 185 Berry Street, Suite 550, San Francisco, CA 94107, USA. CookieNamen und Laufzeiten: __stripe_sid(Laufzeit: 30 Min), __stripe_mid (Laufzeit: 1 Jahr), Ckies_stripe (Laufzeit: 1 Jahr). CookieRichtlinie: https://stripe.com/cookiespolicy/legal Datenschutzerklärung: https://stripe.com/privacy
Postfinance Dies ist zwingend erforderlich, um den von Postfinance betriebenen Zahlungsverkehr über diesen Shop zu ermöglichen. Anbieter: PostFinance AG, Kontaktcenter, Mingerstrasse 20, 3030 Bern, Schweiz. CookieName: ckies_postfinance. CookieLaufzeit: 1 Jahr. CookieRichtlinie: https://www.postfinance.ch/de/detail/datenschutz.html Datenschutzerklärung: https://www.postfinance.ch/de/detail/datenschutz.html
Jimdo Control Cookies SteuerungsCookies zur Aktivierung der vom WebsiteBesucher ausgewählten Dienste/Cookies und zur Speicherung der entsprechenden CookieEinstellungen. Anbieter: Jimdo GmbH, Stresemannstraße 375, 22761 Hamburg, Deutschland. CookieNamen: ckies_*, ckies_postfinance, ckies_stripe, ckies_powr, ckies_google, ckies_cookielaw, ckies_ga, ckies_jimdo_analytics, ckies_fb_analytics, ckies_fr CookieLaufzeit: 1 Jahr CookieRichtlinie: https://www.jimdo.com/de/info/cookies/policy/ Datenschutzerklärung: https://www.jimdo.com/de/info/datenschutzerklaerung/
Funktionelle Cookies ermöglichen dieser Website, bestimmte Funktionen zur Verfügung zu stellen und Informationen zu speichern, die vom Nutzer eingegeben wurden – beispielsweise bereits registrierte Namen oder die Sprachauswahl. Damit werden verbesserte und personalisierte Funktionen gewährleistet.
POWr.io Cookies Diese Cookies registrieren anonyme, statistische Daten über das Verhalten des Besuchers dieser Website und sind verantwortlich für die Gewährleistung der Funktionalität bestimmter Widgets, die auf dieser Website eingesetzt werden. Sie werden ausschließlich für interne Analysen durch den Webseitenbetreiber verwendet z. B. für den Besucherzähler. Anbieter: Powr.io, POWr HQ, 340 Pine Street, San Francisco, California 94104, USA. Cookie Namen und Laufzeiten: yahoy_unique_[unique id] (Laufzeit: Sitzung), POWR_PRODUCTION (Laufzeit: Sitzung), ahoy_visitor (Laufzeit: 2 Jahre), ahoy_visit (Laufzeit: 1 Tag), src (Laufzeit: 30 Tage) Security, _gid Persistent (Laufzeit: 1 Tag). CookieRichtlinie: https://www.powr.io/privacy Datenschutzerklärung: https://www.powr.io/privacy
Die PerformanceCookies sammeln Informationen darüber, wie diese Website genutzt wird. Der Betreiber der Website nutzt diese Cookies um die Attraktivität, den Inhalt und die Funktionalität der Website zu verbessern.
Google Analytics Diese Cookies sammeln zu Analysezwecken anonymisierte Informationen darüber, wie Nutzer diese Website verwenden. Anbieter: Google LLC, 1600 Amphitheatre Parkway, Mountain View, CA 94043, USA oder Google Ireland Limited, Gordon House, Barrow Street, Dublin 4, Irland, wenn Sie in der EU ansässig sind. CookieNamen und Laufzeiten: __utma (Laufzeit: 2 Jahre), __utmb (Laufzeit: 30 Minuten), __utmc (Laufzeit: Sitzung), __utmz (Laufzeit: 6 Monate), __utmt_b (Laufzeit: 1 Tag), __utm[unique ID] (Laufzeit: 2 Jahre), __ga (Laufzeit: 2 Jahre), __gat (Laufzeit: 1 Min), __gid (Laufzeit: 24 Stunden), __ga_disable_* (Laufzeit: 100 Jahre). CookieRichtlinie: https://policies.google.com/technologies/cookies Datenschutzerklärung: https://policies.google.com/privacy
Jimdo Statistics Die sog. Jimdo Statistikfunktion ist eine TrackingTechnologie, basierend auf Google Analytics, welche von der Jimdo GmbH betrieben wird. Anbieter: Jimdo GmbH, Stresemannstrasse 375, 22761 Hamburg, Deutschland. CookieNamen und Laufzeiten: ckies_jimdo_analytics (Laufzeit: 2 Jahre), __utma (Laufzeit: 2 Jahre), __utmb (Laufzeit: 30 Minuten), __utmc (Laufzeit: Sitzung), __utmz (Laufzeit: 6 Monate), __utmt_b (Laufzeit: 1 Tag), __utm[unique ID] (Laufzeit: 2 Jahre), __ga (Laufzeit: 2 Jahre), __gat (Laufzeit: 1 Min), __gid (Laufzeit: 24 Stunden), __ga_disable_* (Laufzeit: 100 Jahre). CookieRichtlinie: https://www.jimdo.com/de/info/cookies/policy/ Datenschutzerklärung: https://www.jimdo.com/de/info/datenschutzerklaerung/
Marketing / Third PartyCookies stammen unter anderem von externen Werbeunternehmen und werden verwendet, um Informationen über die vom Nutzer besuchten Websites zu sammeln, um z. B. zielgruppenorientierte Werbung für den Benutzer zu erstellen.
Facebook Analytics Hierbei handelt es sich um eine TrackingTechnologie, die den sogenannten "FacebookPixel" vom sozialen Netzwerk Facebook verwendet und für die Analyse der Website, die Ausrichtung der Werbung, die Messung der Werbung und Facebook Custom Audiences eingesetzt wird. Anbieter: Facebook Inc, 1 Hacker Way, Menlo Park, CA 94025, USA oder Facebook Ireland Ltd, 4 Grand Canal Square, Grand Canal Harbour, Dublin 2, Irland, wenn Sie in der EU ansässig sind. Cookie Namen: _fbp, fr Cookie Laufzeit: 90 Tage CookieRichtlinie: https://www.facebook.com/policies/cookies Datenschutzerklärung: https://www.facebook.com/policy.php
Diese Website verwendet Cookies, um dir das bestmögliche OnlineErlebnis zu geben. Lass uns bitte wissen, dass du damit einverstanden bist, indem du weiter unten auf die Option "Ja, ich stimme zu" klickst. Falls du mehr über die von uns verwendeten Cookies herausfinden und eigene CookieEinstellungen verwenden möchtest, schau dir bitte unsere CookieRichtlinie an.