Evaluation of Capital Market Interventions:
Lessons from recent OECD work on Seed and Early Stage Financing
Expert Group on the Evaluation of Industrial Policy
September 2013 Karen Wilson
Structural Policy Division
Science, Technology & Industry Directorate, OECD
Recent Seed & Early Stage Financing Work
• Seed and Early Stage Financing Questionnaire (2012) – Mapping of existing policies in OECD countries
• Financing instruments (supply)
• Regulatory environment
• Entrepreneurial ecosystem (demand)
– Empirical analysis
• Links between policies and equity financing
• Financing Policy Workshops:
– Norway (September 2012)
• Policy rationale, equity fund structures (public, FoF, co-investment)
– Switzerland (April 2013)
• Policy objectives, evaluations & policy mix
– Netherlands (tentatively planned for October 2013)
• Incentives for private investors, etc.
– Canada (connected with PPF)
2
Countries with Financing Instruments
Type of Instrument Number of OECD Countries
Change in Support (last 5 years) Grants, Loans and
Guarantees
30 Increased in 25 countries
Tax: YIC 9 New in 3 countries
Tax Incentives: Front- end
15 Increased or new in 9 countries
Tax Incentives: Back-end 12 Unchanged in most
Equity Funds: Public 14 Increased in 7 and new
in 3 countries
Equity : Fund-of-Funds 21 Increased in 8 and new in 8 countries
Equity Funds: Co- Investment
21 Increased in 11 and new in 6 countries
3 (32 out of 34 Member Countries Responding)
Policy Rationales for Intervention
• Policy rationales for intervention in seed and early stage finance
– Financing gaps and market failures (information asymmetries) – Spill-over effects (job creation, economic growth)
– Part of a broader economic development strategy
• Supply versus Demand side interventions – The majority of focus remains on the supply side
• Visible programmes demonstrating that action is being taken.
– However, demand side interventions are increasingly important
• Shortage of innovative entrepreneurs willing and able to start and grow companies.
• Lack of experienced investors, particular in equity investment.
4
Access to Debt Financing Has Become More Difficult
5 Note: Scale from 1 to 7 from hardest to easiest, weighted averages.
Source: World Economic Forum (2012), The Global Competitiveness Report 2012-2013 and World Economic Forum (2008), The Global Competitiveness Report 2008-2009, World Economic Forum, Geneva.
Ease of access to loans, 2007-08 and 2011-12
123456
SWE FIN NOR NZL LUX IDN BEL CAN USA AUS CHL NLD CHE ZAF ISR IND AUT DEU BRA CHN DNK JPN GBR FRA SVK TUR CZE EST ISL MEX RUS POL HUN PRT KOR SVN ESP ITA IRL GRC Index
2011-2012 2007-08
Shortage of VC Investment Europe vs U.S.
(USD million)
6
0 2000 4000 6000 8000 10000 12000 14000
0 20 40 60 80 100 120
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
Number of deals USD billions at current
exchange rates
VC investment in the United States (left-hand scale) VC investment in Europe (left-hand scale) Number of deals in the United States (rght-hand scale) Number of deals in Europe (right-hand scale)
Source: OECD (2013), Science, Technology and Industry Scoreboard 2013, calculations based on PwCMoneyTree, EVCA/Thomson Reuters/PwC and EVCA/PEREP Analytics.
Note: Data for the United States refer to market statistics, data for Europe refer to industry statistics.
Number of Tax & Equity Instruments
7 Equity relative to tax instruments
Note: These statistics do not reflect amounts committed or invested through the programmes.
By type of equity instrument
Source: 2012 OECD Financing Questionnaire and OECD Secretariat research
(2007-2012)
Evaluation
• Increasing support for financing instruments over the past 5 years, however, limited evaluation of these policies to date.
– Fewer evaluations of tax and equity instruments as compared to debt instruments (although many of these programmes are newer).
– Many evaluations of risk capital policies in OECD countries to date seem to have been more qualitative than quantitative.
• Better data collection is needed to monitor results and adjust programmes accordingly.
– Seed and early stage investment data is difficult to obtain
– Firms receiving support are not always tracked accurately by the implementing agencies – Appropriate counter factuals may not be available
• Other factors
– Impact of the institutional setting – Policy mix/interaction of policies
– Level of intervention: local, regional, national – Focal point of evaluation
– Timing
8
Findings from Policy Workshop in Glion, Switzerland
• The evaluation of SES market interventions entails significant data and methodological challenges.
– Having an ex-ante and well defined evaluation strategy is critical.
• This includes having a well-defined policy objective and putting thought into the policy questions and evaluation design at the beginning of the process.
– Evaluations should take into account the set of considerations related to the supply-side, demand-side and framework conditions.
• Challenging as it involves a mix of direct, indirect and external effects.
• Quantitative evaluations need to be complemented with qualitative information.
– Data is a key factor in successful evaluations but also a major challenge and cost component.
• Data needs to be tracked and harmonized.
• Evaluations in SES financing usually require a combination of different statistical methods.
• A cost benefit analysis should be done in terms of whether a large scale evaluation is warranted for relatively small programmes.
9
Findings from Policy Workshop in Glion, Switzerland (Cont.)
• There should be periodic evaluations and an effort to improve the quality of evaluations.
– Most importantly, the results of evaluations should feed back into the policy process to identifying areas for adjustment in the programme being evaluated.
– More and better evaluations are needed but trade-offs may exist between conducting good evaluations and delivering clear and timely policy messages.
• Most evaluations focus on supply-side SES financing policies.
– Due to their “softer” nature, it is difficult to evaluate demand side policies. The impact of framework conditions is also difficult to assess.
– Challenges remain in identifying complementarities between the different policies and it was suggested that further OECD work in this area would be useful.
• An understanding what other OECD countries are doing in terms of evaluation of financing policies is helpful in working towards developing better evaluation processes.
10
Types of Evaluation
• What is evaluated?
– General polices vs specific instruments
• What level is evaluated?
– Agency, Programme, Instrument
• What elements are evaluated?
– Design: Was it structured correctly?
– Process: Was it implemented according to plan?
– Governance: Was it managed properly?
• What type of evaluation is used?
– Quantitative vs Qualitative – Metrics: inputs, outputs, outcomes
– Benchmarking: How did it compare to other programmes (also in other countries)?
• Timing & frequency of evaluation
• Who conducts the evaluation?
– Government agency responsible – Other government officials
– External/third party: expert/academic
11
Selected Recent Evaluations*
12
Category Acronym Country Date Internal External- Gov
External-
Other Qual Quant Impact Pub
Tax: YIC
YIC BEL 2012
SR&ED CAN 2007
JEI FRA 2008
Tax: Front-end MEF FRA 2010
EII IRL 2011
SCS IRL 2006
EIS GBR 2008
Equity: Direct
VF DNK 2010
IM DNK 2012
HPSU IRL 2009
NSCS NOR 2009
Almi SWE 2010
IF SWE 2012
Equity: Fund-of-Funds
ARK BEL 2008
FII FIN 2003
CDC FRA 2012
YOZMA ISR 2004
VIF NZL 2009
UKIIF GBR 2012
CfEL GBR 2011
Equity: Co- Investment
IIF AUS 2011
PSF AUS 2008
HTG GER 2010
BPSV NLD 2008
ECF GBR 2010
Classified by type of instrument and evaluation
*From OECD financing questionnaire - see appendix for full names of instruments. Note classifications based on authors’ interpretation.
Tax Example: EIS (UK)
• Enterprise Investment Scheme (EIS)
– Programme in place since 1994 with periodic evaluations: 1999, 2003, 2008 – In 2008 EIS (and Venture Capital Trusts) was evaluated by independent
consultants
• Objective of evaluation: ex-post impact of scheme
• Data: obtained from HMRC and combined with company demographic and financial information (94-05) resulting in large panel of recipient and non-recipient firms
• Approach: recipient and non-recipient firms matched on observable characteristics to disentangle impact of scheme(s)
• Outcome measures: firm level (e.g. profitability, debt, employment, productivity, survival)
– Other EIS studies and reports:
• NESTA (2011) published report comparing previous evaluations and evaluation of other VC schemes (equity funds)
• OTS (2011) produced a report that includes recommendations regarding the clarification and simplification of EIS
• HMRC (2011) produced a report with outcomes of a stakeholder consultation launched in 2008
• Annual budget reports by HM Treasury
• HMRC regularly discloses statistics (http://www.hmrc.gov.uk/statistics/enterprise.htm) 13
Equity Example: PSF (Australia)
• Pre-seed Funds (PSF)
– Programme launched in 2001 (now closed to new investments) – External evaluation published in 2008:
• Objective of evaluation: effectiveness of scheme and whether it crowds out related scheme (Innovation Investment Fund, IIF)
• Data: obtained from Venture Economics (Thomson Reuters) and brief overview of PSF statistics based on governmental data
• Approach:
∙ Level analysis at the VC deal
∙ Comparison with competing scheme (IIF) allows inference on extent of crowding out
∙ Distinction of PSF, non-PSF and PSF-affiliated investee firms
• Outcome measures: fund level (e.g. development stage, industry of investment, portfolio size, staging, syndication, location, exits)
– Related studies and reports:
• Independent evaluations of the IIF scheme in 2007 and 2011
• PSF scheme regularly profiled in Innovation Australia annual report (up to present) and Industry Research and Development Board report (up to 2005-2006)
14
Conclusions
• Support for supply-side interventions in SES financing is growing – Yet evidence of the impact of these programmes is not conclusive.
– Better tracking of outcomes is needed.
• The full policy mix should be taken into account
– Understanding the full financing policy “treatment” received by firms.
– More attention should be paid to demand side measures, not just supply side.
– Framework conditions and potential regulatory barriers need to be assessed.
• Policy should focus on leveraging (not replacing) private funding.
– Creating incentives for institutional investors, funds, firms and individuals to invest in high growth firms.
– Ensuring that investment decisions are made by experienced professional investors.
• Design, structure and implementation of policies can make the difference between success and failure
– Data, evaluation and experience sharing is important.
– Experimentation and continual adjustment of policies is needed.
15
[email protected] [email protected]
For further information or
questions contact:
17
List of Abbreviations Used
IIF AUS Innovation Investment Fund PSF AUS Pre-Seed Fund
YIC BEL Federal Public Service Finance — Subventions et incitations fiscales sur la recherche et le développement des entreprises ARK BEL Arkimedes fonds
SR&ED CAN Scientific Research and Experimental Development Tax Credit VF DNK Vaekstfonden
IM DNK Innovationsmiljøer FII FIN Finnish Industry Investment JEI FRA Jeune Entreprise Innovante
MEF FRA Ministère de l'Economie et des Finances — Dispositifs fiscaux en faveur du capital-investissment das les PME CDC FRA CDC Entreprises
EII IRL Employment and Investment Incentive (formerly Business Expansion scheme) SCS IRL Seed Capital Scheme
HPSU IRL Innovative High Potential Start Up Funding
YOZMA ISR YOZMA Fund
BPSV NLD BioPartner Start-up Ventures NSCS NOR Nationwide Seed Capital Scheme
VIF NZL Venture Investment Fund Almi SWE Almi Invest
IF SWE Industrifonden EIS GBR Enterprise Investment Scheme UKIIF GBR UK Innovation Investment Fund CfEL GBR Capital for Enterprise Ltd
ECF GBR Enterprise Capital Funds HTG GER High-Tech Gründerfonds
