Can randomized controlled trials help develop effective green innovation policies?

Randomized controlled trials are a proven tool for understanding the causal effects of policies. Esin Serin asks if they can play a greater role in informing the development of effective green innovation policies in the

Randomized controlled trials are a proven tool for understanding the causal effects of policies. Esin Serin asks if they can play a greater role in informing the development of effective green innovation policies in the UK at this critical time for climate action.

Technology, alongside behavioral change, is essential for the UK to bring its emissions to net zero by 2050. Progress is needed to develop new technologies to reduce emissions where current technologies fall short. The UK is are already implementing policies to drive the required innovation in low-carbon technologies, with significant capital being deployed along the way. But how can policy makers know if their policies are working? Or works better than the alternatives? Or that they create additional benefits and not just subsidize the innovation that would have happened anyway? Randomized controlled trials offer a rigorous way to answer these questions.

Understanding the effectiveness of green innovation policies

Evidence of the effectiveness of green innovation policies is needed to identify the most cost-effective pathways to net zero, ensuring that public resources are not wasted but are spent on policies that create real positive impacts . After careful investigation, many policies may not work as intended. For example, a study review The evaluation of direct funding programs for innovation finds eight papers showing positive impacts of these programs on patent outcomes, but six others showing mixed or no impact. Another paper – drawing on evidence from randomized controlled trials that goes beyond a single innovation goal – goes further, suggesting “80% of things don’t work”.

Evidence on what works can help unlock significant growth opportunities for the UK if it can create a comparative advantage and capture a valuable share of the growing global green tech market. The UK government has already adopted growth driven by innovation as a long-term ambition and has included it in its net zero strategy. Research from the LSE’s Center for Economic Performance and the Grantham Research Institute has shown that the UK has a strong foundation, being specialized in green technologies overall and in some specific technologies where the estimated returns on public investment in innovation are particularly high, such as in offshore wind, tidal currents and carbon capture and storage.

Using Randomized Controlled Trials to Inform Green Innovation Policies

Randomized controlled trials involve assigning subjects such as individuals, households, or businesses (randomly) to a treatment group that receives the policy under study (e.g., a research and development grant) or to a control group who does not receive it.

Since subjects are randomly assigned to one group or the other, the control group should, in theory, be similar to the treatment group, and so the difference in mean scores between the two groups (e.g., research and development expenditure, patent applications) may be associated with the policy itself rather than individual characteristics or broader trends or influences.

Despite their potential, randomized controlled trials have so far been used very little in the context of green innovation. In effect, we identified only 29 trials globally (five in the UK) that are planned, conducted or underway. This is not surprising given the challenges of implementing randomized controlled trials, ranging from cost and time requirements to the ethical implications of assigning a policy on a random basis, even if it is for learning purposes.

Views on randomized controlled trials are also varied: some argue they represent a methodological “gold standard” for proving causal impact, while view of others they too are insensitive to the importance of context because they can measure impact with high confidence, but by themselves tend to provide little information about how and why the measured impact occurred. There is also unique challenges the implementation of randomized controlled trials in the specific context of green innovation.

Can randomized controlled trials play a greater role?

From priority areas for green innovation for the UK, we found that energy storage and flexibility, homes and buildings, and hydrogen may present fertile ground for evidence based randomized controlled trials building on policy innovation. This is primarily because these areas involve some nationwide, consumer-facing technologies that are likely to be developed by a large number of innovators, making them conducive to building evidence based on controlled trials. randomized.

The contribution of randomized controlled trials to achieving green outcomes can be maximized if they are used as a means to make innovation policy more effective. innovativeshifting the emphasis from “evaluating” if a single policy design works to approaching policy as an experiment in its entirety – i.e. exploring a range of policy ideas, testing those that show promise on a small scale and basing subsequent decisions on those lessons.

The Basic funds in business of the Department of Business, Energy and Industrial Strategy uses an experimental approach, supporting the testing of different ideas that encourage small and medium-sized enterprises to adopt technologies and management practices that boost productivity. Large trials seeking access to the fund must have a evaluation methodology such as a randomized controlled trial this will allow strong conclusions to be drawn about what works or does not work. The recently launched Green Home Funding Accelerator is taking a similar approach, allocating part of its £20m funding to pilot a range of innovative financing mechanisms that encourage home energy efficiency and low-carbon heating retrofits.

Randomized controlled trials as part of a broader culture of evidence-based policy-making

Of course, randomized controlled trials are not the only source of evidence. Policy makers can draw on years of learning-by-doing and other rigorous methods of evaluating broader innovation policies (see this toolbox) as well as on green innovation in particular. For example, we know that progress to date with some key green technologies has not been driven by generic investments in technology-neutral research and development or by carbon pricing, but rather by sets of innovation policies involving deliberate technological choices.

The UK contracts for difference mechanism has not been tested in a randomized controlled trial, but no one can deny its role in reducing offshore wind costs in the UK by around 70% over a decade. The investment signal that the policy sent also had broader, longer-term impacts on the development of the sector that may not have been captured in a randomized controlled trial.

Randomized controlled trials are an important enabler of evidence-based policymaking rather than an end in themselves. We cannot randomize every green innovation policy, but we can be more proactive in recognizing opportunities to use randomized controlled trials to learn more about the role different policy instruments can play in achieving specific technology goals. . As a result, we can learn that not all policies have the intended impact, but in doing so, we could build an evidence base to accelerate innovation and growth aligned with net zero.


about the authors
Esin Serin is a Policy Analyst at LSE’s Grantham Research Institute on Climate Change and the Environment.

Featured image credit: L.Filipe C. Sousa on Unsplash

Printable, PDF and email version