Energy Open Data Challenge

By Daniel Epstein

The Energy Problem

The UK faces huge challenges in the area of energy with social, environmental and economic consequences at individual, city and national levels. We are in an energy trilemma: the costs of fossil fuel-derived energy in our homes and businesses is increasing, we are struggling to limit our greenhouse gas emissions and our reliance on energy purchased from abroad raises questions about our energy security.

Wrongly or rightly, trust in “big six” energy companies who supply 98% of our energy has broken down. Whilst the politicians are busy arguing about how to open up the market to competition and cap energy prices, the real issue eludes us. For too long now the UK has lacked a long-term comprehensive strategy for energy that addresses the ‘trilemma’ head on.

That is not to say that the market does not need to be reformed. Many people argue that vertical integration in the energy sector means that whilst the energy companies’ quoted retail profits may not be extortionate, generation profits are far less transparent. Further, switching can be sufficiently complicated that customers are deterred from doing so, reducing competition between the suppliers to attract customers with low prices. It is also difficult for new suppliers to enter the energy market due to high start up costs. These issues have led to discussions over appropriate levels of government intervention in energy markets, though questions around the impacts of centralisation, market distortion and vested interests make this a complex topic in itself.

Big and Open Data

Big data is a buzzword – everyone’s talking about it. Big datasets contain lots of data, which are often complex, and might have several different formats, making it difficult to use a simple analysis technique to understand what it shows. It’s been called “black gold” because many think it could be the new fuel for gaining insights into the world around us and innovating on a whole new scale.

Open datasets are openly available – anyone can make use of or distribute them. The UK is the world leader on open data, coming top out of 70 countries in the Open Knowledge Foundation’s Open Data Index. From the website a range of datasets on health, economic, geographical, transport, business and crime can be downloaded. A lot of data is generated by governments and councils. The government is gradually requiring these bodies to make that data freely available, both to make the public sector more accountable but also to promote innovation through data, and many feel that data on all publicly funded operations (except of course where this would raise a risk e.g. to security) should be made open. Clearly data protection must be taken into account and datasets are sometimes aggregated to a level where individuals or individual addresses are not identifiable.

Why Open Data can help us with our energy problem

Our energy challenges have complex and far-reaching social, economic and environmental implications. Yet they are shrouded with opacity due to a lack of available data, for example, energy consumption data is held by energy companies. Conflicting drivers and the lack of trust in energy providers make it difficult to know who should act to resolve these problems, and how.

Energy presents a variety of challenges. We design for peak loads, meaning that all of our systems have to be fit to meet the very highest demand, even though it only happens a couple of times a year. Our systems have always been one way, the idea of moving to a smart grid is that systems can transition from reactive and responsive, to active. To do this data must flow both ways.

Perhaps the most challenging aspect of understanding energy use, and changing our consumption, is understanding how it connects to our behaviour and lifestyle. Do you know whether you use a lot of energy compared to other people in your area? How much difference does turning off your light or your computer, or your heating, or getting double glazing, really make? There are now devices on the market which gather data on behaviour to adjust the way energy is used in your home. Google recently acquired Nest, who designed a thermostat that learns your behaviours, schedule and preferences, and allows you to control your heating from your phone. Nest claims that their thermostat can reduce energy consumption by up to 20%. Google clearly think there’s something in it as they bought Nest for $3.2 billion. This idea of getting systems to work around us could deliver fantastic benefits. These systems are data-hungry; the more data you put in, the more benefits you get out.

Google has also recently acquired DeepMind, an artificial intelligence company which builds learning algorithms. This next generation analysis uses vast quantities of data to “train” computers to detect patterns in data, rather than programming in the rules, inspired by the way the human brain works. Here the data takes centre stage.

In looking to data to help reduce energy consumption, we can also draw links with nudge theory which uses positive reinforcement and indirect suggestions to change behaviour. The provision of energy data to users in a digestible format can provide reinforcement of their actions through demonstrating the real impacts they have had, giving users a “feel” for their energy use.

Open data provides yet more opportunities to provide the right information, presented in the most appropriate way, to individuals, families, groups, councils and policy-makers.

Firstly, we need to ask the right question. This needs to go right to the heart of the issue and be specific, relevant and feasible to solve. Getting the question right is essential to give direction to work in this area and ensure that solutions match the challenge.

Now imagine that a whole range of relevant datasets could be combined on a single platform. A dataset wishlist might include energy consumption, housing stock condition, energy saving measures installed to date, meteorological and socio-economic data, depending on the problem to be solved.

Next bring together teams of programmers who can collaboratively hack the datasets in innovative and creative ways to shed light on the underlying situation, creating visualisations that allow governments, councils, companies or citizens understand how we use energy individually and collectively.  A multitude of problems could be addressed – how to discover fuel poor households, how to target energy saving schemes like Green Deal and ECO, how to communicate energy use to consumers and engage them in energy saving, how to understand which energy saving interventions are most effective in reality, rather than just in the lab.

Faced with such a wide range of potential challenges, we have engaged many experts to understand what the most pressing issues are in this area. One theme keeps coming up – community energy. It’s a hot topic, DECC have identified over five thousand energy community groups with diverse interests from energy bulk-buying to micro generation, and have just published a Community Energy Strategy. In the face of the market failure described above, working out the best way to generate, use, conserve or share energy in a local community group is becoming increasingly popular, with over 3.5 million bill payers keen to form groups across the UK.

The Open Data Challenge Series

The TSB recognises the opportunities that open data provide in solving social challenges and has funded the Open Data Institute and Nesta to run seven open data challenge series where  teams innovate to develop new products and services using open data. We are the series lead for the Energy and Environment series.

We will support the ODI and Nesta to run an innovation and creation weekend, or hackathon, on March 22nd-23rd 2014. We have formed a partnership with Bristol to be our host city for the challenge weekend and to help us build the network and define the challenge. We will bring together teams from a range of sectors including programmers, academics, public and private sector companies, students and entrepreneurs in social, economic and energy issues. The teams will take part in the hackathon weekend, thinking of new and interesting ways develop a new product or service into the community energy sector. The best three ideas will take part in an accelerator programme, getting incubator support for three months along with £5k to realise their concept into an economically viable product; the best of those will win £40k to get their idea up and running.

There are lots of datasets out there that could provide real insight, and hence make progress towards carbon reduction targets or solving social problems.  It will be essential to open up the right datasets so that we can get to grips with these complex, challenging problems and really start to illuminate solutions and ways forward.

Open data in itself will not solve the energy challenge. It’s by no means a silver bullet. But a programme like this gives us new insights, promotes new thinking, and can deliver new products and services that can help people help themselves. We think this is just the beginning for using open data to impact the way we understand, use and generate energy in the UK. More importantly it provides new useful tools and services that will support the 3.5 million people that want to engage in the energy sector.

We would love to hear from others working in this space; if you are a data owner, have knowledge in this area or would like to compete in the innovation and creation weekend please get in touch. On March 3rd 18:00-20:30 we will hold a meet up in Bristol – come along to discuss the challenge and meet others working in this area. See who else is involved and look for team mates on our Collabfinder page, and follow @odchallenges on Twitter.

Post by Dan Epstein and Rosemary Willatt who are working on this project.

Share This:

Facebook Twitter Google+ LinkedIn