Evaluating the experience of buying Heat as a Service
new experience has been working with the Energy Systems Catapult as a UX research partner since July 2016 to trial and evaluate the concept of buying heat as a service, one of the longest and most important projects on which we’ve been engaged.
The Catapult is a not-for-profit organisation working with industry and government to clear the barriers blocking innovators from bringing new products, services and business models to market.
Domestic heating accounts for 20% of carbon emissions so reducing this source of carbon is important for meeting targets. Earlier research indicated that people care more about their heat experience than the technology that keeps them warm. So the Catapult’s ingoing hypothesis was that if people could get the heating outcomes they wanted, then their service provider could deliver these outcomes using a heating system based on zero-carbon energy – as long as it was as good or better in terms of customer experience, than what had gone before. The Department for Business Energy and Industrial Strategy (BEIS) commissioned the Catapult to develop a proof of concept and the capability to enable businesses to sell energy as a service.
Bespoke system development and installation
To test out the hypothesis and the thinking behind it the Catapult developed a bespoke system to simulate the experience of buying heat as a service. The system incorporated advanced features including setting temperature by individual room and time, control by phone, and ‘smart warm up’ designed to achieve target temperatures, at pre-set times. The thinking was that such control over heating would allow customers to articulate what they wanted in terms of heating outcomes, as well as understand more about the characteristics of their home that affect their heat experience. The system simulated the experience of buying desired heating outcomes in units of Warm Hours delivered, rather than by kWhs of gas. A Warm Hour was defined as an hour of heating at a requested temperature, regardless of how many rooms were being used. This idea of buying Warm Hours was also reflected in the purpose-built user interface.
The system used sensors to capture temperature and humidity data in each room, as well as recording all request data. This body of data could be used to understand what people were doing with the system as well as learn how much outcomes might cost to deliver for a provider in a commercially-viable environment. Following a smaller-scale trial in 2016/17 the system was refined and then installed in 100 homes in Birmingham, Newcastle and Bridgend for winter 2017/18.
Choice of three Heat Plans
After using the system for a few weeks trialists could then choose to take out one of three ‘Heat Plans’ tailored to their home in terms of number of Warm Hours and the price per Warm Hour. The number of Warm Hours was based on their heating schedule and the price per Warm Hour was calculated according to the data that the Catapult had been collecting over the previous weeks. The plans had a fixed weekly price and were offered on a 12-month contract.
Fixed was the cheapest plan offering a specific number of Warm Hours based on the trialist’s heating schedule. They couldn’t change the schedule from then on, but they could buy ‘Extra’ Warm Hours as and when needed, but at a higher price. Flexitime was mid-priced and offered the trialist’s scheduled number of Warm Hours plus a bundle of ‘Spare’ Warm Hours to use as and when needed; the trialist was free to change their schedule as they wanted.
The Unlimited option was the most expensive and offered unlimited Warm Hours for a fixed cost. There were also a number of ‘fair use’ restrictions such as heating rooms at times they hadn’t previously been heated, or consistently requesting temperatures warmer than previously scheduled.
Although trialists never really bought a Heat Plan, they did have some skin in the game because the amount they thought would pay related to their use of the Heat Plan not their actual gas consumption. The Catapult ensured that no one paid more for their heating than the cost of their plan.
Data collection methodologies
Throughout the trial new experience researchers worked closely with the Catapult team to collect qualitative data. It was very important to establish rapport with trialists who we hoped would provide us with feedback over a long period so we started with introductory calls, partly to learn about the trialists but also to introduce ourselves and answer their questions and concerns.
During the trial period we visited trialists in their homes to interview them about their experiences with the system, but also, to have them take us on a tour of their home where we could discuss issues relating to thermal comfort in each room, and observe factors that might impact on thermal comfort, such as curtains or a sofa obscuring a radiator, draughts, or radiator size, and use of radiators to dry clothes. These observations would help explain findings later in the study.
We also had trialists keep blogs about their lived experiences with the system, and with Heat Plans. They could update their blogs from their mobile, allowing them to give us in-the-moment thoughts about their experience; they could upload videos and screen recordings, and we also set them periodic tasks to complete and report on. To dig deeper into trialists’ experiences of choosing and living with Heat Plans based on Warm Hours, we also conducted workshops with a sub-sample of the trialists, using various creative and other exercises.
The system collected passive data from sensors in every room of each home as well as active usage data such as turning on and off, overriding, changing target temperatures, and making setting changes. During one round of visits in the fieldwork we came armed with plots of both sensor and request data, allowing us to discuss and make sense of some of the patterns we were seeing.
Analysis approach
We started by analysing the qualitative data we had collected from home visits, interviews, blogs and workshops. As a distributed team of researchers working with data from so many homes collected across many channels over many months we found using an online, collaborative analysis tool invaluable for managing and tagging this vast quantity of mixed-method qualitative data. From this analysis we developed our findings around the user experience of the system and of buying Heat as a Service.
We also looked to marry quantitative sensor and request data with the qualitative data to understand the motivations behind users’ behaviour patterns. Working with the Catapult team, we developed a methodology using parameters of Time, Space and Temperature to look for clusters in patterns of behaviour that might constitute groups of users. Once we had created the clusters we then layered qualitative data over them to help explain the reasons why groups of trialists behaved the way they did.
User experience findings
Early data collection during the winter of 2016/17 – before the introduction of Heat Plans – focused on improving the user experience of the system to support trialists in being able to better articulate and then achieve the heating outcomes they were looking for. We made some key recommendations relating to system behaviour and the user interface that could be implemented before the winter of 2017/18 and the rollout of Heat Plans.
One very important early finding was the importance that some trialists placed on experiencing radiant as opposed to just ambient heat, whether to dry laundry or because it was fundamental to them in achieving thermal comfort. And in these homes we also observed from the quantitative data that these trialists were making frequent increases to their requested temperatures in certain rooms to achieve radiant heat. An unforeseen consequence of the sophistication of the system with its room-by-room thermostatic temperature control was that the radiators would stop radiating heat when the target temperature had been achieved, a phenomenon that those who had not previously had thermostatic control were unused to. For some the frustration was immense nearly leading to rejection of the system. This requirement of some trialists to experience radiant heat had not been factored into the design of the system. The finding led to the inclusion of Laundry Dry and Heat Burst features into the user interface, allowing users to request radiant heat in specified rooms, on demand.
This phenomenon of trialists demanding radiant heat acted as a good example of people learning to articulate the type of heating outcomes they required and so they could be said to have become more discerning heating consumers.
Reactions to buying and living with Heat Plans
Full analysis is ongoing at the time of writing but we learnt a lot about people’s attitudes towards Heat Plans from workshops we ran in the three locations, among trialists who contracted into a Heat Plan.
The basic concept of Heat Plans and Warm Hours was very well received, giving trialists a feeling of greater control and certainty over the cost of their heating; they readily accepted the idea of paying according to Warm Hours rather than kWhs and liked the idea that the risk of providing the outcome – no matter the weather – was transferred to the supplier; indeed in taking out the Heat Plans some of them demonstrated that they were potentially prepared to pay a premium for the heat outcomes they wanted.
Most chose a Fixed or Flexitime plan, the higher price of Unlimited putting people off, and they doubted they would benefit from being able to have heating on for more hours than they previously had, in some cases feeling it would be wasteful. From a user experience point of view it was important that the UI make very clear how many Spare hours were left in a week, the cost of Extra hours incurred and when the weekly hours would refresh. Some people didn’t like the idea that the Heat Plans were taken out on 12-month contracts based on 52 weekly charges, so they felt they were paying for heating when they didn’t need it in Summer. This isn’t really different to the way people currently pay for gas by direct debit across the year, except that from a perceptual point of view they did not feel like they were building up a credit during months they didn’t have their heating on.
This finding has led the Catapult to highlight the potential of other types of contract e.g. Winter contracts where the costs would be spread over 6 months only, or Pay As You Go, where you pay according to the number of Warm Hours that you use. A related finding was that trialists felt it unfair that the plans worked on a weekly use-it-or-lose-it basis - despite the fact that this is commonplace with a mobile contract albeit monthly; the Catapult is now exploring the potential for rollovers to address this.
Making sense of behaviour patterns to offer plans in an automated manner
An important aspect of bringing to market a concept like Heat as a Service will be for energy service providers to offer tailored Heat Plans to prospective customers with the plans and pricing created by algorithms. While full analysis is ongoing, findings relating to data from the first year of the trial are pointing the way to which algorithms will need to be ‘clever’ enough to make sense of behavioural patterns in terms of deducing underlying motivations, when offering Heat Plans in a low-carbon environment. An interesting example is how some trialists who made frequent up and down adjustments to temperatures, according to when rooms were or were not occupied, were driven by a desire to minimise their spend on heating. In a low carbon environment predictability of heating behaviour could allow providers to offer the lowest price plans. So systems will need to be able to interpret behaviours and then perhaps re-educate customers about the plans and behaviours that will best suit their underlying objectives.
The trial is now moving into a third season with the Catapult’s Living Lab, while we continue to analyse data from the second season. You can find out more about the Catapult and its Living Lab on their website https://es.catapult.org.uk/projects/smart-systems-and-heat-ssh
What the Energy Systems Catapult had to say about working with us
“Over the course of two years we have worked very closely with researchers from new experience. We have benefitted greatly from the team’s research skills, their experience of running longitudinal live trials, their ability to develop insights by combining quantitative with qualitative data, as well as the flexibility, friendliness and smart thinking of the new experience people.” Matt Lipson, Head of Consumer Insight, Energy Systems Catapult
