TRACKING RESULTS - Making Excellence A Competitive Advantage

Promises of savings without tracking DRIVES ME NUTS.  People are being harmed and we don't have any idea how big the harm is.  Comprehensive Home Performance really takes off once you connect tracking results because now you can offer a thing very rare.  

You can offer TRUTH.  You can offer PROOF that what you say will happen, will happen.  Delivering the "5th 

How do we create pressure for Utilities to open up their data so tracking can be automated?  

We demand it!!  

We create pressure from all sides.  

Scroll down to the first COMMENT to see a letter I recently sent this to HPwES contractors in NY, and EE advocates in the US, Canada, and Mexico:

Dear Contractors,


Below are; Some metrics, Competing for results, and a much better way of managing data:
 
Some of you may be aware that last spring I requested and received a large amount of data under the Freedom of Information Act from NYSERDA regarding your HPwES projects.  

You also may know that I feel this program needs to move toward a new competitive metric - one of published ability to deliver savings projected.  This will do for our industry what MPG ratings did to the auto industry, it will create incentive to excel around a metric that benefits the consumer and adds credibility to our promises.  It will allow us to market based upon truth and accountability, not smoke, mirrors and hope.  Sell to reason, not fickle emotion.  
 
A program that can deliver TRUTH is unique in today's marketplace.  This could elevate our profession immeasurably! 

Tracking before and after energy would be ideal but right now there is an access problem to energy consumption data "after" improvements.  Until we can solve this, do we have anything that might indicate what results to expect by contractor?  Well, we do have access to "reported energy use before" and "modeled energy use before".  
 
 
Certainly someone who uses 1000 therms who has a model showing they use 2000 therms will save 40% is not likely to see 800 therms savings.  
 
 
So we can rank ability to accurately model energy consumption pre-improvement with the expectation it will correlate to accuracy of savings projections?  Highly likely.   And seeing these numbers helps contractors understand how accurate their modeling is.   

Do No Harm.  To roll out a ranking or registry without contractor understanding of what is coming is likely to cause harm, so providing this interim step information in a format indicating what will eventually be public information provides opportunity to adjust behavior and policies ahead of time.  Forewarned is forearmed. 

Below is a representation of the data available, and an idea of what a realization ranking might look like.  Those closest to the line have the models that most closely represent consumption of the homes they audit, and those further to the left tend to overstate consumption:
 
Inline image 3

 
Inline image 4
 
Inline image 5
 

There is a lot that can be done here, for example these numbers could be run quarterly to show who is improving and who is sliding.  You can't manage what you don't measure, and this should be managed. Nick Taylor is working on making these numbers so people can dig in and understand how to improve results.  You can play with the charts here: http://bit.ly/NickTaylorsCharts
 
This Christmas I will be receiving a new data set, and a lot of additional information, which I will share with anyone interested.  
 
I'm also happy to discuss what happens to the work once you commit to tracking results.  (HINT: When you have 2 tests coming up, one you automatically pass just for showing up and the other one is graded, which do you study hardest for?)
 
Next year NY HPwES will be transitioning from CSG's citrix to Energy Savvy, which looks like a step in the right direction! Maybe NYSERDA will consider tying things to Salesforce and Hubspot.  Things may look discouraging now, but they're poised to get better!  I'd love to hear from those who do not plan to renew their Accreditation, that might be a mistake.  

Sincerely,

Ted Kidd
(585) 205-8674 Direct 
Energy Efficiency Specialists
EES Blog

Published Realization Rates - Let's take the "blind buy" out of home performance.
P.s.:  Thank you to those who've sent words of encouragement and appreciation.  And as always, to those who wish to be removed from this list please let me know.   
 
Good faith attempts to remove clearly bad data were made.  The analysis was performed by an independent 3rd party with absolutely no connection or bias toward any NY contractors or the program.  Anyone having issue with accuracy of the data please contact NYSERDA and CSG.  

Views: 130

Tags: energy, monetize, promise, realization, tracking, truth

Comment

You need to be a member of Home Energy Pros to add comments!

Join Home Energy Pros

Comment by Curt Kinder on December 16, 2012 at 6:56am

We work quite closely with the conservation group at our local utility, JEA. They readily manually provide any consumption data they have to further the cause.

I'm told they are rolling out web access to 15 minute whole house consumption data. That will certainly help.

What we really need is inexpensive, easy to install, robust, reliable, web-enabled sub metering.

Comment by tedkidd on December 13, 2012 at 8:16am

Dear Contractors,


Below are; Some metrics, Competing for results, and a much better way of managing data:
 
Some of you may be aware that last spring I requested and received a large amount of data under the Freedom of Information Act from NYSERDA regarding your HPwES projects.  

You also may know that I feel this program needs to move toward a new competitive metric - one of published ability to deliver savings projected.  This will do for our industry what MPG ratings did to the auto industry, it will create incentive to excel around a metric that benefits the consumer and adds credibility to our promises.  It will allow us to market based upon truth and accountability, not smoke, mirrors and hope.  Sell to reason, not fickle emotion.  
 
A program that can deliver TRUTH is unique in today's marketplace.  This could elevate our profession immeasurably! 

Tracking before and after energy would be ideal but right now there is an access problem to energy consumption data "after" improvements.  Until we can solve this, do we have anything that might indicate what results to expect by contractor?  Well, we do have access to "reported energy use before" and "modeled energy use before".  
 
 
Certainly someone who uses 1000 therms who has a model showing they use 2000 therms will save 40% is not likely to see 800 therms savings.  
 
 
So we can rank ability to accurately model energy consumption pre-improvement with the expectation it will correlate to accuracy of savings projections?  Highly likely.   And seeing these numbers helps contractors understand how accurate their modeling is.   

Do No Harm.  To roll out a ranking or registry without contractor understanding of what is coming is likely to cause harm, so providing this interim step information in a format indicating what will eventually be public information provides opportunity to adjust behavior and policies ahead of time.  Forewarned is forearmed. 

Below is a representation of the data available, and an idea of what a realization ranking might look like.  Those closest to the line have the models that most closely represent consumption of the homes they audit, and those further to the left tend to overstate consumption:
 
Inline image 3

 
Inline image 4
 
Inline image 5
 

There is a lot that can be done here, for example these numbers could be run quarterly to show who is improving and who is sliding.  You can't manage what you don't measure, and this should be managed. Nick Taylor is working on making these numbers so people can dig in and understand how to improve results.  You can play with the charts here: http://bit.ly/NickTaylorsCharts
 
This Christmas I will be receiving a new data set, and a lot of additional information, which I will share with anyone interested.  
 
I'm also happy to discuss what happens to the work once you commit to tracking results.  (HINT: When you have 2 tests coming up, one you automatically pass just for showing up and the other one is graded, which do you study hardest for?)
 
Next year NY HPwES will be transitioning from CSG's citrix to Energy Savvy, which looks like a step in the right direction! Maybe NYSERDA will consider tying things to Salesforce and Hubspot.  Things may look discouraging now, but they're poised to get better!  I'd love to hear from those who do not plan to renew their Accreditation, that might be a mistake.  

Sincerely,

Ted Kidd
(585) 205-8674 Direct 
Energy Efficiency Specialists
EES Blog

Published Realization Rates - Let's take the "blind buy" out of home performance.
P.s.:  Thank you to those who've sent words of encouragement and appreciation.  And as always, to those who wish to be removed from this list please let me know.   
 
Good faith attempts to remove clearly bad data were made.  The analysis was performed by an independent 3rd party with absolutely no connection or bias toward any NY contractors or the program.  Anyone having issue with accuracy of the data please contact NYSERDA and CSG.  

Home Energy Pros

Home Energy Pros was founded by the developers of Home Energy Saver Pro (sponsored by the U.S. Department of Energy,) and brought to you in partnership with Home Energy magazine.

Latest Activity

Kevin Jordan posted photos
1 hour ago
Susan E. Buchan's event was featured

EEBA Excellence in Building Conference at Doubletree Union Station Hotel

September 23, 2014 at 8am to September 25, 2014 at 2pm
17 hours ago
Susan E. Buchan posted an event

EEBA Excellence in Building Conference at Doubletree Union Station Hotel

September 23, 2014 at 8am to September 25, 2014 at 2pm
18 hours ago
Rem Husted replied to Andy Gostisha's discussion Disguising Ductless Heat Pump Units
"I think most consumers wouldn't want a big box called a refrigerator or a stove or…"
18 hours ago
Scott Katznelson replied to Scott Katznelson's discussion Database of zip code or county by climate zone
"You know those maps you can find for energy code climate zone specific reference.  That's…"
19 hours ago
JEFFREY M HUGO, CBO replied to Richard Beyer's discussion Sprinkler Mandate Debated.. What do you think should be done?
"Fire sprinklers operate when the fires are small. Typically it only takes approx 20 gallons per…"
19 hours ago
Scott Katznelson replied to Scott Katznelson's discussion Database of zip code or county by climate zone
"Thanks.  That's the best source I've yet found, and maybe the best that's out…"
19 hours ago
Diane Chojnowski replied to Scott Katznelson's discussion Database of zip code or county by climate zone
"I posted the question on HEP's facebook page and Bill Spohn commented: Try…"
20 hours ago

© 2014   Created by Lawrence Berkeley National Laboratory.

Badges  |  Report an Issue  |  Terms of Service