What two federal field studies found when they measured real heat pumps
The short version
- PNNL instrumented 22 cold climate heat pumps and measured a median COP of 1.9 in the 0 to 5F bin, with median COPs below 30F ranging from 1.6 to 2.7.
- NREL instrumented 12 variable speed heat pumps and found auxiliary heat energy exceeding 35% of compressor heating energy at 5 of the 12 sites. At 2 sites the auxiliary strips used more electricity than the compressor did.
- The two studies look contradictory and are not. PNNL tested advanced prototypes sized by Manual J load calculation. NREL measured ordinary installed equipment. The spread between installations is wider than the spread between machines.
- This is the single best argument for treating backup heat share as an input rather than a constant, which is what this site does.
The Department of Energy has funded two separate programmes to instrument heat pumps in occupied American homes and measure what they actually do. Read together, the results explain more about heat pump bills than any efficiency rating does.
Why field data and not ratings
Efficiency ratings are measured in a laboratory, against a standard test procedure, in one climate region, paired with an assumed building. They are honest numbers and they are useful for comparing one machine to another. What they cannot tell you is what a machine will do in a specific house with a specific installer and a specific thermostat.
For that you need somebody to put instruments in real houses and leave them there through a winter. The US Department of Energy has funded two programmes that did exactly this, and both published their raw findings. I have read both and pulled the numbers below directly from the reports rather than from anyone's summary of them.
PNNL: 22 cold climate heat pumps, measured through a winter
The Pacific Northwest National Laboratory ran field validation for the DOE Cold Climate Heat Pump Technology Challenge, a programme that pushed manufacturers to build heat pumps that hold performance at low temperature. PNNL instrumented 22 of the resulting prototype units in occupied homes and reported in January 2025. [PNNL-37127 (2025)]
The COP numbers
| Condition | Measured COP |
|---|---|
| Median across sites, outdoor temperature below 30F | 1.6 to 2.7 |
| Median in the 0 to 5F bin | 1.9 |
| ENERGY STAR cold climate requirement at 5F | above 1.75 |
| DOE Challenge requirement at 5F | 2.1 or 2.4 |
Two things are worth sitting with. The first is that a median COP of 1.9 at 0 to 5F is still comfortably better than electric resistance heat at 1.0, which is the relevant comparison for an all electric house. The second is that 1.9 is a long way below the numbers people carry around in their heads from equipment marketing.
PNNL put it plainly: while the laboratory performance of the prototype units is significantly more efficient, the observed COP of most units in the field is also higher than 1.75. That is a carefully worded sentence. The machines cleared the bar. They cleared it by less than the laboratory suggested they would.
The auxiliary heat numbers
In the coldest bin PNNL reported, minus 10 to 0F, auxiliary heat was engaged 25% of the time. Observed switchover temperatures, where supplemental heat first appears, clustered around 30 to 35F. Above 0F, auxiliary only operation was minimal, and the highest stage of auxiliary heat was essentially never used, under 0.001% of operation.
One caveat PNNL flag themselves, and it matters: many sites had milder winters than normal, and only 8 of the 22 sites recorded 10 or more hours of heating below 0F. The coldest conclusions rest on the smallest samples.
NREL: 12 ordinary installations, and a very different picture
The National Renewable Energy Laboratory ran a separate study, instrumenting 12 centrally ducted variable speed air source heat pumps in single family homes, mostly in Washington and Montana with one outside Denver. Measurements were taken at five second intervals across the 2021 to 2022 winter, with six sites continued through a second winter. [NREL/TP-5500-84745 (2023)]
Their auxiliary heat findings are the ones that should change how you think about this.
Read that second sentence again. At two houses in this study, the expensive part of the system was doing more of the work than the efficient part. Those households owned a heat pump and were paying resistance heat prices for the majority of their heating electricity.
Converting NREL's figure into the one that matters
NREL reports auxiliary heat as a share of compressor energy. This site models backup as a share of heat delivered. Those are different quantities and confusing them will mislead you, so here is the conversion.
Compressor electricity of E at a COP of c delivers heat equal to c times E. Auxiliary electricity of A at COP 1.0 delivers heat equal to A. So the auxiliary share of heat delivered is A divided by the quantity c times E plus A.
| Aux energy as share of compressor energy | Heat share at COP 2.5 | Heat share at COP 3.0 |
|---|---|---|
| 20% | 7% | 6% |
| 35% | 12% | 10% |
| 50% | 17% | 14% |
| 100% | 29% | 25% |
So NREL's 35% threshold corresponds to roughly 12% of heat delivered at a COP of 2.5. The two sites where auxiliary energy exceeded compressor energy correspond to roughly 29% of heat coming from resistance strips.
Put that last figure through the effective COP formula and a nominal 3.22 becomes 1.97. The machine on the label and the machine on the bill are not the same machine.
Why the two studies disagree, and what that tells you
PNNL says auxiliary heat is a small fraction. NREL says it exceeded compressor energy at two sites. Both are correct, and the difference between the samples is the finding.
| PNNL | NREL | |
|---|---|---|
| Equipment | DOE Challenge prototypes, purpose built for cold climates | Ordinary market variable speed units |
| Sizing | Manual J load calculation performed at every site | As installed |
| Sites | 22 | 12 |
| Auxiliary heat result | A small fraction of delivered heat | Exceeded compressor energy at 2 sites |
The PNNL sample is what a heat pump installation looks like when the equipment is at the front of the market and somebody ran the load calculation. [ACCA Manual J] The NREL sample is closer to what gets installed.
What I changed on this site because of these studies
Three things.
1. Backup heat stays an input, and I stopped apologising for the default
This site defaults to 10% of heat from resistance backup. Against PNNL's well sized prototypes that is a fair number. Against NREL's ordinary installations it is optimistic, and for the two worst sites it is off by a factor of five. Rather than pick a number and hide it, the calculator exposes it and this page tells you what the real range looks like.
2. I now cite a measured COP range rather than only a derived one
The site converts HSPF2 to COP arithmetically, which is correct as arithmetic. PNNL's measured range of 1.6 to 2.7 below 30F is the reality check on that conversion, and it belongs next to it.
3. I corrected a claim I had wrong
I had written that roughly 40% of US homes do not heat with natural gas. The actual figure is higher. Natural gas is the main heating fuel in 47% of homes, down from 49% in 2010, and 42% of households report electricity as their main heating fuel. [EIA Today in Energy] So slightly more than half of American homes do not heat with gas. That correction is logged in the corrections log.
What this means for someone getting a quote
The research points at a small number of questions that separate a good installation from an expensive one.
- Was a Manual J load calculation performed on this house? Every PNNL site had one. It is the single clearest difference between the two samples.
- What is the balance point, and what share of annual heat do you expect from auxiliary? A contractor who has done the load calculation can answer both. See balance point.
- What is the published COP at 47F, 17F and 5F? Not the seasonal average. The curve. ENERGY STAR's cold climate threshold is a COP above 1.75 at 5F [ENERGY STAR cold climate criteria], and PNNL measured a field median of 1.9 in the 0 to 5F bin, so that is the realistic band to judge a claim against.
- Is the backup electric resistance, or an existing furnace? If a furnace is already there, keeping it as backup removes the COP 1.0 penalty entirely. See dual fuel.
Common questions
What COP do heat pumps actually achieve in cold weather?
PNNL measured 22 cold climate heat pumps in occupied US homes and found median COPs below 30F ranging from 1.6 to 2.7, with a median of 1.9 in the 0 to 5F bin. That is well above electric resistance heat at 1.0 and well below the figures typically quoted from laboratory ratings.
How much auxiliary heat do real heat pumps use?
It varies enormously by installation. NREL found auxiliary heat energy exceeding 35% of compressor heating energy at 5 of 12 sites, and exceeding compressor energy entirely at 2 sites. PNNL, testing better equipment that had been sized by load calculation, found auxiliary heat to be a small fraction of delivered heat.
Do these studies mean heat pumps do not work in cold climates?
No. Even the lowest measured field COPs are well above electric resistance heat, and PNNL's units cleared the ENERGY STAR cold climate threshold. What the studies show is that installation quality, particularly load calculation and sizing, drives more variation in running cost than the choice of equipment does.
Where can I read the underlying data?
Both reports are public. PNNL-37127 is on the PNNL publications site and NREL/TP-5500-84745 is on OSTI. The Department of Energy also maintains a public Heat Pump Database collecting field studies. All three are linked in the references below.
How this was checked
I read both reports directly rather than working from secondary summaries. Every figure quoted on this page appears in the source document, and I have named the report number and page context so you can check any of them.
The unit conversion in the section above is mine, not the studies'. NREL reports auxiliary heat as a share of compressor energy and this site models it as a share of heat delivered, so a conversion is necessary and I have shown the formula rather than just the result. If you think the conversion is wrong, the working is on the page and I would like to hear about it.
One thing I have not done: I have not independently verified PNNL's or NREL's instrumentation or analysis. I am reporting what two national laboratories measured and published under Department of Energy contract, and treating that as reliable.
What this page does not cover
Neither study is a random sample of American heat pump installations, and neither claims to be. PNNL tested prototype equipment from a specific DOE programme. NREL's 12 sites were concentrated in the Pacific Northwest and Montana. Both skew toward climates and equipment that are not representative of the whole country, and PNNL note that many of their sites had milder than normal winters.
This page also does not cover ground source heat pumps, air to water systems, or ductless installations, none of which were the subject of these studies.
References
- Mendon, V., Keene, K., Rosenberg, S., Rotondo, J. A., Nwe, K., Young, J., Wind, W., and Goetzler, B. (2025). Performance Results from DOE Cold Climate Heat Pump Challenge Field Validation. PNNL-37127. Pacific Northwest National Laboratory, prepared for the U.S. Department of Energy under Contract DE-AC05-76RL01830. January 2025.
- Winkler, J. and Ramaraj, S. (2023). Field Validation of Air-Source Heat Pumps for Cold Climates. NREL/TP-5500-84745. National Renewable Energy Laboratory. May 2023.
- U.S. Department of Energy, Heat Pump Database, field study repository.
- U.S. Environmental Protection Agency. ENERGY STAR Program Requirements for Central Air Conditioners and Heat Pumps, cold climate heat pump criteria requiring a COP above 1.75 at 5°F.
- U.S. Department of Energy, Building Technologies Office (2021). Residential Cold Climate Heat Pump Technology Challenge Specification.
- Air Conditioning Contractors of America. Manual J: Residential Load Calculation, the standard method for sizing residential heating and cooling equipment.
- U.S. Energy Information Administration (2025). Electricity use is becoming more common for residential heating. Today in Energy, citing U.S. Census Bureau American Community Survey and the Residential Energy Consumption Survey.
Related reading
Try it with a realistic backup share
The default is 10%. The field data says try 15% and 25% too, and see how much the answer moves.
Open the calculator