Exactly How Bad Are Zillow “Zestimates”? [Case Study]

So it’s really no secret, Zillow can be a bit misleading when it comes to estimating your home’s value. Even their CEO has admitted to this.



The question is: How far off are they? Is this issue specific to certain locations, price ranges, markets with lots/little inventory, or what??!

For those unfamiliar with Zillow “Zestimates®”, a Zestimate is basically an estimate of any particular home’s market value based on “millions of public and user-submitted data points.” - according to Zillow.

Their official description is the following:

"The Zestimate® home value is Zillow's estimated market value for an individual home and is calculated for about 100 million homes nationwide. It is a starting point in determining a home's value and is not an official appraisal. The Zestimate is automatically computed daily based on millions of public and user-submitted data points.”

In this case study, we sampled 3 homes in each of the 3 largest cities: Chicago, New York, Los Angeles

Map of data samples

Each sample set was found on Zillow using the following search criteria:

  • City: Chicago, New York, Los Angeles
  • Home Type: SFR
  • Status: Recently Sold
  • Price: Over $500k

Each house was chosen as #1, #2 and #3 as the first, second and third home displayed in the search results, respectively.

We also spoke to top realtors in each of these locations regarding their experiences with how their clients perceive the Zestimate for pricing in their respective housing markets.

Not many homeowners know, that Zillow has a Data Coverage and Accuracy Table on their site which has data error rates based on location.

The shocking part of these charts is that it states that for most major areas, around 90% of the listing Zestimate is “Within 20% of Sale Price” of the home.

Within 20% of the Sale Price??! - That’s a really large margin for error.


Chicago, IL

Based on the Zestimate Accuracy Table, data from Chicago IL has a Median Error of 4.7%.

Chicago Zestimate Accuracy Table

Data Set:

Chicago House Search

  • House #1: 1845 N Leavitt St, Chicago, IL 60647
    Zestimate: $621,423
    Sold: $674,500 on 11/17/16
    Error Margin: Sold +8.5% above Zestimate

  • House #2: 1311 W Wellington Ave, Chicago, IL 60657
    Zestimate: $745,484
    Sold: $675,000 on 11/14/16
    Error Margin: Sold -9.45% below Zestimate

  • House #3: 3149 N Honore St, Chicago, IL 60657
    Zestimate: $2,536,121
    Sold: $617,500 on 11/14/16
    Error Margin: Sold -75.65% below Zestimate (obviously a data glitch or input error. Dismiss this result)

New York, NY

Based on the Zestimate Accuracy Table, data from New York, NY has a Median Error of 5.0%.

New York Zestimate Accuracy Table

Data Set:

New York House Search

  • House #1: 22 Pilot Ln, Staten Island, NY 10309
    Zestimate: $437,243
    Sold: $699,900 on 11/18/16
    Error Margin: Sold +60.07% above Zestimate

  • House #2: 15 Rose Ln, Staten Island, NY 10312
    Zestimate: $510,764
    Sold: $680,000 on 11/18/16
    Error Margin: Sold +33.13% above Zestimate

  • House #3: 201 Bryson Ave, Staten Island, NY 10314
    Zestimate: $481,800
    Sold: $800,000 on 11/17/16
    Error Margin: Sold +66.04% above Zestimate

Los Angeles

Based on the Zestimate Accuracy Table, data from Los Angeles has a Median Error of 3.9%.

Los Angeles Zestimate Accuracy Table

Data Set:

Los Angeles House Search

  • House #1: 15123 Camarillo St, Sherman Oaks, CA 91403
    Zestimate: $1,933,458
    Sold: $1,838,500 on 11/18/16
    Error Margin: Sold -4.9% below Zestimate

  • House #2: 4534 Vista Superba St, Los Angeles, CA 90065
    Zestimate: $712,188
    Sold: $690,000 on 11/18/16

    Error Margin: Sold -3.11% below Zestimate

  • House #3: 17206 Vose St, Van Nuys, CA 91406
    Zestimate: $589,115
    Sold: $550,000 on 11/18/16
    Error Margin: Sold -6.63% below Zestimate

Summary

From my little data study, it seems that Los Angeles and Chicago fell pretty much within the margin of error as indicated by Zillow's "Zestimate Accuracy Table".

However, the 3 homes sampled in New York were way off; i.e., +60.07, +33.13% and +66.04% sold above Zestimate figure, respectively.

My first thought, was that it might have something to do with the algorithm handling densely populated regions.

I reached out to several real estate professionals who service this particular area and shared my findings.

This is what one of them had to say:

"With regards to the Zestimate the reason the amounts are so off is because Zillow is NOT taking into consideration that the market right now on Staten Island is very hot - it is in fact a sellers market. Literally most homes that are going on the market these days, have an offer within days, and more than most likely it ends being a bidding war between the buyers."

"Those Zestimates are based on what the value would truly have been had the market not be where it is right now. As we all know, this sellers market will only last for a certain time and will bottom out again - probably sometime in the latter half of 2017."

"When it comes to the feedback from the customers, I can say with 100% certainty that a lot of the clients have in fact complained as to why Zillow underpriced their homes and we have no clear answer for them. For we ourselves have no idea how Zillow is coming with the estimates."

Homes R Us Realty, Staten Island, NY

Another Staten Island agent had this to say:

"Zillow's estimates are never accurate, but in this case it's the Staten Island market. Staten Island has the lowest inventory it has ever had in history & actual appraisals are not keeping up with the prices."

Elizabeth Del Priore
Real Estate Associate Broker
RealEstateSINY, Staten Island, NY

The Problem

Standard real estate pricing models always take into account recent sales into the pricing a home. In fact, the amount the sold price is pretty much the end factor of deciding what a home is worth.

A home is only worth what someone is willing to pay. However, it’s apparent that the Zestimate algorithm does not put much emphasis into the actual recent sale price of the home.

If you excluded trust sales and probate sales or family-to-family type Quit Claim transfers (where the property is transferred at a considerably lower value from its market value) shouldn't the Zestimate reset to the exact amount of the property's last sold price?

Their Solution

To their credit, Zillow has given homeowners the option to edit their Zestimates to improve their accuracy.

Their official statement:

"To improve Zestimate accuracy, we allow homeowners to edit their home facts and then we incorporate this information into our Zestimate calculations.”

And just how many homeowners will edit this information if the value is higher than expected? Moreover, what percentage of homeowners actually review their Zestimate for inaccuracies? A very low percentage compared to the number of homes on the site I would suspect.

Problems arise when you allow users with emotional connections to their home decide its value. This is a very biased approach to data input.

What is the point??

Given the magnitude of this error margin, what exactly is the point of the Zestimate? Is it like a fun thing to look at?

Love Meter Dave and Kylie are only 45% compatible. Time to break up!!

Just stop it already!

It's really time to just stop referring to the Zestimate in any type of negotiation in buying or selling real estate.

Statements like, "Our Zestimate is way above the other comps in the neighborhood", or "If you look at Zillow, we are way under-priced" are just nonsense.

Unlike written advertisements, word of mouth statements like the above do not come with disclaimers such as "within a possible 20% margin of error".


GET ARTICLES LIKE THIS DELIVERED TO YOUR INBOX - EVERY WEEK

* indicates required