Wrapping Up Our Zillow Test

By Dave Fratello | February 21st, 2014

Late last Summer, MBC began a test of Zillow's accuracy by seeing how their Zestimates of newly listed properties worked out in the real world. (See "Testing Zillow's Estimates.")

We began with a snapshot of the 12 (then-) newest listings in mid-August 2013, then tracked progress on those over time. We're now 6 months past the start of the test, and ready to call it a wrap.

Before we ask how, exactly, Zillow performed, let's recall how well Zillow says they expect to perform.

As you can see on this page, Zillow's engineers publish their own statistics on the accuracy of Zestimates. In the LA Metro area, for Q3 2013, the Zestimate at the time of a public listing's launch was within 5% of the eventual closed sale price 32% of the time.

Rephrase that: One-third of the time the Zestimate is reasonably accurate.

So how'd they do in MB? Here's the chart.

1202 Tennyson Street #1 3/3 1755 13120 $880,000     $1,231,970 09/30/13
1509 Magnolia Avenue 3/2 1342 4867 $1,335,000     $1,079,017 10/03/13
1300 12th Street #d 2/3 1130 11928 $683,000     $683,492 10/03/13
594 33rd Street 6/6 3500 5023 $2,848,000     $2,313,612 10/04/13
1027 Boundary Place 4/4 3263 4212 $1,770,000     $1,811,135 10/04/13
3201 Palm Avenue 3/3 2469 4637 quit     $1,765,274 n/a
509 North Dianthus Street 4/3 2157 3251 $1,620,000     $1,746,052 09/19/13
1757 Voorhees Avenue 4/3 2460 7498 $1,340,000     $1,479,691 11/22/13
112 18th Street 6/7 4237 2693 $4,800,000     $4,270,400 11/12/13
25 Sausalito Circle 3/3 2050 3915 $1,525,000     $1,272,286 11/06/13
1336 2nd Street 3/2 1223 5001 $1,027,000     $1,045,625 11/27/13
1806 Marine Avenue 3/2 1147 5076 quit     $945,996 n/a

Out of 10 solds, Zillow was within 5% on 1027 Boundary, 1300 12th #D and 1336 2nd St.

That's a 30% accuracy rate, exactly as advertised. And each of these was actually within 2% – when they're right, they're right. For the condo at 12th St. (pictured), Zillow's screaming line-drive home run, they were within $492.00.

In addition, Zillow was within 10% on 509 Dianthus and 1757 Voorhees. That brings Zillow to 50% within 10%, close to the LA County average of 58%.

Zillow failed spectacularly on three of these listings: 1202 Tennyson, 1509 Magnolia and 594 33rd. That was due to poor information, but we fully expected that, and wrote those off in our first post. (On Tennyson, Zillow had mega lot size for the whole condo complex, not just the unit; on Magnolia, the robots didn't know the home was completely remodeled and, on 33rd, they didn't know it was new construction.)

So one lesson there would be, if it is clear that Zillow is missing an important detail, you should ignore the Zestimate completely. (If we had actually dropped these from our data set at the beginning, due to these obvious input errors, Zillow's success rates would be far higher.)

Finally, Zillow wasn't very close on 25 Sausalito Circle (super remodeled) or 112 18th St. (pictured), though both pricing errors were "understandable," as we explained in our first post.

One of the three big "understandable" gaps in our initial post was between the sellers and Zillow on 1027 Boundary, where the sellers began at $1.980M but Zillow was thinking more like $1.8M. Turns out both were high, and the $1.770M closed price was 11% under the start and just 2% under the Zestimate.

A little pat on the virtual back is in order for the robots on that one, which we had said was "very hard to value, given the unusual style, location and changes in the market since last year [when the home had last traded]."

So, can you trust the Zestimate on a home? It depends. If they didn't obviously botch the basic details, there's a good chance the algorithms are reasonably close – and some chance they're dead on. That's a starting point, anyway, before you look to other sources.

Please see our blog disclaimer.

Latest Listings Among
Manhattan Beach Homes For Sale