Testing Zillow's Estimates

By Dave Fratello | August 12th, 2013

Zillow... people love to hate their estimates.

And want to believe them. Sometimes.

But all that really matters is: How accurate are the so-called "Zestimates" in practice?

Turns out, Zillow tells you. At least by region of the USA.

As you can see on this page, in their effort to be transparent, Zillow's engineers publish their own statistics on the accuracy of Zestimates.

In the LA Metro area, for Q1 2013, the Zestimate at the time of a public listing's launch was within 5% of the closed sale price... wait for it... 31% of the time.

Not bad for a robot, but you wouldn't want this one flying your plane.

If you were buying a $1.5M house and relying on the Zestimate only, you'd have almost a 1 in 3 chance that Zillow was putting you within $75K of the right price – high or low. Kind of a big margin of error.

Zillow also reports that they're "within 10%" about 56% of the time (that includes the first 31% within 5%) and "within 20%" about 83% of the time.

Well, at least we know they are more likely than not to be within 10%. High or low. Achem.

Now, Zillow does not tell anyone to rely on these valuations. By their own description, a Zestimate is "Zillow's estimated market value. It is not an appraisal. Use it as a starting point to determine a home's value." A starting point.

What we're interested in is seeing how Zestimates perform in Manhattan Beach.

We'll run a little experiment, starting now, by taking the most recent 12 listings to hit the market, and tracking their sale prices over time. We'll see how they compare with the Zestimates from today.

Here's the list starting now, with the Zestimates included in the grid:

1202 Tennyson Street #1 3/3 1755 13120 $870,000     $1,231,970 08/12/13
1509 Magnolia Avenue 3/2 1342 4867 $1,349,000     $1,079,017 08/08/13
1300 12th Street #d 2/3 1130 11928 $699,000     $683,492 08/06/13
594 33rd Street 6/6 3500 5023 $2,899,000     $2,313,612 08/05/13
1027 Boundary Place 4/4 3263 4212 $1,980,000     $1,811,135 08/01/13
3201 Palm Avenue 3/3 2469 4637 $1,775,000     $1,765,274 08/01/13
509 North Dianthus Street 4/3 2157 3251 $1,625,000     $1,746,052 08/01/13
1757 Voorhees Avenue 4/3 2460 7498 $1,425,000     $1,479,691 07/30/13
112 18th Street 6/7 4237 2693 $5,195,000     $4,270,400 07/29/13
25 Sausalito Circle 3/3 2050 3915 $1,685,000     $1,272,286 07/27/13
1336 2nd Street 3/2 1223 5001 $1,100,000     $1,045,625 07/25/13
1806 Marine Avenue 3/2 1147 5076 $899,000     $945,996 07/22/13

That's an interesting start.

Zillow is clearly going to blow it completely on 1202 Tennyson (pictured). They're almost $500K high, probably not realizing that's a condo and giving too much credit for the listed lot size. 

And 1509 Magnolia has been completely redone and just hit the market looking super. (See our video walkthru of 1509 Magnolia.) Zillow doesn't know that, either.

Similarly, the computers also seem not so recognize 594 33rd as new construction that will likely near that $2.9M asking price. It is not selling for $2.3M.

So we've taken 3 of the 12 off the board completely. The Zestimates will fail.

Now, on 3 more, the Zestimate is very close to the list price, but under:

In a few more cases, the Zestimates are higher than the list prices. Homeowners, did you underprice?

In most of those 6 cases above, the Zestimate is already within 5% of the asking price. The numbers are very similar to what real estate pros and their clients (a.k.a., actual people) came up with in setting an asking price. Impressive.

And then there are the final 3, all with understandable, big differences. 

25 Sausalito Circle (3br/3ba, 2050 sq. ft.) is a very stylishly (and uniquely) redone Manhattan Village condo. Most of the comparably sized units seem to sell around $1.3M, roughly, and the most recent comp (56 Sausalito Circle East) sold in April for $1.375M.

Zillow hasn't seen the remodel and says $1.272M, but the sellers request $1.685M.

So Zillow is 24% below the sellers, but we're not sure where the market will be on that one.

At 1027 Boundary Place the difference is 8.5%, with Zillow saying $1.811M and the sellers asking $1.980M.

The sellers only paid $1.475M about a year ago before some remodeling, so really Zillow is giving them a nice boost. It is a property that is very hard to value, given the unusual style, location and changes in the market since last year. This is one to watch.

Finally we have 112 18th Street, an 80s build with some modern-style remodeling and situated in a great Strand-adjacent location on one of the city's great streets.

Zillow is low by $925K on a $5.195M list price, or 18% off the sellers' request. But this difference is one we'd call "understandable" for the simple reason that a very comparably sized and located property at 112 20th St. just sold for $4.2M in January, and nearby 128 20th also hit $4.2M in mid-July.

So if Zillow thinks a similar property in the same basic area is worth about $4.2M (actually more, $4.270M), who can blame the computer? We'll see where the market goes on that one.

It's funny to say, but this sample shows promise.

Half of the Zestimates look pretty good. A few more might be prescient, but if they are way off, it's pretty easy to understand why.

We'll bet right away on Zillow doing better in MB than their 31% success rate for LA generally, of being within 5% of final sale prices.

We recently attended a conference where Zillow gave an overview of their Zestimate process. There are multiple, multiple inputs and at least 5 different methods of calculation used for each property. A central brain tries to "learn" which algorithms are most reliable for a given neighborhood and gives greater weight to those. When you see a Zestimate float around, it's responding to new data, but also to the weighting of different algorithms.

Message: They are really, really trying.

Not even Zillow thinks their Big Blue will supplant working real estate agents or appraisers, who know their markets best and can account for things the computer misses.

Let's see how well they do on this batch of MB listings. We'll have more updates as sales close.


Disclosure: Dave pays for ads on Zillow. This post was not requested, influenced or reviewed by Zillow, and confers no benefit from Zillow to Dave.

Please see our blog disclaimer.

Latest Listings Among
Manhattan Beach Homes For Sale