Checking In on Zillow's Performance in 2017

By Dave Fratello | April 25th, 2017

Exactly 3 months ago, we launched a new test of Zillow's automated valuations for Manhattan Beach. (See: "Testing Zillow Again.")

We took the first 15 listings of the year, noted the asking price and "Zestimate" (automated valuation) for each, and gave it all some time to play out.

So how are the robots doing?

Here's the initial chart:

ADDRESSBD/BASQFTLOT SZSTART $ ZESTIMATESTART DT
1842 10th Street 5/6 4707 7504 $2,500,000   $2,955,477 01/19/17
412 Highland Avenue 3/4 2100 3334 $2,650,000   $2,466,703 01/18/17
409 3rd Street 3/1 1186 2705 $2,500,000   $2,702,306 01/17/17
716 12th Street 6/7 4475 3007 $4,690,000   $4,709,530 01/15/17
831 Duncan Place 4/3 1578 5318 $2,950,000   $3,032,117 01/13/17
1315 9th Street 6/8 5585 7396 $3,700,000   $3,305,559 01/12/17
2600 Grandview Avenue 6/8 4410 2700 $6,975,000   $2,594,739 01/12/17
599 35th Street 3/3 3078 5401 $2,495,000   $2,555,841 01/11/17
1 Tiburon Court 3/3 1972 2285 $1,490,000   $1,482,094 01/10/17
1240 5th Street 6/7 10009 0 $7,500,000   $7,681,486 01/09/17
2615 Laurel Avenue 4/4 3101 4329 $3,199,000   $2,872,893 01/09/17
1657 Nelson Avenue 5/6 4354 7500 $3,350,000   $3,322,311 01/05/17
1600 1st Street 2/1 756 4273 $1,199,000   $1,211,758 01/05/17
1535 11th Street 3/2 1745 6506 $1,599,000   $1,580,602 01/04/17
441 1st Street 5/5 4676 2799 $4,899,999   $5,209,932 01/02/17

2600 Grandview Avenue Manhattan Beach CAAs we noted in that first post, with one listing, 2600 Grandview, Zillow was obviously going to fail the test, because it didn't have the data on the new construction there.

The 'bots estimated $2.6M at the time, more than $4M under asking.

It's worth noting that Zillow now does have accurate info on the listing and estimates its value at $6,432,659, putting Zillow within 2% of the current (reduced) asking price of $6.575M.

At any rate, with the initial confusion, our base for comparison is really 14 listings, not 15.

Remember, we're holding Zillow to the original Zestimate from the day these properties listed. The idea is simple: When a property first comes to market, is Zillow giving an accurate gauge of its market value, or not? No "cheating" by taking account of the public list prices and market activity and evolving the Zestimate over time!

Here's how Zillow performed. See the bottom under the listings for a summary.

CLOSED SALES (8)

409 3rd Street Manhattan Beach CA409 3rd (2700 sqft. lot)

Listed: $2,500,000

Zestimate: $2,702,306

Sold: $2,550,000

Zestimate performance: HIGH by $152K (+6%)

 

716 12th Street Manhattan Beach CA716 12th (6br/7ba, 4475 sqft.)

Listed: $4,690,000

Zestimate: $4,709,530

Sold: $4,575,000

Zestimate performance: HIGH by $135K (+3%)

 

599 35th Street Manhattan Beach CA599 35th (3br/3ba, 2450 sqft.)

Listed: $2,495,000

Zestimate: $2,555,841

Sold: $2,400,000

Zestimate performance: HIGH by $156K (+6%)

 

1315 9th Street Manhattan Beach CA1315 9th (6br/8ba 5585 sqft.)

Listed: $3,700,000

Zestimate: $3,305,559

Sold: $3,650,000

Zestimate performance: LOW by $344K (-9%)

 

1 Tiburon Court Manhattan Beach CA1 Tiburon Ct. (3br/3ba, 1975 sqft.)

Listed: $1,490,000

Zestimate: $1,482,049

Sold: $1,570,000

Zestimate performance: LOW by $88K (-6%)

 

1240 5th Street Manhattan Beach CA1240 5th (6br/7ba, 10,000 sqft.)

Listed: $7,500,000

Zestimate: $7,681,486

Sold: $7,200,000

Zestimate performance: HIGH by $481K (+6%)

 

1600 1st Street Manhattan Beach CA1600 1st (2br/1ba, 750 sqft.)

Listed: $1,199,000

Zestimate: $1,211,758

Sold: $1,100,000

Zestimate performance: HIGH by $112K (+10%)

 

1535 11th Street Manhattan Beach CA1535 11th (3br/2ba, 1750 sqft.)

Listed: $1,599,000

Zestimate: $1,580,602

Sold: $1,425,000

Zestimate performance: HIGH by $156K (+11%)

PENDING SALES (2)

412 Highland Avenue Manhattan Beach CA412 Highland (3br/4ba, 2100 sqft.) began at $2.650M and ran 6+ weeks.

Zillow was thinking more like $2.466M (-7%).

It's in escrow now. We'll see. 

2615 Laurel (4br/4ba, 3100 sqft.) began at $3.199M early in January. Too high. It was down $450K to $2.750M when it recently went into escrow.

Zillow was thinking $2,872,893. That was 10% under the start price, and 4% over the last listing price.

STILL ON MARKET (4)

1842 10th Street Manhattan Beach CA1842 10th (5br/6ba, 4700 sqft.) began at $2.500M.

Zillow improbably estimated it much higher at $2.955M, and it's now down to $2.256M.

Due to its terrible start #, this one will go down as a very bad loss for Zillow, but only if the property sells. It's a rental now, and if it stays that way, it doesn't really count against Zillow.

831 Duncan Place (4br/3ba, 1600 sqft.) is really a lot sale in the Hill Section. It began at $2.950M this year, and is down to $2.790M.

Zillow thought this Hill Section lot was worth more in the first place: $3.032M. That's gonna be high.

1657 Nelson Avenue Manhattan Beach CA1657 Nelson (5br/6ba, 4350 sqft.) is new construx that listed for $3.350M initially and had a pretty quick deal in early March after cutting to $3.149M.

Zillow liked the original price more and estimated $3,322,311, which is now $173K and 5% higher than asking. So, no, that's not happening. Zillow's high again.

441 1st (5br/5ba, 4675 sqft.) was at $4.992M part of last year and $4.899M this year when we did our first post. It's down now to $4.495M.

Zillow was thinking higher (again): $5.209M in January, 16% higher than the current asking price.

 

SUMMARY OF ZILLOW'S PERFORMANCE

On SOLD listings, Zillow's value estimate was high on 6 and low on 2. The margins:

High: 3%, 6%, 6%, 6%, 10%, 11%

Low: 6%, 9%

Now, remember what Zillow promised. Their formulas have supposedly improved, so that their Zestimates are within 5% almost 60% of the time (58.8%), and within 10% fully 80.4% of the time.

That's not really how it's worked out here.

Only 1 of 8 sales was within 5%.

You could say 7 of 8 were within 10%, which is 87.5%, assuming we count that one right at 10%. And the other was within 11%.

There are two pending sales, and Zillow will likely hit the "within 10%" mark again on those. Not sure if they'll be within 5% on either.

As to the 4 remaining on-market properties, Zillow is high on each one, and likely to wind up more than 10% too high on 3 of them if they sell.

So far these results are a letdown if you want to rely on the Zestimates.

Pricing within 10% of the right number is not exactly rocket science, or an example of AI outperforming human real estate brokers. That's just a nifty approximation.

We entered this year's test hoping/expecting to see something impressive, but the collection of listings we chose simply hasn't worked out like Zillow pledged.

Zillow did better in our first test, back in 2014, when their claims were less bold. Just to quote from that prior post:

"Out of 10 solds, Zillow was within 5% on 1027 Boundary, 1300 12th #D and 1336 2nd St."

"That's a 30% accuracy rate, exactly as advertised. And each of these was actually within 2% – when they're right, they're right. For the condo at 12th St., Zillow's screaming line-drive home run, they were within $492.00."

"In addition, Zillow was within 10% on 509 Dianthus and 1757 Voorhees. That brings Zillow to 50% within 10%, close to the LA County average of 58%."

We'll keep tracking, but unless/until we run another test from scratch, Zillow's not leaving a good trail this year.

Good news for human real estate agents!

 

-----------------

NOTE: This post was updated to address incorrect data for 1240 5th in the original post.

Please see our blog disclaimer.

Latest Listings Among
Manhattan Beach Homes For Sale