Exactly 3 months ago, we launched a new test of Zillow's automated valuations for Manhattan Beach. (See: "Testing Zillow Again.")
We took the first 15 listings of the year, noted the asking price and "Zestimate" (automated valuation) for each, and gave it all some time to play out.
So how are the robots doing?
Here's the initial chart:
ADDRESS | BD/BA | SQFT | LOT SZ | START $ | | ZESTIMATE | START DT |
1842 10th Street |
5/6 |
4707 |
7504 |
$2,500,000 |
|
$2,955,477 |
01/19/17 |
412 Highland Avenue |
3/4 |
2100 |
3334 |
$2,650,000 |
|
$2,466,703 |
01/18/17 |
409 3rd Street |
3/1 |
1186 |
2705 |
$2,500,000 |
|
$2,702,306 |
01/17/17 |
716 12th Street |
6/7 |
4475 |
3007 |
$4,690,000 |
|
$4,709,530 |
01/15/17 |
831 Duncan Place |
4/3 |
1578 |
5318 |
$2,950,000 |
|
$3,032,117 |
01/13/17 |
1315 9th Street |
6/8 |
5585 |
7396 |
$3,700,000 |
|
$3,305,559 |
01/12/17 |
2600 Grandview Avenue |
6/8 |
4410 |
2700 |
$6,975,000 |
|
$2,594,739 |
01/12/17 |
599 35th Street |
3/3 |
3078 |
5401 |
$2,495,000 |
|
$2,555,841 |
01/11/17 |
1 Tiburon Court |
3/3 |
1972 |
2285 |
$1,490,000 |
|
$1,482,094 |
01/10/17 |
1240 5th Street |
6/7 |
10009 |
0 |
$7,500,000 |
|
$7,681,486 |
01/09/17 |
2615 Laurel Avenue |
4/4 |
3101 |
4329 |
$3,199,000 |
|
$2,872,893 |
01/09/17 |
1657 Nelson Avenue |
5/6 |
4354 |
7500 |
$3,350,000 |
|
$3,322,311 |
01/05/17 |
1600 1st Street |
2/1 |
756 |
4273 |
$1,199,000 |
|
$1,211,758 |
01/05/17 |
1535 11th Street |
3/2 |
1745 |
6506 |
$1,599,000 |
|
$1,580,602 |
01/04/17 |
441 1st Street |
5/5 |
4676 |
2799 |
$4,899,999 |
|
$5,209,932 |
01/02/17 |
As we noted in that first post, with one listing, 2600 Grandview, Zillow was obviously going to fail the test, because it didn't have the data on the new construction there.
The 'bots estimated $2.6M at the time, more than $4M under asking.
It's worth noting that Zillow now does have accurate info on the listing and estimates its value at $6,432,659, putting Zillow within 2% of the current (reduced) asking price of $6.575M.
At any rate, with the initial confusion, our base for comparison is really 14 listings, not 15.
Remember, we're holding Zillow to the original Zestimate from the day these properties listed. The idea is simple: When a property first comes to market, is Zillow giving an accurate gauge of its market value, or not? No "cheating" by taking account of the public list prices and market activity and evolving the Zestimate over time!
Here's how Zillow performed. See the bottom under the listings for a summary.
CLOSED SALES (8)
409 3rd (2700 sqft. lot)
Listed: $2,500,000
Zestimate: $2,702,306
Sold: $2,550,000
Zestimate performance: HIGH by $152K (+6%)
716 12th (6br/7ba, 4475 sqft.)
Listed: $4,690,000
Zestimate: $4,709,530
Sold: $4,575,000
Zestimate performance: HIGH by $135K (+3%)
599 35th (3br/3ba, 2450 sqft.)
Listed: $2,495,000
Zestimate: $2,555,841
Sold: $2,400,000
Zestimate performance: HIGH by $156K (+6%)
1315 9th (6br/8ba 5585 sqft.)
Listed: $3,700,000
Zestimate: $3,305,559
Sold: $3,650,000
Zestimate performance: LOW by $344K (-9%)
1 Tiburon Ct. (3br/3ba, 1975 sqft.)
Listed: $1,490,000
Zestimate: $1,482,049
Sold: $1,570,000
Zestimate performance: LOW by $88K (-6%)
1240 5th (6br/7ba, 10,000 sqft.)
Listed: $7,500,000
Zestimate: $7,681,486
Sold: $7,200,000
Zestimate performance: HIGH by $481K (+6%)
1600 1st (2br/1ba, 750 sqft.)
Listed: $1,199,000
Zestimate: $1,211,758
Sold: $1,100,000
Zestimate performance: HIGH by $112K (+10%)
1535 11th (3br/2ba, 1750 sqft.)
Listed: $1,599,000
Zestimate: $1,580,602
Sold: $1,425,000
Zestimate performance: HIGH by $156K (+11%)
PENDING SALES (2)
412 Highland (3br/4ba, 2100 sqft.) began at $2.650M and ran 6+ weeks.
Zillow was thinking more like $2.466M (-7%).
It's in escrow now. We'll see.
2615 Laurel (4br/4ba, 3100 sqft.) began at $3.199M early in January. Too high. It was down $450K to $2.750M when it recently went into escrow.
Zillow was thinking $2,872,893. That was 10% under the start price, and 4% over the last listing price.
STILL ON MARKET (4)
1842 10th (5br/6ba, 4700 sqft.) began at $2.500M.
Zillow improbably estimated it much higher at $2.955M, and it's now down to $2.256M.
Due to its terrible start #, this one will go down as a very bad loss for Zillow, but only if the property sells. It's a rental now, and if it stays that way, it doesn't really count against Zillow.
831 Duncan Place (4br/3ba, 1600 sqft.) is really a lot sale in the Hill Section. It began at $2.950M this year, and is down to $2.790M.
Zillow thought this Hill Section lot was worth more in the first place: $3.032M. That's gonna be high.
1657 Nelson (5br/6ba, 4350 sqft.) is new construx that listed for $3.350M initially and had a pretty quick deal in early March after cutting to $3.149M.
Zillow liked the original price more and estimated $3,322,311, which is now $173K and 5% higher than asking. So, no, that's not happening. Zillow's high again.
441 1st (5br/5ba, 4675 sqft.) was at $4.992M part of last year and $4.899M this year when we did our first post. It's down now to $4.495M.
Zillow was thinking higher (again): $5.209M in January, 16% higher than the current asking price.
SUMMARY OF ZILLOW'S PERFORMANCE
On SOLD listings, Zillow's value estimate was high on 6 and low on 2. The margins:
High: 3%, 6%, 6%, 6%, 10%, 11%
Low: 6%, 9%
Now, remember what Zillow promised. Their formulas have supposedly improved, so that their Zestimates are within 5% almost 60% of the time (58.8%), and within 10% fully 80.4% of the time.
That's not really how it's worked out here.
Only 1 of 8 sales was within 5%.
You could say 7 of 8 were within 10%, which is 87.5%, assuming we count that one right at 10%. And the other was within 11%.
There are two pending sales, and Zillow will likely hit the "within 10%" mark again on those. Not sure if they'll be within 5% on either.
As to the 4 remaining on-market properties, Zillow is high on each one, and likely to wind up more than 10% too high on 3 of them if they sell.
So far these results are a letdown if you want to rely on the Zestimates.
Pricing within 10% of the right number is not exactly rocket science, or an example of AI outperforming human real estate brokers. That's just a nifty approximation.
We entered this year's test hoping/expecting to see something impressive, but the collection of listings we chose simply hasn't worked out like Zillow pledged.
Zillow did better in our first test, back in 2014, when their claims were less bold. Just to quote from that prior post:
"Out of 10 solds, Zillow was within 5% on 1027 Boundary, 1300 12th #D and 1336 2nd St."
"That's a 30% accuracy rate, exactly as advertised. And each of these was actually within 2% – when they're right, they're right. For the condo at 12th St., Zillow's screaming line-drive home run, they were within $492.00."
"In addition, Zillow was within 10% on 509 Dianthus and 1757 Voorhees. That brings Zillow to 50% within 10%, close to the LA County average of 58%."
We'll keep tracking, but unless/until we run another test from scratch, Zillow's not leaving a good trail this year.
Good news for human real estate agents!
-----------------
NOTE: This post was updated to address incorrect data for 1240 5th in the original post.