The Early 2022 AVM Bot Challenge
It's time to check in on the various home-valuation algorithms, to see how accurate they are these days.
This will be our first post, just to establish a baseline. We are taking a snapshot of 14 of the most recent active listings on the market in Manhattan Beach. Twelve of the 14 listings began within 2022,…
Some of Our Content is for Registered Users Only
Unlock PostIt's time to check in on the various home-valuation algorithms, to see how accurate they are these days.
This will be our first post, just to establish a baseline. We are taking a snapshot of 14 of the most recent active listings on the market in Manhattan Beach. Twelve of the 14 listings began within 2022, while two tried last year and have just restarted.
We're testing more online automated valuation models than we ever have before. This time we include AVMs from Zillow, Redfin and Realtor.com.
Over the next couple/few months, we'll see how accurate these bots were, as the subject homes actually sell.
For historical reference, we've publicly "tested" Zillow twice. (See this 2017 test and this 2014 test.) We've also been critical of Redfin's estimates here twice, because the value "history" can be fake, and because Redfin, as a brokerage, can hide estimates on its own listings when "inconvenient."
For each property, we'll share current estimates and offer some initial comments.
200 40th Street (4br/5ba, 2468 sqft.)
Start price: $5,499,999
Estimates
Zillow: $1,964,500
Redfin: $5,463,632
Realtor.com: $2,002,700 / $2,267,907
Comment:
Start with the property. This is new construction in El Porto on a somewhat larger than typical (half) lot: 1706 sqft. That asking price is hefty for a new home west of Highland, but giant for Porto.
Redfin's bots might have been cheating a bit, crediting the list price pretty heavily to be within 1% of asking.
But Zillow and Realtor seem to think this is a 20-year old house being sold 7-8 years ago. That is, they're very, very low.
There may be no winners on this one.
1508 Magnolia Avenue (5br/2ba, 1896 sqft.)
Start price: $2,200,000
Estimates
Zillow: $2,136,500
Redfin: $2,298,652
Realtor.com: $2,198,000 / $1,981,600 / $1,858,432
Comment:
A relatively conventional East Manhattan cottage benefits from all the sites featuring an estimated value very close to the list price of $2.200M. Maybe it's fairly simple - there are plenty of similar sales to go by. Here, Zillow shoots low by 3% and Redfin shoots high by 4%.
Realtor displays multiple estimates for this one, ranging from list price down to 15% under.
We kinda like the multiple-estimate approach. It's a way of admitting that some algorithms are just going to fail miserably, so the consumer should have all the options to review.
Zillow and Redfin, we hear, fall back on master algorithms to sort between multiple estimates before publishing the one number. Both approaches make some sense.
Two of these three Realtor.com estimates (-$220K and -$350K) are definitely trash, though.
1516 The Strand (4br/4ba, 4953 sqft.)
Start price: $17,500,000
Estimates
Zillow: $10,888,200
Redfin: $16,631,767
Realtor.com: $7,918,000 / $7,993,107
Comment:
This property is actually on its fourth run on the market, and perhaps could benefit from some objective advice about value.
Unfortunately, Zillow is suggesting a price 38% under asking, which only looks good compared with Realtor suggesting 50% discounts with prices that are arguably below land value.
One can make the case that the bots have no idea that this home is on The Strand, or about Strand values. (Bots don't go to the beach.)
Redfin is "only" $900K under the current start price, and, hey, the current price is already $1.400M below the first price in July 2020.
The fact that the other bots came in so far off once again suggests that Redfin's method gives a lot of credit to the list price as an indicator of inherent value.
501 5th Street (4br/6ba, 4483 sqft.)
Start price: $4,488,000
Estimates
Zillow: $6,830,400
Redfin: $4,491,847
Realtor.com: $5,604,000 / $2,161,000 / $4,128,431
Comment:
This walkstreet-adjacent early-90s fixer creates the widest disagreement so far among the bots.
Is it worth $4.5M (Redfin), $5.6M (Realtor) or $6.8M (Zillow)?
Well what do you know, Redfin's estimate is $3K over asking. Once again, closely aligned with the sellers.
Zillow, no, sorry, this one won't get bid up 52%. And Realtor, you're not much better aiming so high. (Or low... what is that $2.1M estimate even about?)
Redfin might win this one if the sellers are basically right about start price.
Otherwise, perhaps Realtor will pull this one off with Estimate #3 ($4.1M).
Quick thought: A great strategy for blackjack would be to run 3 separate hands, and be able to just play the one that's best when the dealer's cards are revealed.
341 4th Street (2br/1ba, 879 sqft.)
Start price: $3,899,000
Estimates
Zillow: $3,899,000
Redfin: $3,901,506
Realtor.com: $2,437,000 / $2,581,100 / $1,975,817
Comment:
Back to the South End walkstreets we go. Land values have been pretty consistent in the mid-to-high 3's for some time, and that's where 341 4th starts out.
Actual human appraisers often have trouble understanding walkstreet values. But neither Zillow nor Redfin missed this one! Zillow just reprints the list price. (Is that an estimate?) Redfin also cheats off the listing agent's paper, adding a rough $2,506 to make it look like their bots did the reading before class.
Do cheaters ever win?
Realtor.com sees that behavior and tries, instead, to have the bots do it all by themselves. Nice try, guys, but 341 4th is not going to sell between $1.975M-$2.437M. Good luck on the next one.
Update: This property went into escrow soon after this post went live.
1500 Walnut Avenue (3br/2ba, 1212 sqft.)
Start price: $2,399,000
Estimates
Zillow: $2,518,200
Redfin: $2,535,894
Realtor.com: $2,385,000 / $2,447,100 / $2,218,439
Comment:
Here's a pretty, pretty house in the Tree Section with a reasonable set of comps to go by.
Oh, look, the bots are all singing off the same sheet of music.
Redfin and Zillow looked at the photos, toured the house, got a great sense of this tricked-out cottage, decided people are going to fall in love and compete for it. So they both think it will sell $120-$135K over asking in the $2.5s.
Realtor is a little bit skeptical of that scenario, with two estimates a little low, and one around $50K over asking.
Disclosure: Past clients of ours owned this house many years ago. Does that require disclosure? Here it is JIC.
3216 The Strand (5br/8ba, 7454 sqft.)
Start price: $30,000,000
Estimates
Zillow: $16,480,600
Redfin: $27,191,401
Realtor.com: $11,400,900
Comment:
We've never seen a $30M public-market sale on The Strand in Manhattan Beach. Maybe that's why Realtor ($11.4M) and Zillow ($16.5M) shoot so low.
Guys! Guys! You've wasted your turns! It's The Strand! And this house is almost 3000 sqft. bigger than a typical full-size, 3-story Strand house. One of those "standard" size builds just sold for nearly $20M. (See our review of 2021 Strand sales.)
We know you bots haven't toured in person (we did, and reported on it), but, guys, can't you build in a "Strand exception" to your algorithms?
Redfin comes in 10% under asking, like the bot took a look at the list price and said, Whoa, there must be something to that number, but we have to be a little skeptical.
665 19th Street (5br/7ba, 3598 sqft.)
Start price: $4,899,000
Estimates
Zillow: $5,143,600
Redfin: $4,893,785
Realtor.com: $5,156,000 / $4,935,600 / $4,913,273
Comment:
This newer Martyrs-adjacent home is kind of by itself right now, the only option for now in the Tree Section for buyers who want this sort of product.
We have another semi-convergence here among the bots, with 5 estimates that are in sight of the list price.
Zillow and Realtor's high estimates suggest there will be a bidding war and the price will go up $250K. That didn't happen exactly last year when the home was listed, although it was in escrow for a bit.
Three estimates (two of Realtor's) are extremely close to the asking price.
42 Dover Place (3br/3ba, 2042 sqft.)
Start price: $1,950,000
Estimates
Zillow: $1,891,100
Redfin: $1,927,612
Realtor.com: $1,943,000 / $1,960,600 / $1,990,989
Comment:
This Village Plan 7 townhome generates just about the most complete agreement we've seen among the bots.
Zillow's low by $60K, Redfin's low by $23K, and Realtor is a tiny bit low, a tiny bit high and thinking, well, maybe +$40K.
We don't really think this one will sell under asking, but TBD.
The near-unanimity among the AVMs here might have something to do with the Village being a planned community with several comparable sales. A pure PPSF calculation can be more valuable in a gated community like this.
14 Monterey Court (2br/2ba, 1465sqft.)
Start price: $1,325,000
Estimates
Zillow: $1,347,200
Redfin: $1,358,435
Realtor.com: $1,441,000 / $1,383,200 / $1,453,369
Comment:
Oh look, another Village listing, another chorus of similar estimates from all the sites.
All of the estimates suggest that this Plan 1 court home is going to be a sensation and get bid up, even though the phenomenon of Plan 1s getting $1.300M or more is a brand-new development. (See our analysis here.)
The unit has nice bathroom updates, but is not, by any stretch, 100% modern from top to bottom.
Realtor, no, we don't think it is getting to $1.450M, but some of these other numbers are possible.
207 S Aviation Boulevard (2br/1ba, 906 sqft.)
Start price: $1,488,000
Estimates
Zillow: N/A
Redfin: $1,507,154
Realtor.com: $1,451,000 / $ 1,423,100 / $1,394,266
Comment:
The Zillow bots simply opted out of this one.
Most of the others hew close to list price.
True, there are a lot of factors that make this one hard to value, and hard to figure.
Very nicely refreshed inside! But only 2br and 900 sqft.
Location location location... on Aviation. Tough.
But it comes with a partly built sunroom that will more than double the house size! (You just have to finish it!)
Yeah, just copy the list price, tweak it a bit and move on.
625 33rd Street (3br/3ba, 2080 sqft.)
Start price: $2,499,000
Estimates
Zillow: $2,551,300
Redfin: $2,554,755
Realtor.com: $2,489,000 / $2,517,400 / $2,108,835
Comment:
Here's a major fixer/lot sale on a treasured street-to-alley lot in the Tree Section near the Sand Dune.
Bots are all pretty close to the asking price of $2.499M, thanks in large part to a surplus of comps from last year in the same basic area of the Trees.
Could it go up? Yes. Three out of five estimates suggest that.
2501 Walnut Avenue (4br/4ba, 3766 sqft.)
Start price: $3,099,000
Estimates
Zillow: $3,102,000
Redfin: $3,023,275
Realtor.com: $4,088,000 / $3,219,600 / $3,480,422
Comment:
This spacious early-90s Tree Section house is kind of alone in its market tier, and should be getting plenty of looks.
Move it off of Marine Ave., and you could be in the middle-3s, but no, sorry, the city won't actually let you move it.
So, while this house is where it is, it's probably going to sell close to the current list price ($2.995M), give or take.
Zillow and Redfin mostly align with the start price of $3.099M. (Pssst! They cut $104K already!)
Realtor... we can guess why they're blowing it. As we said, if not for Marine Ave., it might be worth $3.2-$3.5M, exactly like two of the estimates suggest.
Can someone teach these bots about location?
It's only, like, the most cliché (and important) thing about real estate. "Location, location, location!"
If you can't turn that into math, why even try to automate home valuations?
(That's a good point, Dave. Worry no one will get here to read it. - Ed.)
You can see the temptation to set an algorithm to start from the premise that the listing price is correct. Then you don't need as many other mitigating factors to try to add to equations.
Realtor doesn't seem to try to do that, so they even have one estimate suggesting a million-dollar overbid. Ha!
2612 Alma Avenue (3br/2ba, 1233 sqft.)
Start price: $2,999,990
Estimates
Zillow: $3,000,000
Redfin: $3,009,198
Realtor.com: $2,966,000 / $2,998,600 / $3,001,976
Comment:
The bots are in very close alignment here, like iron filings on a sheet of paper exposed to a magnet. They really believe in that list price.
But.
Guys, bots, anyone still here with us...
We don't get it. In the Edge office, we just sold the high comp for this property, 3420 Alma (3br/3ba, 1761 sqft., same approx size lot at 1667 sqft.). With multiple offers it was bid up to $2.400M.
We're all for the seller getting the most they can get, and we're always willing to admit what we don't know, but here... we'll be the minority report.
--------------------------------
Watch for updates on these properties as the sell in upcoming months.
We'll see how close the bots got, and if any one website's algorithm seems to be more reliable than the others over time.
Please see our blog disclaimer.
Listings presented above are supplied via the MLS and are brokered by a variety of agents and firms, not Dave Fratello or Edge Real Estate Agency, unless so stated with the listing. Images and links to properties above lead to a full MLS display of information, including home details, lot size, all photos, and listing broker and agent information and contact information.
Based on information from California Regional Multiple Listing Service, Inc. as of March 18th, 2024 at 8:50pm PDT. This information is for your personal, non-commercial use and may not be used for any purpose other than to identify prospective properties you may be interested in purchasing. Display of MLS data is usually deemed reliable but is NOT guaranteed accurate by the MLS. Buyers are responsible for verifying the accuracy of all information and should investigate the data themselves or retain appropriate professionals. Information from sources other than the Listing Agent may have been included in the MLS data. Unless otherwise specified in writing, Broker/Agent has not and will not verify any information obtained from other sources. The Broker/Agent providing the information contained herein may or may not have been the Listing and/or Selling Agent.