From Fragmented Permit Data to a Teardown-Probability Model

The Challenge

A teardown is worth knowing about early. A developer who can spot, a year or two ahead, which aging house on an outsized lot is likely to be demolished and rebuilt has a real edge over the market. The signal that precedes almost every teardown is a building permit: a demolition filing, then a new single-family-residence filing on the same parcel.

The trouble is that permits are the messiest data in real estate. On the San Francisco Peninsula alone they are filed across four different portal technologies — Accela ACA (Palo Alto, Cupertino, Santa Clara), Tyler EnerGov CSS (San Mateo, San Carlos), eTRAKiT (Hillsborough, Atherton, Woodside, Burlingame), and San Jose's open-data CKAN API. Each has its own schema, its own free-text vocabulary for what a permit is, its own history depth, and no shared identifier that ties a permit to a specific parcel or listing. The hard part is not the model. It is making the underlying data joinable and trustworthy at all.

What We Built

A permit pipeline and a teardown-probability model inside AddressIntel, our predictive real-estate intelligence platform. The pipeline normalizes those four sources into one schema, resolves every record to a parcel, and classifies free-text permit types into a fixed taxonomy. On top of that sits the part most predictive PropTech skips: a label and a backtest built so the numbers can be trusted.

One schema, joined on the parcel APN

The parcel Assessor's Parcel Number (APN) is the join key that makes everything else possible. San Jose's open-data API carries the APN natively on every permit, along with clean fields for permit type, issue and final dates, valuation, and contractor — the highest-volume market, unlocked with a free JSON call rather than a brittle scrape. For the Selenium-scraped portals, the APN is recovered from the assessor record and used to stitch permits, sales, and listings onto a single parcel timeline. Permit type descriptions, which range from "NEW SINGLE FAMILY RESIDENCE" to "Single Family (New) - New" to a bare "Demolition", are classified into a stable taxonomy (demolition, new construction, addition, remodel, roof, ADU, and so on) so a query means the same thing in every city.

A label that doesn't cheat

The first version of the label was a proxy: California's Proposition 13 caps assessed-value growth except on new construction, so a teardown-rebuild shows up as a sharp jump in a parcel's building value. It was cheap to compute and looked plausible. We cross-validated it against real permits anyway — and it failed. Of 140 proxy-flagged rebuilds, exactly one had a confirming demolition or new-build permit at the same APN. The proxy was catching ownership resets and additions, not teardowns.

So the real label comes from the permits themselves: a parcel is a teardown if it has a demolition or new-single-family permit in the forward window. Because that label is built from permits rather than assessed value, it is independent of the model's strongest feature (the land-to-improvement value ratio) — which means the backtest measures prediction, not an artifact of the label sharing arithmetic with the feature.

Point-in-time, or it means nothing

A teardown changes the structure, and that is the whole trap. Score a parcel as of 2018, but read its year_built, square footage, or listing photos today, and every one of those attributes describes the new house that replaced it. The features would silently encode the answer. This is the point-in-time leakage principle, and it disqualifies most of the obvious features from the backtest: only dated history — assessment rolls, sales, and permits, each stamped with the year it was true — is clean.

The label construction, reduced to its essentials, makes the discipline explicit:

# Point-in-time teardown label from real permits — no lookahead, no leakage
for T in (2016, 2018, 2020):                    # historical scoring dates
    for parcel in universe:
        roll = assessment_asof(parcel, T)        # land/improvement split dated <= T
        if not roll:
            continue

        permit_years = teardown_permits[parcel.apn]   # DEMO or NEW-SFR at this APN
        if any(y <= T for y in permit_years):
            continue                             # already torn down by T — drop it

        label = 1 if any(T < y <= T + 3 for y in permit_years) else 0

        # Features use ONLY dated history (assessed values, sales, permits).
        # year_built, house sqft, and photos describe the *rebuilt* house — they leak.
        observations.append((point_in_time_features(roll), label))

What the honest number turned out to be

The first run, scored on the proxy label before it was cross-validated, looked like a strong result: 17,589 parcel-date observations across three scoring dates, a 0.72% base rate, and a single heuristic (sort parcels by land-to-improvement value ratio) capturing about half of all "teardowns" in the top decile, a 5.3x lift. A hand-blended model, land value alone, and a cheapest-structure heuristic all trailed it. It looked like an undefeated baseline.

It wasn't. Once the label was rebuilt from real permits, a well-powered, non-circular test on 36,896 San Jose parcels and 266 confirmed teardowns, the ratio's edge evaporated: no signal, arguably inverted, on the low-power sample (11 positives) where it could be checked. The 5.3x had been an artifact of the proxy label sharing arithmetic with the ratio. A building-value spike triggers both, so scoring by the ratio was, in effect, scoring against a shadow of the label it was supposed to predict. The chart below is that reversal.

The one signal that held up on the clean test was plainer than any of the discarded ones: parcels with an outsized lot for the neighborhood carry a real, if modest, 2.0x lift, well-powered at 266 positives. Maintenance-permit history, hoped to be a clean negative signal (an owner investing in a structure they intend to keep), added nothing blended with lot size. Neighborhood-contagion, tenure, and assessment-growth features, tested earlier against the still-trusted proxy label, had already added nothing or diluted the ratio; none of them got a second look once the ratio itself stopped being trustworthy.

That is a result, not a failure, and a sharper one than the model simply beating its baselines would have been. A backtest that would have shipped as a "proprietary 5x teardown score" turned out, on closer inspection, to be measuring its own label. The honest output is a much smaller claim (an outsized lot is worth about 2x) and an explicit rule against overselling it. It also clarified where a real edge would have to come from if one exists at all: explicit seller intent in listing language ("value in the land," "as-is," "contractor special") remains untested and is the cheapest next thing to try. Just as important, it clarified the line between the backtest and the live product: physical features like lot coverage and structure age leak when backtesting a rebuilt parcel, but are clean and strong when scoring a house that is still standing today.

What It Shows

The hard, unglamorous discipline underneath predictive ML: unifying fragmented public records onto a single joinable spine, engineering a label that is independent of the feature it validates, and enforcing point-in-time hygiene so a backtest reports what was actually knowable at scoring time. It is the same through-line as ClearTrace — finding signal in messy, real-world data — applied to civic records instead of on-chain flow.

What It Proves to a Client

That we will tell you whether a model works before you bet a roadmap on it. This case study is itself the proof: an early 5x number looked good enough to ship, and got walked back to a smaller, defensible 2x once a cleaner test was possible. Anyone can produce a flattering backtest by leaking the future into the features; the value is in the rigor that refuses to. For a business sitting on fragmented, underused data, that means the difference between a predictive product you can defend to a customer and a demo that quietly falls apart in production. The same capability transfers to any domain with messy records and a hard-to-measure outcome: insurance, lending, marketplaces, logistics.