Case Study GIS Enrichment Rules as Code

Outside the Polygon, or No Polygon at All

The spatial join is the easy part. Four times the pipeline had a definite answer to give and gave it, where something weaker was called for, and once it got that right.

Andrew Maury
Andrew Maury
• October 2026

AddressIntel, our predictive real-estate intelligence project, fuses San Mateo and Santa Clara County GIS layers onto parcels: zoning, fire hazard severity, flood, fault, liquefaction, landslide, farmland, Williamson Act contracts, Coastal Zone. The mechanism is a point-in-polygon lookup against each county's ArcGIS REST endpoint, and then a derived answer about what can legally be built. The scored table holds 489,915 parcels across 16 cities.

Writing the lookup took an afternoon. Every expensive mistake since has come from the same place: the pipeline had a definite answer to give and gave it, where something weaker was called for. Four failures are below, alongside the one case the pipeline handled correctly. Two are the pure form, a confident answer standing in for one the pipeline did not have: a layer emptied upstream that read as a clean negative, and a fallback rule chosen to be conservative that was not conservative where it mattered. The third is a confident answer that was simply wrong, a polygon hit reported as a legal conclusion it does not support. The fourth turned up while this was being written, a resume mark recording a question the pipeline had never asked. The case it handled correctly is a city declining to assess a parcel and the refusal surviving the trip, which earns its place because a careless query nearly flattened it. Two further mistakes in what follows are mine rather than the pipeline's, and both are marked where they fall.

The fixes converge even where the faults differ: make the weaker answer representable, and keep it distinct from no.

A sibling piece, Scraping Adversarial Municipal Portals, covers the same shape in a different layer: a source that answers cheerfully and wrong, a rejected login returning an empty list. That one is about scraping HTML portals and detecting silent truncation. This one is about the spatial join and what a polygon lookup is entitled to conclude.

What the lookup is

There are two shapes of lookup, and the difference matters later. The listing path sends one point per property, geometryType=esriGeometryPoint with spatialRel=esriSpatialRelIntersects, and reads back either the intersecting feature's attributes or a count. The scoring path cannot afford that across half a million parcels, so it pulls each layer once for the city's bounding box and runs the point-in-polygon against parcel centroids locally. Every per-parcel figure here comes from that second shape. San Mateo County's layers sit on a self-hosted gis.smcgov.org MapServer, and two of its towns publish their own zoning from their own ArcGIS organisations because the county layer answers NO DATA ASSIGNED for them. Santa Clara County's sit on ArcGIS Online. Counting only the scoring path there are fourteen distinct hosts and twenty publishers, because a hostname is not a publisher. Nine of the fourteen are single-tenant hosts the publisher runs itself: seven cities with their own GIS servers, San Mateo County's self-hosted MapServer, and FEMA. The other five are shared services*.arcgis.com tenancies carrying eleven publishers between them: Santa Clara County, two state agencies for the geologic and fire layers, seven more cities publishing from their own ArcGIS organisations, and a regional zoning aggregator. Sharing a host does not mean sharing it widely: two of those five carry a single publisher apiece. Three more cities' zoning arrives as a local file rather than a REST layer, two of them because they publish nothing on a host of their own. The scoring module that reaches them is 2,377 lines, and most of that is neither geometry nor SQL.

The reliability work is the visible part, and it sits on the per-point listing path, not on the bulk scoring one. The Santa Clara host answers bursts with read timeouts and connection resets, so calls to it pass through a two-permit semaphore, two attempts with a jittered backoff sleep between them, and a staggered release so threads queued on the semaphore do not all fire the instant a slot frees. Above that sits a circuit breaker that opens after three consecutive call failures and skips the rest of the run. The scoring path makes larger and rarer bulk calls and carries none of this: three attempts by default, an unjittered backoff, no breaker. The breaker's own counters had to be put under a lock: without one, concurrent threads each crossed the threshold independently and logged the same trip line several times a few milliseconds apart, which is how the race was found.

None of that is the interesting failure. A timed-out call is loud. The failures below were all quiet.

A layer can be emptied without erroring

Santa Clara County republished two of its planning layers with zero rows countywide on 25 September 2026. The pipeline asked one of them, the Agricultural Preserves layer, whether each point fell inside a preserve, using returnCountOnly. An empty hosted layer answers that query perfectly well. It returns a count of zero. So the lookup did not error, did not time out, and did not trip the breaker. It simply began answering no for every parcel in the county, in the same shape as a true negative, and it would have gone on doing so indefinitely.

This is the failure mode worth naming, because a spatial join over someone else's layer is exactly where it hides. A join has three possible answers and the API gives you two. There is inside the polygon, outside the polygon, and no polygon to be inside of, and the third arrives wearing the second's clothes.

The discriminator we ended up with is a quirk of the hosting. A zero-row layer answers a bare returnCountOnly request fine, but rejects with HTTP 400 and Cannot perform query. Invalid query parameters. any request that both filters by geometry and materialises features. So when a feature fetch fails, the collector asks the layer for its unfiltered total row count. Zero means the layer really is empty upstream, and zero coverage is the correct report. Anything else, including a count it cannot obtain, leaves the layer marked unknown so the run still reads as failed: 

if _layer_row_count(url) == 0:
    print(f"    WARN {label}: layer is EMPTY upstream (0 rows countywide) "
          f"— reporting zero coverage, not a fetch failure ({e})")
    return []
print(f"    WARN {label}: fetch failed ({e}) — layer treated as unknown")
return None

An empty list and None are different values, deliberately, and every caller has to handle both.

That discriminator sits on the bulk collection path, and it is worth being exact about which path it does not sit on. The per-point lookups the listing-level enrichment uses still ask returnCountOnly one parcel at a time and still end in return c > 0, with no row-count probe. The farmland, Williamson Act and county SB 9 layers all go through that path, and every one of them would still read an emptied layer as a clean negative. The ag-preserve signal was not fixed by teaching that path the difference. It was fixed by retiring the live lookup and reading a snapshot taken before the layer was emptied, which the file itself qualifies: it was recovered from pre-emptying caches, two cities' caches had already been overwritten, and it covers the scored footprint and is not a countywide extract. The general lesson survives the specific fix, and so does the exposure.

A polygon hit is not a legal conclusion

The second failure produced wrong numbers instead of silence. SB 9, California's urban lot split law, disqualifies a site that is prime farmland or farmland of statewide importance. Santa Clara County publishes several agriculture-related layers, and for a while the pipeline folded three of them into one primeFarmland flag: the state farmland mapping, Williamson Act contracts, and agricultural preserves. A hit on any of them disqualified the parcel as prime farmland.

They are three different things, and only one of them is the statutory category. Measured against the scored table today, after the three were separated on 28 September 2026:

The three overlap only partly. All 216 Williamson parcels are also inside an ag preserve, but 16 of the 17 prime-farmland parcels are in neither other set, and the union of the three labels is 4,123 parcels today. The repository's record of the fix puts the old flag's reach at 4,104 Santa Clara parcels blocked as prime farmland, 104 of them ordinary San Jose tract houses the state farmland map classifies as urban and built-up land. That historical set is not the same set as today's union, because the old flag read the farmland layer unfiltered and today's reads only the two statutory categories. Of the 4,123, exactly 233 now carry a farmland-family disqualifier and only 17 of those carry the one the column used to give. The rest of the union is not thereby buildable: 3,157 of the 4,123 are still SB 9-blocked, almost all on fire or fault rather than on anything agricultural: 2,880 for very high fire and 707 for a fault zone, against 132 for lot size and 37 for flood.

The Williamson parcels are still blocked, under their own name. The contract forbids the housing project for the length of its rolling term, so reporting such a parcel as eligible would be wrong in the more dangerous direction: it would tell someone they can split land they legally cannot. Blocked, with its reason named accurately, is the correct report, and the reason string is what carries the accuracy.

Two layers of answer, because one is not enough

What makes this expressible is that the derivation emits two lists where a single verdict would collapse them. sb9BlockReasons holds what bars the split: the statutory site criteria, plus the Williamson Act contract, which is not one of them and bars it anyway. developmentBlockers holds everything that raises cost, risk or process friction, disqualifying or not. Several signals appear in one list and not the other, and the choice is load-bearing each time:

Writing the overlay as a Coastal Zone hit flips eligibility would be the same defect class as the farmland flag, just in the other direction. The accurate statement is narrower and less quotable: the overlay changes what the parcel costs and how long it takes, and sometimes, for specific named layers, it changes what is permitted.

One figure here cuts against the way this layer gets described, including in our own earlier notes. Of the 7,579 listing-level enrichment rows, 45 carry a Coastal Zone hit, and 42 of those are in Half Moon Bay. The other three sit in El Granada and Woodside, and across the remaining thirty cities the layer is correct and silent, with a single Burlingame row still unanswered. It is a real signal in one coastal town, and a case study that led with it would be presenting that town's overlay as a pipeline-wide feature.

The fallback that was not conservative

The second half of the pipeline is rules as code. A screening signal we care about is how built out a lot is, living area over allowable floor area, which needs each jurisdiction's floor-area rule. Those are 19 jurisdictions in a table of formulas, every entry carrying source, asOf and confidence: 9 high, 7 medium, 3 estimated, researched between 30 June and 3 September 2026, with 12 unincorporated county districts added on 27 October 2026. None of the three marked estimated is the raw default, and the labels repay reading closely. One is estimated from its city's 40 percent coverage standard times a stories factor, which lands near 0.52 effective, above the curated cluster instead of below it. One still applies the default, now capped by a table in that city's own code, with the slope half of its rule still missing. The third is not a floor-area ratio at all. The estimated tier marks what has not been finished, and what each entry does under that label still has to be read.

The default of 0.40 was chosen to be conservative. It sits at the low end of the curated cluster, so for a city near that cluster it understates allowable area, which biases against false this lot has headroom signals. That reasoning is sound and it is not a safety property, and the difference is the whole of what follows.

Four entries were curated on 3 September 2026 after an audit, all of them inheriting the default until that day, and the audit cut three ways. In one city the default held: across all 9,217 of its scored parcels it overstates only 11 percent of them and never by more than 1.20 times, which is the result the reasoning predicted. In two towns it did not. One has a ratio of 0.25 tapering to 0.15 above an acre, against which the default overstates allowable floor area by 1.60 times on a half-acre lot and 2.33 times at 200,000 square feet. The other is worse by an order of magnitude, because its caps are absolute rather than proportional: the repo records the default overstating it by 2.7 times at the minimum R-1 lot, 5.8 times at an acre and 29 times on a ten-acre parcel. In both towns the error is largest exactly where lots are biggest and a teardown call carries the most money. The fourth entry is the capped-default one described above, which is why it is still labelled estimated. An estate town is the shape a flat default gets most wrong, and it is also the shape where the signal gets used hardest.

The lesson is not that the default was badly chosen. It is that a default is an answer. Uncurated should read as unknown, not as safe, and the confidence field exists so that a caller can tell the difference instead of inheriting our optimism.

When the city itself declines to answer

The best-behaved source in this pipeline is the one that publishes its own uncertainty. Palo Alto's assessor parcel layer carries a floor-area field, and for some parcels it does not state a figure. It says instead that it cannot assess one, naming a creek, a flag lot or a right-of-way. That is a city telling you a constraint binds that it will not quantify.

1,317 of the 20,849 scored Palo Alto parcels carry the resulting far_not_assessable blocker. The arithmetic for those parcels is still run and still correct for the lot, and the flag rides alongside it to say that something we do not model is known to apply. That is a different statement from a missing value, and it is more useful than either a number or a blank.

Extracting it took two tries, and the first try is instructive. Matching the field loosely on the phrase can't assess also catches a different sentence the same layer uses, which means the district is not residential rather than that a physical constraint binds. The loose predicate pulled 7,593 parcels where the specific one pulls about 1,359, so more than four fifths of the matches were a different statement entirely. Two sentences in the same column, both reading as absence of a number, meaning unrelated things.

We ingest that layer's judgement and not its numbers, on purpose. Its floor-area values agree with our own formula for Palo Alto on 99.23 percent of 14,044 parcels, checked 28 August 2026, with the residual explained by rounding in its acreage strings. That agreement is what makes the layer trustworthy on the question, and it is also why its figures are not worth importing: they are our own arithmetic coming back to us. The non-redundant content is the refusal.

Unknown is also the resume cursor

Keeping unknown distinct from no stops being an honesty argument once the backfill runs. The overlay backfill walks already-enriched rows with a four-worker pool, and rows whose lookups time out are left undone and picked up next run. What makes that work is that a failed lookup writes None rather than False, so the row stays unmarked and the resume predicate finds it again: 

# Only mark the SCC pass done when BOTH resolved; a None in either
# means that lookup errored -> retry next run. …
if r["city"] in SCC_CITIES and prime is not None and wact is not None:
    payload["sccPrimeChecked"] = True

If a transient ArcGIS timeout had been recorded as a negative, the pipeline would be self-consistent, would converge, and would be wrong, with nothing left in the data to indicate which rows had never actually been asked. That is the theory. The column I reached for to demonstrate it does not show it. 2,217 enrichment rows carry a null corner-lot value, and I took all of them for a retry queue. Measured by development class, 1,121 are condominiums, 692 townhomes and 398 multifamily, and six are single-family homes. The first three groups are marked not-applicable and skipped by a check that runs before the resume predicate is ever evaluated, so their nulls are not outstanding work and never will be.

A further meaning hides under that one. Six more rows carry no corner-lot key at all, because a first-ever failed lookup writes nothing where a later one writes a null, and the predicate reads the field with a lookup that cannot tell an absent key from a null value. All six are single-family. Twelve rows are the queue, and finding the second six meant reading the predicate instead of the column.

A third set makes the point without rescuing it. 1,190 rows carry a null farmland value against 20 positive and 6,369 negative, and they are San Mateo County listings. The backfill records farmland as unknown for that county by design, because no state farmland source is wired for it, so no run will ever resolve those either.

So one column, three meanings. Not yet asked, which is work. Does not apply, which never was. No source wired, which is a gap in the pipeline rather than in the run. All three are stored as null, and the schema says which is which only if you go and look at a different field. A null records that a value is not known. It says nothing about whether anyone is still going to look, and the lesson of this section is that I read my own column wrong before I measured it.

What this pipeline still does not know

Three limits belong on the page, not in a footnote.

Zoning is mostly absent at the listing level. 5,487 of the 7,579 enrichment rows have a null GIS zoning value, and a further 1,124 carry the county layer's own NO DATA ASSIGNED sentinel, which the backfill treats as absent. By the pipeline's own test, 6,611 of 7,579 have no usable zoning and 968 do. Without zoning the derivation cannot reach a hard eligible and returns likely instead. That is one of two routes into the middle state and the smaller one: of the 109,132 scored parcels sitting there, 9,089 have no zone at all and 100,043 carry a zone that is simply not a single-family district, mostly multi-family and planned-development codes. Absence of a block reason means no disqualifier was found rather than that eligibility was established.

A resume cursor can mark a new question answered. Writing this piece turned up one more instance of its own subject, in the mechanism the previous section praises. The Williamson Act check became its own signal on 28 September 2026 and the backfill was updated to require it, but rows marked complete under the previous rule were never re-asked, because the predicate that would re-ask them is the one reading the stale mark. Measured on 28 October 2026, before the fix: 4,552 rows marked as having completed the farmland pass, and 3,768 of them carrying no williamsonAct key at all, with another 21 carrying it null. The per-parcel counts elsewhere on this page are unaffected, because the parcel-scoring path does not use the mark.

The fix merged on 28 October 2026, and it took two merges, which is the part worth having. The obvious version tests whether the row carries the field at all, and that version is wrong in a quieter way: a failed lookup writes a null, so the presence test would be satisfied by the error and lock it in permanently. So the completeness test mirrors the write-back condition instead, both answers present and not null.

That was not enough, because the write-back condition was itself unsafe. Making those 3,789 rows re-query opened a path that had been unreachable while they were skipped: the write stored the lookup's result without checking it, so a failed re-query replaced a resolved farmland answer with a null. Not one row at a time either, since once the circuit breaker opens every remaining call in the run returns null at once. A second change the same morning guards the two farmland writes, so a null replaces an answer only when there is no answer to lose. A third guards the coastal write just above them, which had been left alone and is the field the resume predicate keys on, so an unanswered lookup there both erased a resolved hit and re-opened the row. Three unguarded writes sat on one screen and the first repair fixed two of them, which is about the hit rate you should expect from a fix aimed at a symptom. The first fix's own test checked that a failed lookup is not marked complete and never checked what it writes, which is exactly the distance between a mark and the data it stands for. A mark records that a pass ran rather than that it answered what the pass asks today, and a field added afterwards has to be able to invalidate it. Merging the predicate did not answer anything by itself; the next scheduled run had to ask the question. It has since started doing so. Against the canonical copy the rows carrying a resolved Williamson value have gone from 763 to 962, and those still owing one from 3,768 to 3,575, which is the first evidence from production that the repair does what the test said it would.

Two copies of this database exist, and the figures above come from different ones. AddressIntel keeps a canonical copy that the application serves and a working copy on a development machine, both called the same thing. On 15 September 2026 four separate published figures were wrong from exactly that confusion, and the provenance test written in response identifies the canonical copy by requiring a marker table it carries and none of the marker tables only a development machine builds.

The two halves of this piece sit on opposite sides of that line. The parcel scoring table is named in the test as a development marker, and the canonical copy genuinely does not carry it, so every per-parcel figure here, the 489,915 and everything derived from it, exists only on the working copy and has no canonical counterpart to check against. The listing enrichment table is named in neither set, and I first read that silence as the same answer. It is not. The canonical copy carries that table too, with 7,579 rows against the working copy's 4,004, and every listing in the smaller set is present in the larger, though the two copies' stored payloads have since diverged, mostly on derived fields. The listing-level figures above are therefore measured on the canonical copy, and where an earlier draft of this piece quoted the working copy's version of one, it was quoting a 53 percent sample and calling it the population.

What to take from it

The overlay does change the answer. What took longest to get right was the smaller claim underneath it: knowing when the overlay has not answered at all.

Joining messy public data to a decision?

Rantum is a senior data science & ML studio. We turn messy, fragmented, and adversarial data into models, APIs, and products that ship, and into numbers a board can rely on.

Work with us