Corrections and data-quality log
Every figure on this site is derived from documents governments published themselves, and some of it is machine-extracted. That means we get things wrong. When we do, it goes here, with the wrong value, the corrected value, who was affected, and what changed in the pipeline so it does not recur.
This log exists because a civic-transparency dataset gets one chance with a reader who looks up their own town. If you have used a number from this site and it appears below, assume it changed.
Entries are newest first. Anything that went out to someone directly, or was published externally, is marked NOTIFIED once the correction has reached them.
2026-08-17 — Surveillance procurement total was wrong in outbound email
Was: "Surveillance procurement is $99.7M identified" (emailed 2026-08-13). Now: ALPR and Flock specifically $120,675,844. All surveillance and police technology $828,468,503 across 2,939 line items, 952 governments, 143 vendors, 50 states.
The $99.7M matched neither figure and could not be reproduced. It appears to have been a half-finished version of the ALPR subset described as the whole category. The corrected figures come from the site's own gated query, which keeps one row per duplicate group, counts only stated amounts (not "not to exceed" ceilings or proposed items), drops anomalies and suspect city attribution, and excludes two-way radios, records-management software and fingerprint biometrics.
Both remain floors — the data cannot always distinguish an amendment from an original award. Reproduce at /mytown-research/surveillance_defensible_total.
Note for context: two-thirds of the all-in total ($549.5M) is a single vendor, Axon.
Affected: one external recipient of a direct data request. NOTIFIED. Prior related correction (2026-07-29): an external audit found a published surveillance total of $2.18B that was 2.9x too high; the defensible figure was $752M. That produced claim_check.py and the ten tests now gating every aggregate.
2026-08-16 — Research site downloads silently truncated at 2,000 rows
Was: any CSV export from research.theboringparts.com stopped at 2,000 rows with no indication in the file. Now: download links force _stream=1, which returns the complete result set. A warning naming the cap sits on the landing page.
surveillance_spending is 5,316 rows, so an export delivered 38% of it. 61 of 73 tables are larger than 2,000 rows, including cf_transactions at 45.9 million. Any share, rate or per-capita figure computed from a truncated download was wrong, and nothing in the CSV said so. JSON responses carry a "truncated" flag; CSV does not.
max_returned_rows remains 2,000 deliberately — the server buffers results in memory and peaked at 30.4 GB — but truncation is now visible and avoidable.
Affected: anyone who exported a table before 2026-08-16. Everyone we had shared the link with directly has been NOTIFIED.
2026-08-16 — Public site had not deployed for 10 days
Was: the site last deployed 2026-08-05. Forty-one consecutive hourly builds failed. Now: builds complete (31,562 pages) and deploy normally.
One meeting row carried a month value of 13. build_site.py parsed it with date() inside the per-city loop, raised ValueError, and killed the entire build — no /coverage, /data, /methodology or sitemap. run_hourly.sh correctly refused to deploy a broken build, so the site simply froze. The date parser now degrades instead of raising, and the calendar button is omitted for an unparseable date.
meetings.meeting_date is TEXT and portals put arbitrary values in it: eight distinct malformed values exist, including 2018-17-04, 2013-65-94 and 2014-04-96, plus 52 empty strings. Presentation is now hardened; the ingest-side bound is not yet.
Affected: every page on the public site was 10 days stale.
2026-08-16 — Coverage claims overstated
Measured against the data and corrected in the site copy:
| Claim | Actual |
|---|---|
| "We mirror every agenda PDF we process" | 91.9% (1,714,860 of 1,751,168) |
| "Every city page links each agenda to its official source" | 89% |
| "Every city council, board and commission meeting in N cities" | not every meeting; not a representative sample |
| "N have posted a meeting in the last ~60 days" | the query measured since Jan 1, not 60 days |
/methodology reported 6,757 jurisdictions while the homepage reported 6,589 — one counted rows, the other counted distinct slugs. Both now count distinct slugs.
City pages with no upcoming meetings now carry an explicit freshness note stating that a gap is a statement about our record, not about whether the government met.
2026-08-16 — Municipality rows were duplicate-inflated
Was: 8,470 municipality rows, including 467 duplicate-slug groups and 649 US rows carrying a full state name instead of the two-letter code. Now: 7,994 rows, zero duplicate slugs, zero unnormalised states, and a working unique constraint.
Any count of "municipalities we cover" published before this date was inflated by roughly 5.8%. Published counts now dedupe by slug.
Root cause, which is broader than this table: the database's ICU collation moved from 153.136 to 153.120 while pg_database.datcollversion was NULL, so Postgres never warned. Text indexes built under the old ordering cannot find an existing equal key, so INSERT ... ON CONFLICT searched the wrong part of the tree, found no conflict, and inserted a duplicate. The constraint was consulted and gave the wrong answer. Bigint primary keys are unaffected, which is the fingerprint of a collation change rather than disk corruption.
Sixty of 324 btree indexes failed verification. Forty-seven have been rebuilt. Thirteen cannot be rebuilt until their existing duplicates are resolved, and those tables are still duplicate-inflated as of this entry:
| Table | Duplicate rows | Inflation |
|---|---|---|
election_results | 17,264 | 19.4% |
food_inspections | 2,634,923 | 15.1% |
ma_election_results | 6,341 | 6.8% |
people_aliases | 1,283 | 5.6% |
official_disclosures | 3,698 | 0.3% |
meetings | 6,548 | 0.3% |
cf_transactions | 4,742 | 0.01% |
lobbying_registrations | 734 | 0.02% |
De-duplicate before aggregating from any table in that list. The duplicates are re-ingests of the same record differing only in id/created_at/updated_at, not conflicting data.
2026-08-16 — Chicago appeared to have stopped meeting; 238 cities were failing silently
Was: Chicago showed 0 upcoming meetings and nothing after 2026-07-22. Now: 1,462 meetings, 14 upcoming, records through 2026-09-23, 756 agenda documents.
Three separate defects. Chicago's Legistar tenant is decommissioned (LegistarConnectionString setting is not set up in InSite for client: chicago) and the working ELMS crawler was in no scheduler. The crawler also discarded upcoming meetings and never stored agenda URLs.
The third is the one that mattered beyond Chicago: mytown.py fetch() caught per-municipality exceptions but never called conn.rollback(). On Postgres one failed statement aborts the transaction, so every alphabetically later municipality failed with "current transaction is aborted" — 238 collateral failures per hourly run, while the log reported a healthy-looking "1,948 meetings across 311 municipalities". Failures went from 246/311 to 9/311; meetings fetched per run went from 1,948 to 10,310.
Genuine publisher-side gaps are now recorded in pull_bounds rather than being invisible, including 26 municipalities whose portals answer normally but have published nothing in 90 days.
Standing limits, not corrections
These are known and disclosed rather than fixed, because they are properties of the sources:
source document. An internal audit of numeric grounding found an error rate well under 1%, but that audit tested dollar figures and vote tallies only. A separate 50-record external spot check found material discrepancies in dimensions that audit never tested — incorrect people, incorrect dates, and one instance of a discussion item described as an anticipated vote. Most sampled records contained no material discrepancy. The honest summary is recurring data-quality problems in specific dimensions, not wholesale fabrication.
of date claims, hand-verified.
it may not publish machine-readable agendas, or we may not have reached it.
time-series claims spanning that boundary need normalising for coverage growth.
is on /methodology.