Extracting Tabular Data from Permit Site Plans with pdfplumber

13 min read

Extracting tabular data from permit site plans means turning the ruled schedules on a digital drawing — door schedules, wall types, plumbing fixture counts, parking tallies — into typed rows a database can query. This guide extends Parsing PDF Permit Applications with OCR and Layout Analysis, part of the broader Automated Permit Ingestion and Parsing Workflows reference, and narrows it to one job: reading tables out of the native text layer of a site-plan PDF with pdfplumber, rather than OCR-ing a scan.

The focused task is deceptively specific. A commercial site plan issued by an architect’s CAD-to-PDF export carries a crisp, selectable text layer, and buried in it are the quantity and schedule tables that a plan reviewer must reconcile against zoning and code: how many parking stalls, how many fixture units, what fire rating on each wall assembly. Miss a row and a reviewer approves a project that is short four required accessible stalls; misread a header and a fixture count lands in the wrong column and silently corrupts a compliance calculation downstream. The input here is favorable — a digital, un-scanned PDF where every glyph has exact page coordinates — so the challenge is not legibility but structure: telling pdfplumber where the table is, which lines are real cell borders versus drawing linework, and how to reshape the recovered grid into the canonical permit schema. This page walks that path end to end, then covers the failure modes that make table extraction quietly lossy in production.

Table extraction data-flow — from a site-plan PDF to schema rows via two detection strategies A left-to-right flow. A digital site-plan PDF page feeds a strategy selector configured by table_settings. The selector fans into two detection strategies: the lines strategy keys off ruled cell borders drawn in the PDF, while the text strategy keys off column and row alignment of words when no rulings exist. Both converge on a recovered cell matrix of rows by columns. The matrix then flows through a bottom rail that cleans and types each cell and maps the header row onto the canonical permit quantity schema, producing validated rows. Recovering schedule tables from a native site-plan PDF One digital PDF page → detection strategy → cell matrix → schema rows Site-plan PDF native text layer Strategy selector table_settings v/h strategy lines strategy ruled cell borders text strategy word alignment · explicit lines Cell matrix Clean & type cells strip · merge wraps coerce int/decimal Map header row → permit quantity schema alias · fuzzy match · reject unknown columns Validated schema rows typed · provenance · page coords

Step 1: Locate the Table Region Before You Extract It

Site plans are visually busy — title blocks, north arrows, hatched linework, revision clouds — and running page.extract_tables() over the whole page invites pdfplumber to hallucinate a “table” out of the drawing border or a legend box. The reliable move is to crop to the region that actually holds the schedule first, using the same fractional-zone idea the parent parser uses for form fields, then extract inside that crop.

from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
import pdfplumber


@dataclass(frozen=True, slots=True)
class TableZone:
    name: str          # e.g. "parking_schedule", "fixture_schedule"
    page_index: int    # 0-based page the schedule lives on
    # Fractions of page width/height so the map is resolution-independent.
    x0: float
    top: float
    x1: float
    bottom: float


def crop_schedule(pdf_path: Path, zone: TableZone) -> "pdfplumber.page.CroppedPage":
    """Return a cropped page bounded to one schedule's region."""
    pdf = pdfplumber.open(pdf_path)
    page = pdf.pages[zone.page_index]
    # pdfplumber bbox is absolute points: (x0, top, x1, bottom) from top-left.
    bbox = (zone.x0 * page.width, zone.top * page.height,
            zone.x1 * page.width, zone.bottom * page.height)
    return page.crop(bbox)

Cropping first does two things: it removes the drawing linework that pollutes the lines strategy, and it lets you keep a per-jurisdiction, versioned map of where each schedule sits — parking tallies top-right, wall schedule bottom-left — so a template revision updates one coordinate record rather than every extractor. Store those zone maps beside the code taxonomy the same way form zones are versioned upstream.

Step 2: Choose a Detection Strategy — lines vs text

This is the decision that determines whether extraction succeeds. pdfplumber offers two independent strategies for finding cell boundaries, set via vertical_strategy and horizontal_strategy in table_settings:

  • "lines" builds the grid from actual ruled strokes in the PDF. Use it when the schedule is drawn as a real bordered table — the common case for architect-issued schedules with visible cell borders.
  • "text" infers columns and rows from the alignment of the words themselves, ignoring graphics entirely. Use it when a “table” is really tabulated whitespace with no rulings, or when the drawing has so much stray linework that lines produces garbage.
  • "explicit" lets you hand pdfplumber the exact x/y coordinates of the separators when neither heuristic is trustworthy — the escape hatch for a stubborn recurring template.
LINES_SETTINGS: dict[str, object] = {
    "vertical_strategy": "lines",
    "horizontal_strategy": "lines",
    "snap_tolerance": 4,        # merge rules within 4pt into one border
    "join_tolerance": 4,        # bridge tiny gaps where CAD strokes don't meet
    "edge_min_length": 12,      # ignore stub ticks shorter than 12pt
    "intersection_tolerance": 4,
}

TEXT_SETTINGS: dict[str, object] = {
    "vertical_strategy": "text",
    "horizontal_strategy": "text",
    "text_x_tolerance": 2,      # words within 2pt share a column
    "text_y_tolerance": 3,
    "min_words_vertical": 3,    # a column needs >=3 stacked words to count
}


def detect_tables(page, ruled: bool) -> list[list[list[str | None]]]:
    """Extract raw string grids using the strategy that fits the schedule."""
    settings = LINES_SETTINGS if ruled else TEXT_SETTINGS
    return page.extract_tables(table_settings=settings)

The snap_tolerance and join_tolerance knobs matter more than they look: CAD exports routinely draw a cell border as several short segments that do not quite meet, and without snapping, pdfplumber sees them as separate edges and shreds one column into three. Start with the values above and widen snap_tolerance if columns are fragmenting, tighten it if adjacent columns are merging.

Step 3: Recover the Cell Grid and Repair Wrapped Text

extract_tables() hands back a list of grids, each a list of rows, each row a list of cell strings (or None for empty cells). Two structural problems are near-universal in permit schedules: a description that wraps across two physical lines becomes two rows, and merged header cells leave None holes. Repair the grid before you interpret it.

def clean_grid(grid: list[list[str | None]]) -> list[list[str]]:
    """Normalize whitespace, drop blank rows, and merge continuation rows."""
    rows: list[list[str]] = []
    for raw in grid:
        cells = [(c or "").replace("\n", " ").strip() for c in raw]
        if not any(cells):
            continue  # fully blank separator row
        # A continuation row has only its description column populated:
        # everything else empty means the prior row's text wrapped.
        populated = [i for i, c in enumerate(cells) if c]
        if rows and len(populated) == 1:
            col = populated[0]
            rows[-1][col] = f"{rows[-1][col]} {cells[col]}".strip()
            continue
        rows.append(cells)
    return rows

Merging continuation rows is the difference between “12 STANDARD” landing in the count column and a phantom row of ["", "STANDARD STALLS", "", ""] corrupting your totals. Always reconcile wrapped descriptions before type coercion, because a stray continuation row that survives will fail integer parsing and mask the real value.

Step 4: Map the Header Row onto the Permit Schema

A recovered grid is still jurisdiction-shaped — one city labels the count column QTY, another NO., another COUNT. Do not persist those raw headers; map them onto the canonical field names defined in your JSON schemas for building permits so a fixture count means the same thing regardless of which architect drew the plan. Keep the mapping declarative and reject unknown columns loudly rather than dropping them.

from pydantic import BaseModel, Field, field_validator

# Canonical field <- accepted header aliases (upper-cased, stripped).
HEADER_ALIASES: dict[str, frozenset[str]] = {
    "item": frozenset({"ITEM", "MARK", "TYPE", "TAG"}),
    "description": frozenset({"DESCRIPTION", "DESC", "NOTES"}),
    "quantity": frozenset({"QTY", "QUANTITY", "NO", "NO.", "COUNT"}),
    "unit": frozenset({"UNIT", "UOM", "UNITS"}),
}


class ScheduleRow(BaseModel):
    item: str = Field(min_length=1, max_length=64)
    description: str = Field(default="", max_length=256)
    quantity: int = Field(ge=0)
    unit: str = Field(default="", max_length=16)

    @field_validator("quantity", mode="before")
    @classmethod
    def coerce_count(cls, v: str | int) -> int:
        # Site-plan counts arrive as "12", "12 EA", or "(12)"; keep the digits.
        digits = "".join(ch for ch in str(v) if ch.isdigit())
        return int(digits) if digits else 0


def map_headers(header: list[str]) -> dict[int, str]:
    """Resolve each column index to a canonical field, or raise on the unknown."""
    resolved: dict[int, str] = {}
    for idx, raw in enumerate(header):
        key = raw.upper().strip()
        match = next((canon for canon, al in HEADER_ALIASES.items() if key in al), None)
        if match is None:
            raise ValueError(f"unmapped schedule column {idx}: {raw!r}")
        resolved[idx] = match
    return resolved

Raising on an unmapped column is intentional: a silently ignored FIRE RATING column is a compliance data-loss bug that no test will catch, whereas a hard failure routes the plan to review and forces the alias table to be extended. Every recovered row should also carry provenance — the page index and cell bounding box it came from — so a reviewer can jump straight to the source region on the drawing.

Parameter and Flag Reference

Setting Type Default Municipal-context notes
vertical_strategy str "lines" "lines" for ruled schedules; "text" when a table is whitespace-aligned with no borders.
horizontal_strategy str "lines" Set independently — many schedules have ruled columns but implied rows.
snap_tolerance int (pt) 4 Merges near-collinear CAD strokes; raise if columns fragment, lower if they merge.
join_tolerance int (pt) 4 Bridges gaps where drawn borders don’t quite meet at corners.
edge_min_length int (pt) 12 Drops stub tick marks and leader lines that aren’t cell borders.
intersection_tolerance int (pt) 4 How close a vertical and horizontal edge must be to form a cell corner.
text_x_tolerance int (pt) 2 Column grouping width for the text strategy; widen for loose CAD kerning.
min_words_vertical int 3 Minimum stacked words for text mode to call something a column.
explicit_vertical_lines list [] Hand-supplied x-coordinates when both heuristics fail on a fixed template.

Common Failure Patterns and Fixes

Drawing linework detected as phantom tables

Running lines over an uncropped site plan turns the sheet border, title block, and hatching into spurious tables. Crop to the schedule region (Step 1) and raise edge_min_length so short graphic strokes are ignored before the grid is built.

One column shredded into several

CAD exports draw a single border as multiple unjoined segments, so pdfplumber sees three edges where you expect one. Increase snap_tolerance and join_tolerance a few points at a time until the column count matches the visible header, and verify against a fixture rather than eyeballing.

Wrapped descriptions creating ghost rows

A long “36” x 80" solid core wood door" wraps to a second physical line and becomes an extra row with an empty count. Merge single-populated continuation rows into their predecessor (Step 3) before any numeric coercion runs.

Quantities polluted by units or parentheses

"12 EA", "(12)", and "12 STALLS" all break int(). Strip to digits in a mode="before" validator (Step 4) and keep the raw string in an audit field so the coercion is reversible and reviewable.

Merged header cells producing None gaps

Spanned headers leave holes that misalign every column to their right. Detect a header row whose non-null count is below the data rows’ width, forward-fill the spanned label, and fail to review if the header still cannot be resolved to the alias table.

Audit and Logging Guidance

Table extraction is lossy in ways that never raise, so log for reconstruction, not just for errors. For each schedule, record the zone name and page, the strategy used (lines/text/explicit), the recovered row and column counts, and the count of rows that failed header mapping or type coercion. Persist the raw cell strings alongside the typed ScheduleRow values so a compliance officer can prove what the drawing actually said if a permit is later disputed — the raw grid is your tamper-evident source of record, and it should be retained for the full state-mandated period like any other permit artifact. When extraction confidence is low (unmapped columns, zero rows recovered from a non-empty crop), route the plan to human review through the same disposition machinery the parent parser uses, and classify unreadable or corrupt PDFs into quarantine via the shared error handling and retry logic for ingestion pipelines rather than dropping a schedule silently. A monthly diff of extracted row counts per template is the cheapest early warning that a jurisdiction changed its schedule format.

Frequently Asked Questions

When should I use the text strategy instead of lines?

Use text when the schedule has no drawn cell borders — the values are simply aligned in columns by whitespace — or when the drawing carries so much stray linework that lines produces phantom grids even after cropping. lines is the right default for architect-issued schedules with visible ruled cells; reach for text when a crop returns an empty or nonsensical grid despite obvious tabular content on the page.

Why crop the page before calling extract_tables()?

Site plans are graphically dense, and the lines strategy cannot tell a real cell border from a sheet border or a hatched wall. Cropping to a versioned per-schedule region removes that competing linework, dramatically improves detection accuracy, and gives you a stable place to map each schedule to its meaning. It also bounds the work, which matters on the constrained VMs common in county IT.

How do I stop unit strings from breaking integer quantities?

Coerce in a Pydantic mode="before" validator that keeps only the digits, so "12 EA" and "(12)" both resolve to 12, and retain the original string in an audit field. Never int() a raw cell directly — permit schedules embed units, footnote markers, and parentheses inline, and a hard parse failure there will abort an otherwise-recoverable row.

Related in this area

Related Choosing Between pdfplumber and PyMuPDF for Permit Extraction Related Extracting Contractor License Numbers from Scanned PDFs with Tesseract