FAF5 as a Decision Dataset, Making 5.7 Million Flows Queryable
Pull-quote: “The federal government publishes the most complete picture of American freight in existence. It publishes it in six vocabularies, five geographies, and four unit systems, and the reconciliation is the product.”
Why this matters
Every corridor study, port investment case, and state freight plan in the country ultimately leans on the same public foundation: the Freight Analysis Framework, version 5, built by federal transportation agencies on the Commodity Flow Survey and its supplementary sources. FAF5 is genuinely remarkable, roughly 5.7 million flow records describing tonnage, value, and mode for commodity movements between more than 130 domestic regions across 42 commodity classes, with both historical years and forecast horizons. It is also, in its raw form, a set of tables that answers exactly one shape of question: how much of commodity X moved from zone A to zone B by mode M in year Y. Decisions need more shapes than that, and getting them is a data engineering problem the spreadsheet era never solved.
One picture, six vocabularies
FAF5 describes flows between zones. It does not say which highway carried them, what the railroad actually waybilled, what moved through the lock, or what the movement emitted. Those answers live in sibling federal datasets, each with its own geography, taxonomy, and units:
| Source | What it contributes | Reconciliation friction |
|---|---|---|
| FAF5 | Zone-to-zone flows by commodity and mode | Aggregate zones, survey-based, forecast years |
| T-100 | Air cargo segment volumes | Airport pairs, not zones; carrier reporting conventions |
| STB waybill | Rail movement detail | Sampled, masked, its own commodity coding |
| USACE | Waterborne commerce through ports and locks | Port and waterway geography; separate commodity scheme |
| NHFN | The designated highway freight network itself | Geometry and designation, no flows; the assignment target |
| EPA | Emissions factors by mode and activity | Activity units that match none of the above directly |
Every pair of sources disagrees somewhere: zone versus county versus airport versus waterway geographies, commodity taxonomies that split and lump differently, short tons against ton-miles against dollar values, suppression and masking rules that hide exactly the cells a specific question needs. None of this is anyone’s error. Each dataset was built for its own statutory purpose. The friction only appears when you ask one question across all of them, which is what every real freight decision does.
The reconciliation layer
FAF5 T-100 STB USACE NHFN EPA
│ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼
┌────────────────────────────────────────────────────┐
│ Harmonization: geography crosswalks, commodity │
│ mapping, unit normalization, suppression handling │
├────────────────────────────────────────────────────┤
│ Reconciliation: cross-source totals compared, │
│ conflicts resolved by documented precedence │
├────────────────────────────────────────────────────┤
│ Validation: back-testing against published │
│ U.S. freight benchmarks │
└───────────────────────┬────────────────────────────┘
▼
One queryable model, sub-second explorationThis is the layer that turns published tables into a decision dataset: a unified federal freight data model that fuses and reconciles FAF, T100, STB, USACE, NHFN, and EPA sources into one queryable whole, paired with a calibration and validation framework that back-tests outputs against published U.S. freight benchmarks. Everything underneath is public federal data, which matters twice: the evidence base is citable in a public document, and a system built on it can run in government cloud environments without licensing entanglements.
What queryable actually buys
The test of the reconciliation is the questions that become answerable in one step: rank the corridors carrying a commodity class and attribute the tonnage to the designated network; compare a zone’s outbound flows against what the mode-specific records support; hold one commodity constant across geography, mode, and year without manually rebuilding crosswalks per study. On top of the unified model, a query layer can answer such questions with cited, structured outputs at interactive speed across all 5.7 million records. The query layer is the visible feature. The reconciliation is why its answers survive scrutiny.
Closing
FAF5 is the decision dataset American freight already paid for. What stands between the published tables and a defensible decision is unglamorous engineering: crosswalks, commodity mappings, unit discipline, conflict precedence, and validation against benchmarks. Do that work once, properly, and months of per-study data assembly collapse into queries. Skip it, and every analysis quietly builds its own incompatible version. The reconciliation is the product.