# TT-008 · Port ledger — method

**Grade: REPORTED.** Counted by US customs from declarations filed under penalty. Not observed
by us, not modelled.

## Why it exists

Two reasons. The second was added on 2026-07-30.

**To check TT-001.** AIS infers that cargo moved because a ship's reported draught fell. Nothing
had ever tested that inference. This counts declared mass at the same gateways, so the proxy can
be confirmed or killed. It is also the only free source that says where the cargo came from.

**To be a mirror, not a query.** The observatory must be able to defend a published figure years
later. Reading a live API cannot do that, for four reasons in descending order of importance:

1. **Census revises prior months.** Read live and a restatement is invisible. Mirrored, with the
   prior value preserved, a revision becomes an *observation about the source itself*.
2. **Reproducibility.** If the API answers differently in 2028 we cannot re-argue a claim.
3. **Availability.** The API goes dark, and the all-ports HS2 request does not complete at all.
4. **Speed.** Interactive lab work against those latencies is unusable.

## Grain

**One row = direction × month × US port × partner country × commodity code.**

There is **no vessel dimension** — customs records a shipment, not a ship. That is precisely why
it complements AIS instead of duplicating it, and why the two can only ever be joined on **port
and time**, never on vessel.

`comm_level='TOTAL'` rows are the port × country total; `HS2` rows are the commodity breakdown
beneath them. **Do not sum across levels or you will double count.**

## Fields

Every measure Census publishes at this grain is collected, for both directions.

| field | what it is |
|---|---|
| `container_weight_kg` | **The comparison field.** Declared mass of containerised cargo — what draught change is a proxy for. |
| `container_value_usd` | Customs value of the same. |
| `vessel_weight_kg` / `vessel_value_usd` | All vessel traffic including bulk and breakbulk. The gap against `container_*` is a containerisation rate nobody publishes — and it is how bulk contamination in the AIS fleet gets bounded. |
| `air_weight_kg` / `air_value_usd` | Air freight, weight and value. Air costs roughly ten times ocean, so air share is the market pricing ocean reliability. |
| `all_modes_value_usd` | `GEN_VAL_MO` on imports, `ALL_VAL_MO` on exports — the two datasets differ here. |
| `census_last_update` | Census's own revision stamp. **Returns `0` on every one of 126,313 rows.** See below. |
| `retrieved_at` | When *we* fetched this exact figure. The only usable revision clock. |

## Revisions — why detection cannot be delegated

Census revises prior months, sometimes materially. Its `LAST_UPDATE` field is supposed to say
when, and **it does not work**: verified 2026-07-30, all 126,313 rows return `0`.

So the source cannot tell us when it changed its mind. **We detect it by comparison instead.**
Every incoming figure is checked against what we hold; when a measure moves, the prior value is
written to `port_trade_revisions` *before* the update, so the original is recoverable rather than
merely flagged.

Each archived change is classified, and the distinction is not cosmetic:

| kind | meaning |
|---|---|
| `revision` | value → different value. **Census genuinely restated a figure.** |
| `withdrawn` | value → NULL. Census removed something it had published. |
| `backfill` | NULL → value. **We did not hold the figure before. Not a Census change.** |

**Why `backfill` exists.** Migration 001 added `air_weight_kg`, a measure never previously
collected, so every pre-existing row held NULL for it. The first sync saw NULL → value on 4,401
rows and recorded them all as revisions. **None were.** Left uncorrected, the one table whose
purpose is to prove when a source changed its mind would have begun life with 4,401 fabricated
entries. Migration 002 reclassified all 4,401 as `backfill` from their own recorded evidence.

**Only `revision` and `withdrawn` may ever be cited as the source having changed.**

## What "port" means — verified, not assumed

`PORT` is the **port of unlading**, not the customs district where cargo cleared. Tested two ways
on 2026-07-30:

- **Landlocked ports report exactly zero vessel weight** — Dallas, Denver, St Louis and Pembina
  all 0. If in-bond cargo were credited where it clears, Dallas would be enormous. Chicago and
  Detroit show small amounts and both are genuine Great Lakes vessel ports.
- **Magnitude reconciles.** Los Angeles containerised imports, May 2026: 3,665,705 t against
  POLA's own published 530,558 loaded-import TEU. That is **6.9 tonnes per TEU**, inside the
  plausible 6–12 t band.

So the in-bond concern does not apply. What genuinely stays invisible is narrower: **foreign
in-transit cargo** (discharged in the US, destined for Mexico or Canada, never entered) and
**empty containers**, which carry no customs value and are never declared.

## Port mapping, and a trap in it

Census reports **LA and Long Beach, New York and Newark, and Seattle and Tacoma separately**,
while TT-001 watches some pairs as one coverage zone. They must be **summed to the AIS zone**
before comparison — comparing one Census port against a two-port AIS zone understates by roughly
half.

Note: `us_seattle_tacoma` is a single AIS zone spanning two ports 35 km apart. That is a
measurement error to fix on the AIS side, not to compensate for here.

## Coverage and cadence

- **11 Census port codes** covering the eight AIS gateways
- **All countries** — 233 on imports, 242 on exports. *Superseded the previous seven-country
  India filter on 2026-07-30.*
- **Both directions.** The export half was missing entirely before that date.
- **TOTAL and HS2** commodity levels. HS4/HS6 exist and are fetched on demand only — HS4 alone
  multiplies row count by roughly six.
- **Backfilled from 2024-01.** Roughly 50,000 rows per month per direction at HS2; about 138 MB
  per year.
- Census publishes roughly **five weeks after month end**, so the newest month is always
  incomplete. `sync --months=N` deliberately re-fetches recent months rather than assuming a
  figure is final — that re-fetch is how revisions get caught.
- Requests are **paginated by port**. The all-ports HS2 request times out and never completes.

## Country groupings — the overcount that corrupted every absolute figure

Census returns **three overlapping layers** in the same response, distinguished only by
`SUMMARY_LVL`, a field the mirror did not collect until 2026-07-30:

| layer | what it is |
|---|---|
| `country = '-'` | TOTAL FOR ALL COUNTRIES. Marked **DET**. The authoritative total. |
| `SUMMARY_LVL='CGP'` | groupings: **APEC, ASEAN, OECD, ASIA, PACIFIC RIM**, and the continental `1XXX`–`7XXX` codes |
| `SUMMARY_LVL='DET'` | individual countries |

**The groupings overlap each other heavily** — China sits inside APEC, Pacific Rim, Asia
and ASEAN's neighbours — so summing every row does not double count, it over-counts by a
variable factor. Los Angeles, May 2026 containerised imports:

```
grand total row      3,666 kt      <- the truth
continental groups   3,666 kt      <- a correct partition of it
"individual" codes  11,588 kt      <- 3.2x, because APEC/OECD/Pacific Rim overlap
```

**Two rules follow, and the second is the one that catches people:**

1. For a port-level figure, read the `country = '-'` row and nothing else.
2. **Filtering on `SUMMARY_LVL='DET'` is NOT sufficient.** Census marks the grand total
   DET as well, so a DET-only sum adds the total to its own members.

What this corrupted: per-port containerisation rates were out by up to **7.9 percentage
points** (New York read 77.7% against a true 69.8%). What it did *not* corrupt: the
port-of-unlading verification above, which used the `-` row and stands.

## Commodity depth — which level, and why

| level | rows per port-month, imports | exports |
|---|---|---|
| TOTAL | 182 | 197 |
| **HS2** | 5,350 | 4,832 |
| HS4 | 26,541 | 21,756 |
| HS6 | 67,649 | 45,172 |
| HS10 | request fails | not offered |

Across 11 ports, both directions, 197 months: HS2 ≈ 20M rows (~2.4 GB), HS4 ≈ 105M
(~12.6 GB), **HS6 ≈ 244M (~29 GB) — more than the free disk, and roughly two weeks of
fetching.**

So the standing archive is **TOTAL + HS2**, and the depth decision is made per analysis:

- **Containerisation rate: HS2 is correct.** Whether cargo fits in a box is a property
  of physical form and is consistent within a chapter — mineral fuel 1%, meat and
  cereals near 100%. Depth would add rows, not information.
- **Unit value / value density: HS2 is WRONG and must not be used.** A chapter is far
  too heterogeneous — Chapter 85 spans transformers at a few dollars a kilo and
  semiconductors at a few thousand. At HS2 a change in $/kg is overwhelmingly product
  mix, not price or quality. Fetch **HS6 on demand** for the specific cells a claim is
  about, never as a standing archive.
- **HS6 is also the deepest COMMON level.** Imports are published to HS10, exports only
  to HS6, so any import-versus-export comparison is capped there regardless of budget.

## Derived series: the containerisation rate

```
containerisation rate = containerised vessel weight / ALL vessel weight
```

Both fields come from the same row, so the ratio is internally consistent and no
cross-source join is involved. Computed at port × direction × month, and at HS2
commodity level pooled across the gateways.

**Why it is worth publishing.** Anyone could compute it from the free API; nobody
publishes it as a series. It answers a question operators actually have — which trades
move in boxes, where, and how fast that is changing. And it is the direct bound on bulk
contamination in TT-009: at a gateway that is 38% containerised, an AIS filter that
cannot separate a container ship from a bulk carrier is measuring a mixed fleet.

Observed May 2026, imports: Oakland 100%, Norfolk 86%, Savannah 85%, Los Angeles 79%,
Newark 57%, Houston 38%. By commodity: meat, cereals and cocoa near 100%; iron and steel
19%; mineral fuel 1%.

### Four traps in it

**Exports are UNDERSTATED and are a lower bound.** Census states the containerised flag
is optional for exporters and that "some filers may omit this information", so the
export numerator is incomplete by an unmeasured amount. Never compare an export rate
directly against an import rate without saying so. The apparent import/export scissor —
Norfolk importing 86% in boxes and exporting 24% — is real in direction but its
magnitude is not trustworthy.

**A zero denominator yields NULL, never zero.** A port-month with no waterborne cargo
has no containerisation rate; printing 0% would assert that nothing containerised.

**Never mix commodity levels.** `TOTAL` rows already contain the `HS2` rows beneath
them. Summing across levels double counts.

**Thin cells cannot carry a ratio.** The commodity series excludes chapters under
50,000 tonnes in a month.

### What is NOT yet supportable

An apparent rise in import containerisation from 38% (2010) to 63% (2026), against a
*fall* in exports from 37% to 26%, is visible in the data today. **It is not publishable
and the series are suspended.** The 2010–2023 backfill is still loading: the archive
holds twelve months for 2024 and 2025 but a partial 2010, so any cross-year comparison
weighs one loaded month against twelve. The volumes give it away — 2010 shows a sixth of
2024's tonnage, which is a loading artifact, not trade growth.

Lift the suspension only when every month from 2010-01 to the last complete month is
present, **verified by a per-year month count, not by row totals.**

## Derived series: value density

```
value density = containerised customs value / containerised shipping weight
```

**This is the method Census itself abandoned.** It discontinued its unit-value trade
indices in 1989 for composition bias and replaced them with survey-based price indexes;
the IMF's manual calls unit-value indices "seriously misleading" as price proxies. So a
naive $/kg series is not a neutral choice — it revives a method the source agency
dropped, and must be built to survive that objection or not built.

**Proven in our own data 2026-07-30.** Chapter 85 at Los Angeles spans **1.1 $/kg**
(filament lamps) to **179.8 $/kg** (multilayer ceramic capacitors) — a **161× range
inside one HS2 chapter**. Its chapter figure of 12.9 $/kg is a weighted average of all
of that and moves whenever the mix moves, with no price change at all.

### What is sound, and what is not

**The SCISSOR level is sound.** Import value density against export value density, same
port, same month, both computed the same way — measurement bias (shipping-weight
conventions, estimated low-value shipments) is common-mode and largely differences out.
May 2026: every gateway imports goods worth more per kilo than it exports, from about
1.0× at New York to **4.1× at Tacoma**. That asymmetry is why a box arriving full often
leaves empty. Textbook qualitatively; not published as a monthly measurement.

**The MOVEMENT is not interpretable at this level.** Same-month year on year the ratio
runs 2.03× (2024) → 2.02× (2025) → **1.83×** (2026), import density falling from 5.17 to
4.68 $/kg. At TOTAL aggregation that is indistinguishable from a mix shift, so all three
series are suspended until an HS6 decomposition can separate the within-line component
from the mix component and publish both side by side.

### Rules encoded in the queries

- **Containerised only.** Air runs $5–15/kg against ocean $0.50–2. An all-modes $/kg
  series is a mode-mix index wearing a price label.
- **Port-total row only** (`country='-'`), so overlapping country groupings cannot
  inflate either side.
- **HS6 for anything about the goods.** It is also the deepest level common to both
  directions — imports publish to HS10, exports only to HS6.
- **Call it unit value.** Never quality, never value added.

## Deep commodity levels, fetched on demand

`cli.js deep --level=HS6 --from=YYYY-MM [--to=] [--ports=] [--direction=]`

Scope is always named explicitly and printed back before the run, so a very large pull
cannot start by accident. Measured 2026-07-30: HS6 returns **~71,000 rows per
port-direction-month** and takes ~30 s per request.

| scope | rows | disk | time |
|---|---|---|---|
| 2 ports, 13 months, both directions | ~3.7M | ~0.4 GB | ~30 min |
| **11 ports, 24 months, both directions** | **~37M** | **~4.5 GB** | **~4.5 h** |
| 11 ports, full 197-month history | ~244M | ~29 GB | ~2 weeks |

**Two years across all ports is affordable and is the right default** for anything about
recent trends; the full history at HS6 is not, and is what the on-demand design exists
to avoid.

## The check this instrument was built for

**CHK-003** — `tt001.us-gateways.cargo-worked` against `tt008.us-gateways.import-weight`.

It cannot run yet and says so: TT-001's archive begins 26 July 2026 and Census reaches only May.
**The first clean shared month is August 2026, arriving in early October.** Until then the checker
reports insufficient overlap rather than a number, which is the correct behaviour.

## Operations

```
node cli.js sync 2026-05             one month, both directions, TOTAL + HS2
node cli.js sync --months=6          re-fetch the last N complete months (catches revisions)
node cli.js backfill --from=2024-01  every month to the last complete one
node cli.js revisions                Census restatements, separated from our own backfills
node cli.js status                   coverage, and what is missing
```