Tracking USGS endpoints¶
water-ogcapi talks to the four services the README lists with their base URLs. Each
service class reads the live /collections index, cached for an hour
(D-07), and checks a query's
collection and parameter names against it before the first page request
(D-47).
A collection USGS adds to a service the package already covers therefore works with the
released package the day it appears. A new service, and a change to an existing one,
take the steps below.
How a change is detected¶
The Drift workflow runs every Monday at 07:00 UTC, and from the Actions tab on request
(D-63). It has two jobs:
network-testsruns the tests markednetworkagainst the live services: the Potomac gage and the Washington, DC bounding box on WaterData, the gage's rating files on WaterDataSTAC, the DC bounding box on FabricData, and the 22 two-digit HUCs on GeoConnex.openapifetches each service's OpenAPI document and compares it withtests/fixtures/<service>/openapi.json: the whole document minus its prose (descriptions, summaries, titles, and examples), withinfocut to the title and version. A wording edit passes, and a$refis kept as the reference, so a change inside the referenced document is invisible. A service that does not answer fails the job and keeps its snapshot.
A failed openapi job prints a unified diff per service. Read it for:
| In the diff | What changed upstream |
|---|---|
Added /collections/<id> paths |
A new collection |
Removed /collections/<id> paths |
A collection withdrawn or renamed |
A changed maximum on a limit parameter |
The page cap that max_limit in _services.py mirrors (D-30); a changed default changes nothing, since the client sends its own limit (D-41) |
| A changed or added parameter | A query form the client may send |
A changed requestBody media type or schema |
The declared POST body; the client sends application/query-cql-json where the documents declare application/json, and the GeoConnex ids test guards that the servers accept it |
A changed info.version and nothing else |
A release without an interface change the snapshot sees |
A failed network-tests job means a stable target moved or a service misbehaves; a
FabricData outage skips its test (D-14). Investigate following
.agents/skills/live-api/SKILL.md.
The diff cannot show a new service, since nothing holds a snapshot for it. Watch the USGS Water Data APIs documentation at https://api.waterdata.usgs.gov/docs/ogcapi/ for a new base URL.
Between runs, the same check runs locally:
Responding to drift¶
-
Decide what the change means for the client, using the table above. A new collection needs no code; add a fixture and an example when it earns one. A changed cap updates
max_limitinsrc/water_ogcapi/_services.pyand the D-30 table. A removed collection or parameter updates the tests, examples, and guides that name it. -
Refresh the snapshots and commit them with their new
recordeddate: -
Open a PR for the change. When it changes what the client sends, release it, and name each snapshot's service version, the
info.versionfield of itsopenapi.json, in the release notes (D-63).
Adding a service¶
A new OGC API Features service gets:
- A
ServiceProfileand a service class insrc/water_ogcapi/_services.py: the base URL with its trailing slash, the default and maximumlimit, andkey_envonly for a USGS host that readsX-Api-Key(D-13, D-18). A blocking twin in_sync.py, and exports in__init__.pyandaio.py. - An entry in
SERVICESinscripts/openapi_drift.pyand a row intests/fixtures/README.md, thenpixi r -e test315 drift --update. - Fixtures through
scripts/record_fixtures.pyand a network-marked test on a stable target (D-14). - A row in the README's endpoint table and a
D-entry in the design record.