Data Connectors — References

What a connector has to reproduce faithfully: the source's change semantics, its quota protocol, and the authorisation it was granted.

Change detection and synchronisation

• RFC 9110: HTTP Semantics — conditional requests, entity tags and Last-Modified semantics, including why a weak validator is not evidence that content is unchanged.
• RFC 6585: Additional HTTP Status Codes — the too-many-requests status and the Retry-After convention a crawler must honour rather than replace with a fixed backoff.

Authorisation to the source

• RFC 6749: The OAuth 2.0 Authorization Framework — the grant types behind delegated and service-to-service access, and the refresh-token lifecycle whose expiry is a common cause of silent ingestion stoppage.
• RFC 7519: JSON Web Token — the assertion format most service-account flows use, and the claims that scope what a connector may enumerate.
• System for Cross-domain Identity Management (SCIM 2.0), RFC 7644 — the provisioning protocol that is often the only reliable way to resolve the group identifiers a source attaches to a document.

Crawl conventions

• RFC 9309: Robots Exclusion Protocol — the standardised exclusion rules, relevant wherever a connector crawls web content rather than calling an application API.

No verified reference is recorded for permission-fingerprint change detection, for shared quota budgeting between backfill and incremental crawls, or for identity reconciliation across source renames. These are operational practice rather than published work.