Model Management and Inference

Model Management and Inference

Multi-model Support — References

Multi model support › References

4 of 49 in this book · 1 min read

Canonical sources for the API shapes, tokenizers, and streaming formats an abstraction layer has to reconcile.

Provider API contracts

Wire formats and schemas

Tokenization

  • tiktoken — the byte-pair encodings used by OpenAI models, and a concrete demonstration that token counts are encoding-specific.
  • Hugging Face Tokenizers — tokenizer implementations and tokenizer.json artefacts for open-weight model families.