Providers must price near marginal cost, so public API pricing tiers directly encode internal architecture choices like KV bytes-per-token and which memory tier backs each cache duration.

Who

Reiner Pope

Topic

Pricing as a signal

Source

Reiner Pope — The math behind how LLMs are trained and served (01:33:02)