Discussion about this post

User's avatar
jeff klugman's avatar

i see 3 tiers in the future: lowest- commoditized, price close to the cost of compute only. mid-tier- strong mid-tier models with strong harnesses to integrate private data and existing corporate software systems, support audit, for finance add kyc, aml, and whatever some corporate user might not want accessible to the ccp. top tier- limited distribution with strong security checks of systems and individuals, defense, national security, pharma drug design- we may never even hear these models exist.

Abel's avatar

Hi Geo great post tks - your Silicon Data index caught my eye and did a bit more digging.

The comment above that "overall spending on tokens peaked in June and has been trending lower ever since" is inaccurate.

Per Claude Fable5 (and reconfirmed by BBG ASKB) - "What it actually measures. Despite the name, SDLLMTK is a price index, not a measure of aggregate spending. It's Silicon Data's daily benchmark for LLM inference token pricing, published as a normalized blended rate in USD per million tokens, drawn from observations across frontier API providers, open-weight inference platforms, brokered dedicated-instance markets, and self-hosted reference deployments. Silicon Data has publicly acknowledged the naming problem: the company said the index "should really have been named the 'Token Expenditure Price Index'" because it's an expenditure- or usage-weighted average token price — telling you what the market currently pays per million tokens irrespective of model — and noted that the name led some people to misread it as total token volume or cumulative spend. So it does not track total token-related expenditure across the software industry (or any industry) in dollar terms. The firm itself frames it as a proxy for marginal willingness to pay for AI."

https://www.silicondata.com/products/silicon-index/llm-token-expenditure-index

FWIW

11 more comments...

No posts

Ready for more?