Hacker News (curated)new | past | comments | ask | show | jobs| show hidden

> Cache hit %

I thought you had to actively manage caches, do you not?



Most providers do what's called "prefix caching", where each turn in a session is cached such that sending new messages with the exact same "prefix" (set of previous messages) gives you the cache read price on that input instead of the full price. As long as you're not changing your system prompt, available tools, etc mid-session, you automatically benefit from this.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact | github