Don’t you think that tokens eventually will be cheap enough as well we learn how to better constrain coding agents to use less token be adding repeatable scripts/workflows, caching etc?
I think we need much better visibility into token usage and spend.
Similar to cloud platforms, where we can use a billing calculator during design time to estimate costs upfront, we need the same capability for GenAI. As I design a prompt or workflow, I should be able to predict what the cost will look like and plan accordingly.
Right now, that visibility is missing! Ideally, I should be able to answer a simple question: how much is this prompt going to cost me?
Will tokens become cheaper - I expect so.
However the AI companies (Anthropic, OpenAI) are looking at IPOs soon, so it will be interesting to see how low they can go, and still meet their quarterly targets and expectations of Wall Street!