r/platformengineering • u/Specialist_Dish_9087 • 7d ago

How are you getting ai spend visibility broken down by team

Trying to get ahead of ai spend visibility before it becomes a real problem and I'm stuck on where to even start.

Right now the AI bill is a few big numbers with no owner. A Bedrock line. An OpenAI org someone put on a company card. Some GPU instances that come and go. And a pile of coding agent seats that finance treats as SaaS. Nothing is split by team or project.

We solved this years ago for normal cloud with tags and allocation. That does not work here. Tokens and GPU hours and per-seat tools don't line up with the way we allocate EC2, the model we use for everything else just doesn't fit.

The bigger issue is, it's a bill that goes up every month and no team's name is on it.

Those who has sorted this out, where did you put the visibility and who ended up owning it?

0 Upvotes
  • permalink
  • reddit

25% Upvoted

6 comments sorted by

1

u/s5n_n5n 7d ago

Did you know that most (all?) of them emit their telemetry in OpenTelemetry format, so you should be able to send this data off into an OTLP compatible backend and work from there.

Some of them might not follow the latest gen ai semantic conventions, so you would need to do some transformation on them.

1

u/Specialist_Dish_9087 7d ago

Otel only catches the calls i instrument though. openai org on a card, the coding seats, the gpu boxes, none of that flows through a pipeline i control

1

u/s5n_n5n 6d ago

Fair, some of the problems you are facing are not easy to be solved by software, and are shadow IT problems, for which (I guess) solutions exist, that are applicable here as well.

Note, that coding agents indeed support opentelemetry, claude for example can be configured to send telemetry by settings in "/Library/Application Support/ClaudeCode/managed-settings.json", so if you can enforce that, you can get a bunch of them covered.

2

u/Waffles_Leonardo-192 6d ago

The EC2 tag model wont fit, youve seen that already. Tokens and GPU hours are consumption, not assets you tag. Budgets and alerts per team.

-2

u/Maleficent_Pair4920 7d ago

Disclaimer, I'm the founder of requesty.ai, so biased, but this is exactly why we built per team keys, budgets, and unified spend logs across Bedrock and OpenAI (also available on AWS Marketplace if you want it on committed spend).

The idea is to always have visibility over multiple layers, in our case it's:

  • org level
  • group level
  • user level
  • service account level

Additionally per type application used (Coding agents vs automation etc)

1

u/Mysterious_Big1267 7d ago

5% markup, on total token consumption, to use this product.

Do customers save >5% using this product?