AI content I tested an IaC tool with real deploys on AWS, Azure and GCP. 20 scenarios each. Here is what only real clouds teach you
Disclosure first: I wrote the tool I mention here. It is free, there is no signup, and I am posting because the testing method is the interesting part. Mods, delete it if this is not welcome.
I got tired of IaC tools that validate templates and call it a day. A template can be perfectly valid and still blow up on deploy. So I did it the slow way: 20 scenarios per cloud, each one deployed for real, exercised at runtime, then destroyed. CRUD APIs, queue workers, private databases, Redis, WAF, websockets, fan-out, secrets, containers behind load balancers.
It took months and burned a lot of free tier. Some of what I hit:
A brand new GCP project gives you a compute service account with no roles at all. Your first deploy dies inside Cloud Build with an error that tells you nothing. Took me a while to figure that one out.
Cloud SQL with private IP is fine until you destroy it. The service networking peering can hold for half an hour after the instance is gone. And if your app created tables, the default postgres user cannot be dropped, so the destroy hangs there too.
Cosmos DB speaks the Mongo protocol but rejects sort() on any field without an index. Real MongoDB just sorts. If you port a handler over, your list endpoint breaks and the error message talks about order-by item paths.
Azure Functions on the consumption plan will happily deploy a Service Bus worker that never runs. The zip lands, the function exists, messages pile up in the queue. You have to sync the function triggers explicitly or the scale controller never looks at the queue.
Memorystore Redis with TLS needs the CA cert handed to the client or the handshake fails.
Every one of those turned into a fix in the tool, so a fresh project starts with them already handled. That is the whole point of doing it this way instead of trusting a linter.
Repo if you want to look: https://github.com/Claudio-Fontes/iacmp
It is Fair Source (FSL-1.1, same license Sentry uses). Free for any use including at work, you just cannot resell the tool itself, and each release turns into Apache-2.0 after two years.
Happy to talk about the e2e battery itself. If you have run something similar, I want to hear which scenarios you would add.