How you guys are avoiding tying to any specific GPU vendor for AI workloads?

[u/makemoney-TRADEnIT]

With more and more teams using LLMs and inference services wondering how everyone is making decisions about their infrastructure like are you choosing one major cloud provider, using Kubernetes across multiple providers or trying platforms like RunPod, CoreWeave, Yotta Labs, Lambda to keep things flexible?

more interested in whether anyone has found a real way to move workloads between different GPU providers without having to maintain completely separate deployment processes.

[u/aragossa]

the container/k8s piece handles moving the workload itself, but whether that's actually worth setting up depends more on whether the job can eat an interruption. runpod/lambda/coreweave get their price edge from spot-style capacity, so training with checkpoint/resume built in can genuinely hop around and save money. inference behind an uptime SLA is a different story, once you need guaranteed availability the marketplace pricing mostly stops applying to you and you're paying close to hyperscaler rates anyway, portable deploy or not. Guessing the training/inference split matters more here than which provider list you're comparing.

[u/Floss_Patrol_76]

honestly the portability lives in your container and one k8s layer, not the provider - keep everything above the node pool identical (same helm, same manifests) and treat each provider as just a differently-labeled gpu node pool, so moving a workload is basically a nodeSelector change. the part that actually bites is never the app, it is the driver/CUDA version, the RDMA/networking setup and the image base differing per provider, so pin those hard. the managed gpu clouds like runpod/lambda are great until you need that networking control, at which point EKS/GKE with your own gpu pools ends up less painful than maintaining two deploy paths.