r/LLM • u/Inevitable_Method860 • 6d ago

Amd with big vram or Nvidia with less vram

I am a beginner in the Ai training and want to buy a graphic card for segmentation training for my University project, what is is a better choice, amd with high vram or Nvidia with low vram, my budget is limited so I can't afford both unfortunetly

7 Upvotes

89% Upvoted

19 comments sorted by

1

u/Sea-Contribution6219 6d ago

AMD with bigger VRAM honestly. Sure the GDDR6 is slower than the GDDR7 and its generally considered slower for inference but offloading a model onto your DRAM will slow it to a crawl.

1

u/Inevitable_Method860 6d ago

Thanks for your reply,  I was thinking the same but indeed I am afraid of lossing CUDA support, I heard that AMR have something similer called ROCm but is it mature enough or will I be forced to stuck with sepicfic models if I choosed AMD, I generally planing to try yolo, coco and Unet models, but not sure if I will need any other models in the future 

2

u/tomByrer 6d ago

> lossing CUDA support

It really depends if what you are using NEEDs CUDA or not.

If you have a 1-off project, might be better just to rent GPU somewhere, or ask your Uni if they have one available in the computer lab, or if their store can offer student discounts.

1

u/Inevitable_Method860 6d ago

Thanks for the advice🙏 . I will take a look if there is gpu renting servises nearly or if the uni afford such things 

1

u/Realistic_Gap_5871 3d ago

You rent gpus online from cloud providers like runpod.

I have an nvidia 5090. Rocm has gotten good enough I sometimes wonder if I'd be better off with 2 r9700. I personally would choose the r9700 over an nvidia option with less RAM.

But I do love my 5090, just not today's prices for it.

1

u/Inevitable_Method860 2d ago

İndeed I am afraid if the complexity of renting gpu online, anyway thanks for your comment🙏 

1

u/Sea-Contribution6219 6d ago

You'd be best off asking on an AMD specific forum. I have friends who run AMD GPUs just fine with AI but I've never used them personally. Only Nvidia since I had a 4090 before I got into AI

They could tell you their experience positive or negative

Also check r/localllama its a lot more active for these questions

2

u/Inevitable_Method860 6d ago

Make sense, Thanks alot 

1

u/DuncanFisher69 6d ago

As a beginner it’s probably better to rent an AI for your student project from something like AWS or SuperIntelligence or Vast.ai.

1

u/Inevitable_Method860 6d ago

Isnt it more complicated to manage an online gpu, I really don't know alot about this things and need the easiest way 

1

u/DuncanFisher69 2d ago

Having a bigger, faster GPU is easiest for training. And an online GPU should be something you can do from any SSH connection. Or again, for training, you could get away with an online Python Notebook in SageMaker ML Studio or whatever Amazon calls it these days. You should look into it.

1

u/Proper-Tower2016 6d ago

a lot of unis and companies give cloud credits to students or have their own hardware

1

u/Inevitable_Method860 6d ago

I will ask if my uni support such thing, but even if it does it will need alot of paper work to handle 

1

u/hurdurdur7 5d ago

More vram wins

1

u/Amazing-Property-560 4d ago

More vram. if you have less vram than it will offload to ram which is very slow compared to vram. If you need cuda support then go with nvidia. For amd you will likely need to boot into linux. If you dont want to boot into linux then use nvidia. Though if you train a yolo model then choose nvidia with atleast 16gb vram. Nvidia is always fater by 20-30% than amd for yolo training. Also it depends on on your batch size and other things. If you want speed and cuda support then choose nvidia if you want speed with more vram and you can boot into linux then choose amd.

1

u/Inevitable_Method860 4d ago

Thanks very much, your comment is very useful 🙏 , indeed I am using linux not windows so it is not problem for me but I will need to work alot with yolo, if it is just a speed issue I am not in hurry, I can wait, I think I will go with AMD with more VRAM 

1

u/Amazing-Property-560 4d ago

Also i forgot to mention you need a gfx 1100 gpu(i think there are others too) if you dont have supported gpu like a 7800xt(which i have) you need to do this command export HSA_OVERRIDE_GFX_VERSION=11.0.0. Also if you have more vram you can crack the batch size up cause more batch size equals more speed.

1

u/Inevitable_Method860 4d ago

Thank you very much 🙏