I’ve read multiple times that CUDA dominates, mostly because NVIDIA dominates. Rocm is the AMD equivalent, but OpenCL also exists. From my understanding, these are technologies used to program graphics cards - always thought that shaders were used for that.
There is a huge gap in my knowledge and understanding about this, so I’d appreciate somebody laying this out for me. I could ask an LLM and be misguided, but I’d rather not 🤣
Nvidia has the money and influence to make CUDA a standard. Popular means better…
Because Nvidia wrote an absolute fuck-ton of their Cuda stack to make it far more performant for many common tasks out of the box.
Early opencl implementations absolutely sucked.
ROCm is an implementation/superset of OpenCL.
ROCm ships its installable client driver (ICD) loader and an OpenCL implementation bundled together. As of January 2022, ROCm 4.5.2 ships OpenCL 2.2
Shaders are computational visual [post-]processing - think pixel position based adjustments to rendering.
OpenCL and CUDA are computation frameworks where you can use the GPU for other processing than rendering. You can use it for more general computing.
nVidia has always been focusing on proprietary technology. Introduce a technology, and try to make it a closed market, where people are forced to buy and use nVidia for it. AMD has always been supporting and developing open standards as a counterplay to that.
Now, I don’t write code. So I can’t really tell you if this is the truth or not — but:
I hear that OpenCL is much more difficult and less accessible to write than CUDA code. CUDA is easier to write, and thus gets picked up and used by more developers.
Someone mentions CUDA “sometimes” having better performance, but I don’t think it’s only sometimes. I think that due to the existence of the tensor cores (which are really good at neural nets and matrix multiplication), CUDA has vastly better performance when taking advantage of those hardware features.
Tensor cores are not Nvidia specific, but they are the “most ahead”. They have the most in their GPU’s, and probably most importantly: CUDA only supports Nvidia, and therefore by extension, their tensor cores.
There are alternative projects, like how leela chess zero mentions tensorflow for google’s Tensor Processing Units, but those aren’t anywhere near as popular due to performance and software support.
AFIK it’s only NVIDIA that allows containers shared access to a GPU on the host.
With the majority of code being deployed in containers, you end up locked into the NVIDIA ecosystem even if you use OpenCL. So I guess people just use CUDA since they are limited by the container requirement anyways.
That’s from my experience using OpenGL headless. If I’m wrong please correct me; I’d prefer being GPU agnostic.
Check implementations before saying shit like that. Nvidia has historical bad open source driver support, which makes it hard for people to implement vGPU usage. They actually actively blocked us from using their cards remotely, until COVID hit. Then they gave out the code to do it. They are still limiting customer level cards usage on virtualization cases. They had to give out a toolkit for us to be able to use their cards on docker. Other cards can be accessed just by sharing dev driver files to the volume.
Can you share sample code I can try or documentation I can follow of using an AMD GPU in that way (shared, virtualized, using only open source drivers)?
Check Wolf (in my other comment), it’s the best example of GPU virtualization usage.
Otherwise you can check other docker images using GPU for computing, like jellyfin for instance, or nextcloud recognize, nextcloud memories and its transcoding instance,…
AFIK it’s only NVIDIA that allows containers shared access to a GPU on the host.
This cannot be right. I’m pretty sure that it is possible to run OpenCL applications in containers that are sharing a GPU.
I should test this if I have time. My plan was to use a distrobox container since that shares the GPU by default and run something like lc0 to see if opencl acceleration works.
Now where is my remindme bot? (I won’t have time).
You really piqued my interest. I use docker/podman.
W/ an AMD graphics card, eglinfo on the host shows the card is AMD Radeon and driver is matching that.
In the container, without --gpus=all, it shows the card is unknown and the driver is “swrast” (so just CPU fallback).
To make --gpus=all work, it gives the error
docker: Error response from daemon: could not select device driver “” with capabilities: [[gpu]
I was doing a bad job searching before. I found that AMD can share the GPU, it just works a little differently in terms of how to launch the container. https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/amdgpu-install.html#amdgpu-install-dkms
But sadly my AMD GPU is too old/junk to have current driver support.
Anyways, appreciate the reply! Now I can mod my code to run on cheaper cloud instances.
(Note I’m an OpenGL/3D app developer, but probably OpenCL works about the same architecturally)
I’d imagine it’s as simple as Nvidia doesn’t want to support OpenCL because CUDA forces people into their market, and AMD doesn’t want to support OpenCL because Rocm forces people into their market. Open/free standards are great for smaller players in a market, but in the graphics space I don’t think a smaller player in the market exists, and if they do, no one seems interested in them.
rocm is open source as well. amd have historically been the ones pushing for open standards in these things, probably because they’ve never been market leaders.
CUDA can (depending on circumstances) give slightly better performance than OpenCL. So if you know that your target hosts will have Nvidia GPUs ( for example ML in your own data centers) that might be beneficial.
OpenCL will run on multiple platforms so if you don’t know the target hosts (for example consumer hardware for gaming) this makes life easier for the developer.
The frameworks and libraries around the different specs differ, so if there is a library that is useful for your use case, that will effect your decision to pick one over the other
