NVIDIA CUDA on Rocky Linux 9 with Support by Easycloud

Once launched, SSH into the instance. The default username is 'rocky'. You can switch to the root user environment by running: sudo su -

This product requires SSH; there is no web console or web UI. There are no default credentials - access is via your own EC2 key pair as user 'rocky'.

Requires a GPU instance type (g4dn / g5 / g6 / p3 / p4 / p5 and similar). Non-GPU instance types will not work.

• OS: Rocky Linux 9 (x86_64)

• NVIDIA Driver: 595.71.05 (open kernel module, DKMS)

• CUDA Toolkit: 13.2 installed at /usr/local/cuda-13.2, with /usr/local/cuda pointing to it

• GPU Communication: NCCL 2.29 (with headers) and gdrcopy 2.5

• Containers: Docker CE 29 with containerd 2.2 (buildx, compose) and NVIDIA Container Toolkit 1.19

• Fabric and MPI: EFA 3.1.0 with libfabric-aws 2.4.0, Open MPI 5.0.9, and the NCCL OFI plugin 1.20 - preinstalled and ready on EFA-capable instance types

• GPU Management: DCGM 4.6 and NVIDIA Fabric Manager (both installed, started on demand)

• Python: 3.12 with JupyterLab, IPython, NumPy, pandas, SciPy, matplotlib, boto3; plus AWS CLI v2

• Environment: CUDA paths are set system-wide via /etc/profile.d/cuda.sh and /etc/ld.so.conf.d/cuda.conf

• Not included by design: cuDNN, PyTorch, and TensorFlow. This is a base layer - you install the framework stack you need.

• No internet access is required to run this product - everything above is preinstalled. Outbound access is only needed if you pull container images or pip install frameworks yourself.

1. Launch on a GPU instance type. Recommended: g5.xlarge (A10G, 24 GB GPU memory); the image was verified on g4dn.xlarge. Non-GPU instance types will not work.

2. Allow about 4 minutes on first boot before the GPU is ready (measured 234 seconds from launch to the first successful nvidia-smi on g4dn.xlarge). This first-boot wait is normal, not a fault.

3. SSH in as 'rocky' and run the first command to confirm the GPU and driver: nvidia-smi

4. Verify GPU access from a container: docker run --rm --gpus all <image> nvidia-smi Replace <image> with an actual image, for example ubuntu:24.04. The 'rocky' user is in the docker group, so sudo is not required.

5. Compile CUDA code. nvcc is already on PATH: nvcc --version, then build with: nvcc -o myapp myapp.cu Build systems expecting CUDA_HOME resolve through /usr/local/cuda.

6. Install your own framework stack, for example: python3 -m venv ~/venv && source ~/venv/bin/activate && pip install torch Match the build to CUDA 13.x when the project offers CUDA-specific wheels.

7. Start JupyterLab on demand when you want a notebook interface: jupyter lab --ip=0.0.0.0 It is installed but not started automatically. If you use it, open its port in your Security Group and restrict it to trusted IP addresses.

8. Manage services with systemctl start/stop/restart/status. Docker and nvidia-persistenced are enabled and start at boot. GPU monitoring and Fabric Manager are installed but disabled by default: sudo systemctl start nvidia-dcgm && dcgmi discovery -l, and enable nvidia-fabricmanager on NVSwitch-equipped multi-GPU instance types.

9. The CUDA, Docker, and NVIDIA Container Toolkit repositories are shipped disabled so a routine dnf update cannot break the driver and DKMS pairing. To upgrade those components, re-enable the relevant repository explicitly with dnf --enablerepo. EPEL remains enabled.

• SSH (Port 22): Required - restrict to trusted IPs. This product requires SSH; there is no web console or web UI.

• No application ports are opened by default. If you later start JupyterLab yourself, open its port and restrict it to trusted IPs as well.

• Security Recommendation: For production environments, strictly limit access to these ports to trusted IP addresses only via cloud Security Groups or the local firewall.