AI accelerator (GPU)
Unlock the Power of AI with Cube.COS GPU Passthrough
Leverage the full potential of AI accelerators with Cube.COS robust support for GPU device passthrough. Designed to meet the demanding needs of modern AI and machine learning workloads, this feature empowers enterprises to harness the raw computing power of dedicated GPUs directly within virtual machines, ensuring unparalleled performance, efficiency, and flexibility.
Connect to cli with terminal or putty
$ ssh admin@IPADDRESS
Warning: Permanently added '192.168.X.X' (ECDSA) to the list of known hosts.
Password:
Welcome to the Cube Appliance
Enter "help" for a list of available commands
controller>
Create device profile
controller:gpu> device_profile_create
Creating device profile for NVIDIA Corporation GA106 [RTX A2000] (resource unit: 1)
+-------------+------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------+------------------------------------------------------------------------------------------------------------------+
| created_at | 2024-07-18 02:34:24+00:00 |
| updated_at | None |
| uuid | bbe5ea28-deed-4131-9579-7fd9b06de9b4 |
| name | rtx_a2000_1 |
| groups | [{'resources:PGPU': '1', 'trait:CUSTOM_GPU_PRODUCT_ID_2531': 'required', 'trait:CUSTOM_GPU_NVIDIA': 'required'}] |
| description | None |
+-------------+------------------------------------------------------------------------------------------------------------------+
Check status
controller:gpu> status
+-----------+
| Node View |
+-----------+---------------------------------------
IOMMU: on
GPU IOMMU Group List
IOMMU Group 14: 18:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA106 [RTX A2000] [10de:2531] (rev a1) (driver: nvidia)
IOMMU Group 14: 18:00.1 Audio device [0403]: NVIDIA Corporation GA106 High Definition Audio Controller [10de:228e] (rev a1) (driver: snd_hda_intel)
Thu Jul 18 10:33:50 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 CUDA Version: N/A |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA RTX A2000 On | 00000000:18:00.0 Off | Off |
| 30% 42C P8 14W / 70W | 0MiB / 6138MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
Thu Jul 18 10:33:50 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 |
|---------------------------------+------------------------------+------------+
| GPU Name | Bus-Id | GPU-Util |
| vGPU ID Name | VM ID VM Name | vGPU-Util |
|=================================+==============================+============|
| 0 NVIDIA RTX A2000 | 00000000:18:00.0 | 0% |
+---------------------------------+------------------------------+------------+
GPU 00000000:18:00.0
Migration capability : No
Virtualization:
No vGPU type configured
Supported vGPU types:
+--------------+
| Cluster View |
+--------------+------------------------------------
Accelerator:
+--------------------------------------+------+--------+----------+-------------------------------------------------------------------+
| uuid | type | vendor | hostname | std_board_info |
+--------------------------------------+------+--------+----------+-------------------------------------------------------------------+
| e24eeaa6-fabd-4262-ae3c-6957b66f5785 | GPU | 10de | sky142 | {"product_id": "2531", "controller": "VGA compatible controller"} |
| 45eeb6b3-02f6-42df-93bd-c278599bc163 | GPU | 10de | sky143 | {"product_id": "2531", "controller": "VGA compatible controller"} |
+--------------------------------------+------+--------+----------+-------------------------------------------------------------------+
GPU Resource Provider:
+--------------------------------------+---------------------+
| uuid | name |
+--------------------------------------+---------------------+
| c419cc67-fbac-3c6c-81c3-cd5188f7a119 | sky142_0000:18:00.0 |
| 00cce127-e7df-3071-99d2-07e62767a049 | sky143_0000:18:00.0 |
+--------------------------------------+---------------------+
Accelerator Request:
Accelerator Device Profile:
+--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------+-------------+
| uuid | name | groups | description |
+--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------+-------------+
| bbe5ea28-deed-4131-9579-7fd9b06de9b4 | rtx_a2000_1 | [{'resources:PGPU': '1', 'trait:CUSTOM_GPU_PRODUCT_ID_2531': 'required', 'trait:CUSTOM_GPU_NVIDIA': 'required'}] | None |
+--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------+-------------+
Enable device passthrough by disable virtual function
sky142:gpu> virtual_function_disable
set device 10de:2531 at 0000:18:00.0 driver to vfio-pci
set device 10de:228e at 0000:18:00.1 driver to vfio-pci
GPU IOMMU Group List
IOMMU Group 14: 18:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA106 [RTX A2000] [10de:2531] (rev a1) (driver: pcieport)
IOMMU Group 14: 18:00.1 Audio device [0403]: NVIDIA Corporation GA106 High Definition Audio Controller [10de:228e] (rev a1) (driver: pcieport)