AI accelerator (GPU)
Unlock the Power of AI with Cube.COS GPU Passthrough#
Leverage the full potential of AI accelerators with Cube.COS robust support for GPU device passthrough. Designed to meet the demanding needs of modern AI and machine learning workloads, this feature empowers enterprises to harness the raw computing power of dedicated GPUs directly within virtual machines, ensuring unparalleled performance, efficiency, and flexibility.
Connect to cli with terminal or putty#
$ ssh admin@IPADDRESSWarning: Permanently added '192.168.X.X' (ECDSA) to the list of known hosts.Password:Welcome to the Cube ApplianceEnter "help" for a list of available commandscontroller>Create device profile#
controller:gpu> device_profile_createCreating device profile for NVIDIA Corporation GA106 [RTX A2000] (resource unit: 1)+-------------+------------------------------------------------------------------------------------------------------------------+| Field | Value |+-------------+------------------------------------------------------------------------------------------------------------------+| created_at | 2024-07-18 02:34:24+00:00 || updated_at | None || uuid | bbe5ea28-deed-4131-9579-7fd9b06de9b4 || name | rtx_a2000_1 || groups | [{'resources:PGPU': '1', 'trait:CUSTOM_GPU_PRODUCT_ID_2531': 'required', 'trait:CUSTOM_GPU_NVIDIA': 'required'}] || description | None |+-------------+------------------------------------------------------------------------------------------------------------------+Check status#
controller:gpu> status+-----------+| Node View |+-----------+---------------------------------------
IOMMU: on
GPU IOMMU Group ListIOMMU Group 14: 18:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA106 [RTX A2000] [10de:2531] (rev a1) (driver: nvidia)IOMMU Group 14: 18:00.1 Audio device [0403]: NVIDIA Corporation GA106 High Definition Audio Controller [10de:228e] (rev a1) (driver: snd_hda_intel)
Thu Jul 18 10:33:50 2024+---------------------------------------------------------------------------------------+| NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 CUDA Version: N/A ||-----------------------------------------+----------------------+----------------------+| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. || | | MIG M. ||=========================================+======================+======================|| 0 NVIDIA RTX A2000 On | 00000000:18:00.0 Off | Off || 30% 42C P8 14W / 70W | 0MiB / 6138MiB | 0% Default || | | N/A |+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+| Processes: || GPU GI CI PID Type Process name GPU Memory || ID ID Usage ||=======================================================================================|| No running processes found |+---------------------------------------------------------------------------------------+
Thu Jul 18 10:33:50 2024+-----------------------------------------------------------------------------+| NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 ||---------------------------------+------------------------------+------------+| GPU Name | Bus-Id | GPU-Util || vGPU ID Name | VM ID VM Name | vGPU-Util ||=================================+==============================+============|| 0 NVIDIA RTX A2000 | 00000000:18:00.0 | 0% |+---------------------------------+------------------------------+------------+
GPU 00000000:18:00.0 Migration capability : No
Virtualization:No vGPU type configured
Supported vGPU types:
+--------------+| Cluster View |+--------------+------------------------------------
Accelerator:+--------------------------------------+------+--------+----------+-------------------------------------------------------------------+| uuid | type | vendor | hostname | std_board_info |+--------------------------------------+------+--------+----------+-------------------------------------------------------------------+| e24eeaa6-fabd-4262-ae3c-6957b66f5785 | GPU | 10de | sky142 | {"product_id": "2531", "controller": "VGA compatible controller"} || 45eeb6b3-02f6-42df-93bd-c278599bc163 | GPU | 10de | sky143 | {"product_id": "2531", "controller": "VGA compatible controller"} |+--------------------------------------+------+--------+----------+-------------------------------------------------------------------+GPU Resource Provider:+--------------------------------------+---------------------+| uuid | name |+--------------------------------------+---------------------+| c419cc67-fbac-3c6c-81c3-cd5188f7a119 | sky142_0000:18:00.0 || 00cce127-e7df-3071-99d2-07e62767a049 | sky143_0000:18:00.0 |+--------------------------------------+---------------------+Accelerator Request:
Accelerator Device Profile:+--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------+-------------+| uuid | name | groups | description |+--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------+-------------+| bbe5ea28-deed-4131-9579-7fd9b06de9b4 | rtx_a2000_1 | [{'resources:PGPU': '1', 'trait:CUSTOM_GPU_PRODUCT_ID_2531': 'required', 'trait:CUSTOM_GPU_NVIDIA': 'required'}] | None |+--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------+-------------+Enable device passthrough by disable virtual function#
sky142:gpu> virtual_function_disableset device 10de:2531 at 0000:18:00.0 driver to vfio-pciset device 10de:228e at 0000:18:00.1 driver to vfio-pci
GPU IOMMU Group ListIOMMU Group 14: 18:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA106 [RTX A2000] [10de:2531] (rev a1) (driver: pcieport)IOMMU Group 14: 18:00.1 Audio device [0403]: NVIDIA Corporation GA106 High Definition Audio Controller [10de:228e] (rev a1) (driver: pcieport)