nvidia_device_plugin_enable

Description

The nvidia_device_plugin_enable parameter enables the NVIDIA GPU device plugin for your Kubernetes cluster. When enabled, this plugin allows Kubernetes to discover and manage NVIDIA GPUs on nodes that have them installed, making these GPUs available to your applications. The plugin deploys as a DaemonSet that runs only on GPU-capable nodes and handles the exposure of GPU resources to the Kubernetes scheduler.

Default Value

The default value for nvidia_device_plugin_enable is false.

Use Cases

Machine Learning Workloads: Enable GPU acceleration for training and inference tasks.
Video Processing: Accelerate video encoding, decoding, and transcoding operations.
Scientific Computing: Support high-performance computing applications that benefit from GPU parallelization.
Rendering: Enable GPU-accelerated rendering for graphics-intensive applications.
Deep Learning Inference: Deploy inference engines that require GPU acceleration for optimal performance.

Setting Parameters

To enable the NVIDIA GPU device plugin, use the following command:

$ convox rack params set nvidia_device_plugin_enable=true -r rackName
Updating parameters... OK

To disable the NVIDIA GPU device plugin:

$ convox rack params set nvidia_device_plugin_enable=false -r rackName
Updating parameters... OK

Additional Information

The plugin only runs on nodes Convox has identified as GPU instances, so enabling it on a Rack with no GPU nodes schedules nothing.
Before enabling this parameter, ensure your AWS EC2 instances have compatible NVIDIA GPU hardware, such as instances from the P3, P4, G4, or G5 families.
The device plugin works in conjunction with the gpu scaling option in your convox.yml file, which allows you to specify GPU requirements for your services:

services:
  ml-service:
    build: .
    command: python train.py
    scale:
      count: 1
      gpu:
        count: 1
        vendor: nvidia

Your container image supplies the CUDA userspace libraries. The kernel driver comes from the node's AMI. To run a driver newer than the one the EKS-optimized AMI ships, see GPU Nodes and Custom AMIs.
GPU resources are whole units and cannot be fractionally allocated. Each container requesting a GPU will receive one or more complete GPUs.
When a service requests GPU resources, it will only be scheduled on nodes with available GPUs, which may affect scheduling and scaling behavior.

gpu_observability_enable: Installs the NVIDIA DCGM exporter for GPU utilization, VRAM, temperature, and power metrics. Requires this parameter to be enabled for the pod-resources socket the exporter relies on.
gpu_tag_enable: Enables GPU tagging, which helps with identifying and tracking GPU resources in your AWS environment.
additional_karpenter_nodepools_config: Put GPU instances on a dedicated Karpenter node pool rather than on node_type, which is Rack-wide and would place every Service on GPU hardware.
additional_node_groups_config: Put GPU instances on a dedicated EKS managed node group on a Rack that does not run Karpenter.

Version Requirements

This feature requires at least Convox rack version 3.21.0.