NVIDIA DALI从入门到放弃之四：Multiple GPU

NVIDIA DALI从入门到放弃之一：概述

NVIDIA DALI从入门到放弃之二：入门示例

NVIDIA DALI从入门到放弃之三：Data Loading

NVIDIA DALI从入门到放弃之四：Multiple GPU

NVIDIA DALI从入门到放弃之五：Image Processing

NVIDIA DALI从入门到放弃之六：Geometric Transforms

NVIDIA DALI从入门到放弃之七：Sequence Processing

NVIDIA DALI从入门到放弃之八：PyTorch Plugin API

DALI通过对数据集Sharding，生成对应于多个GPU的多个shards来实现Multiple GPU运行。

DALI Multiple GPU参数设置：

• shard_id
• num_shards.

示例如下：

def sharded_pipeline(device_id, shard_id, num_shards):
    pipe = Pipeline(batch_size=batch_size, num_threads=1, device_id=device_id)
    with pipe:
        jpegs, labels = fn.file_reader(
            file_root=image_dir, random_shuffle=False, shard_id=shard_id, num_shards=num_shards)
        images = fn.image_decoder(jpegs, device='mixed', output_type=types.RGB)

        pipe.set_outputs(images, labels)

    return pipe

# Create and build pipelines
pipe_one = sharded_pipeline(device_id=0, shard_id=0, num_shards=2)
pipe_one.build()

pipe_two = sharded_pipeline(device_id=1, shard_id=1, num_shards=2)
pipe_two.build()

# Run pipelines
images_one, labels_one = pipe_one.run()
images_two, labels_two = pipe_two.run()