常用pytorch语法trick记录-截止2022.4.11

记录一些好的用法：
1.pytorch权重初始化代码，该代码放入nn.Module的初始化网络里。

for m in self.modules():
    if isinstance(m, nn.Conv2d):
        n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
        m.weight.data.normal_(0, math.sqrt(2. / n))
    elif isinstance(m, nn.BatchNorm2d):
        m.weight.data.fill_(1)
        m.bias.data.zero_()

2.生成带层名的网络的便捷方法

        layers = []
        # 下采样，步长为2，卷积核大小为3
        layers.append(("ds_conv", nn.Conv2d(self.inplanes, planes[1], kernel_size=3,
                                stride=2, padding=1, bias=False)))
        layers.append(("ds_bn", nn.BatchNorm2d(planes[1])))
        layers.append(("ds_relu", nn.LeakyReLU(0.1)))
        
        # 加入darknet模块，res_unit   
        self.inplanes = planes[1]
        for i in range(0, blocks):
            layers.append(("residual_{}".format(i), BasicBlock(self.inplanes, planes)))
        return nn.Sequential(OrderedDict(layers))

3.GPU tensor与CPUtensor之间的转换：

CPU张量 ---->  GPU张量, 使用data.cuda()

GPU张量 ----> CPU张量 使用data.cpu()

4.将参数放入字典，由类内字典直接加载：
“将参数写在类内，由字典进行加载，方便参数修改”

_defaults = {
    "sets": {0.5, 1, 1.5, 2},
    "units": [3, 7, 3],
    "chnl_sets": {0.5: [24, 48, 96, 192, 1024],
                  1: [24, 116, 232, 464, 1024],
                  1.5: [24, 176, 352, 704, 1024],
                  2: [24, 244, 488, 976, 2048]}
}
def __init__(self, scale, num_cls):
    super(ShuffleNet_v2, self).__init__()
    self.__dict__.update(self._defaults)

5.torch.tensor更改维度后保持连续性：

.permute(0, 1, 3, 4, 2).contiguous()