(自学习用，侵删)

ResNet

    当F(x) = 0 时(即F(x)学习到的目标值为0)，跳过权重层，H(x) = x 为最优解；若H(x) = x 不是最优解，那么F(x)可以尽可能去拟合期望值。

f(x)=0                                              经过一个1x1的卷积拟合后的输出              

    BasicBlock：

        为ResNet中的一种网络结构，BasicBlock包含了残差支路和一个short-cut支路，由于比传统卷积结构多了一个short-cut支路用于传递低层信息，使得网络能够训练地很深。   

定义BasicBlock模块： 

    class BasicBlock(nn.Module):

        expansion = 1

        def __init__(self, inplanes, planes, stride=1, downsample=None):

            super(BasicBlock, self).__init__()

            self.conv1 = conv3x3(inplanes, planes, stride)

            self.bn1 = nn.BatchNorm2d(planes)

            self.relu = nn.ReLU(inplace=True)

            self.conv2 = conv3x3(planes, planes)

            self.bn2 = nn.BatchNorm2d(planes)

            self.downsample = downsample

            self.stride = stride

        def forward(self, x):

            residual = x

            out = self.conv1(x)

            out = self.bn1(out)

            out = self.relu(out)

            out = self.conv2(out)

            out = self.bn2(out)

            if self.downsample is not None:  #进行下采样

                residual = self.downsample(x)

            out += residual    #在输出上叠加了一次输入x ( 之前定义过 residual = x )

            out = self.relu(out)

            return out

        BasicBlock架构主要使用了两个3*3的卷积，然后进行BN(BatchNorma2d)，再将特征传递给ReLU。

    Bottleneck：

有bottleneck的ResNet模块

        Bottleneck由1x1，3x3，1x1 三个convolutions构成，其中1x1卷积层的作用是先减少再恢复维度，3x3层是较小输入/输出维度的瓶颈。Bottleneck用于特征降维，减少特征图的层数，减少参数量从而减少计算量。(一般在网络较深时才使用bottleneck结构，如ResNet18 ResNet34用BasicBlock，而ResNet50 ResNet101用Bottleneck)

定义Bottleneck： 

    class Bottleneck(nn.Module):

        expansion = 4

        def __init__(self, inplanes, planes, stride=1, downsample=None):

            super(Bottleneck, self).__init__()

            self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)

            self.bn1 = nn.BatchNorm2d(planes)

            self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride,

                                   padding=1, bias=False)

            self.bn2 = nn.BatchNorm2d(planes)

            self.conv3 = nn.Conv2d(planes, planes * 4, kernel_size=1, bias=False)

            self.bn3 = nn.BatchNorm2d(planes * 4)

            self.relu = nn.ReLU(inplace=True)

            self.downsample = downsample

            self.stride = stride

        def forward(self, x):

            residual = x

            out = self.conv1(x)

            out = self.bn1(out)

            out = self.relu(out)

            out = self.conv2(out)

            out = self.bn2(out)

            out = self.relu(out)

            out = self.conv3(out)

            out = self.bn3(out)

            if self.downsample is not None:

                residual = self.downsample(x)

            out += residual

            out = self.relu(out)

            return out

参考文章

https://blog.csdn.net/csdnldp/article/details/78313087

https://blog.csdn.net/sazass/article/details/116864275

https://zhuanlan.zhihu.com/p/98692254

https://www.zhihu.com/question/413586557