## DenseNet

resnet一文里说了,resnet是具有里程碑意义的.densenet就是受resnet的启发提出的模型.

resnet中是把不同层的feature map相应元素的值直接相加.而densenet是将channel维上的feature map直接concat在一起,从而实现了feature的复用.如下所示:

bottleneck layer

growth rate

conv

## 实现DenseBlock

### DenseLayer

```class DenseLayer(nn.Module):
def __init__(self,in_channels,bottleneck_size,growth_rate):
super(DenseLayer,self).__init__()
count_of_1x1 = bottleneck_size
self.bn1 = nn.BatchNorm2d(in_channels)
self.relu1 = nn.ReLU(inplace=True)
self.conv1x1 = nn.Conv2d(in_channels,count_of_1x1,kernel_size=1)
self.bn2 = nn.BatchNorm2d(count_of_1x1)
self.relu2 = nn.ReLU(inplace=True)

def forward(self,*prev_features):
# for f in prev_features:
#     print(f.shape)
input = torch.cat(prev_features,dim=1)
# print(input.device,input.shape)
# for param in self.bn1.parameters():
#     print(param.device)
# print(list())
bottleneck_output = self.conv1x1(self.relu1(self.bn1(input)))
out = self.conv3x3(self.relu2(self.bn2(bottleneck_output)))

return out```

```def forward(self,*prev_features):
# for f in prev_features:
#     print(f.shape)
input = torch.cat(prev_features,dim=1)
# print(input.device,input.shape)
# for param in self.bn1.parameters():
#     print(param.device)
# print(list())
bottleneck_output = self.conv1x1(self.relu1(self.bn1(input)))
out = self.conv3x3(self.relu2(self.bn2(bottleneck_output)))

return out```

tips:

```## var-positional parameter
## 定义的时候，我们需要添加单个星号作为前缀
def func(arg1, arg2, *args):
print arg1, arg2, args

## 调用的时候，前面两个必须在前面
## 前两个参数是位置或关键字参数的形式
## 所以你可以使用这种参数的任一合法的传递方法
func("hello", "Tuple, values is:", 2, 3, 3, 4)

## Output:
## hello Tuple, values is: (2, 3, 3, 4)
## 多余的参数将自动被放入元组中提供给函数使用

## 如果你需要传递元组给函数
## 你需要在传递的过程中添加*号
## 请看下面例子中的输出差异：

func("hello", "Tuple, values is:", (2, 3, 3, 4))

## Output:
## hello Tuple, values is: ((2, 3, 3, 4),)

func("hello", "Tuple, values is:", *(2, 3, 3, 4))

## Output:
## hello Tuple, values is: (2, 3, 3, 4)```

### DenseBlock

```class DenseBlock(nn.Module):
def __init__(self,in_channels,layer_counts,growth_rate):
super(DenseBlock,self).__init__()
self.layer_counts = layer_counts
self.layers = []
for i in range(layer_counts):
curr_input_channel = in_channels + i*growth_rate
bottleneck_size = 4*growth_rate #论文里设置的1x1卷积核是3x3卷积核的４倍.
layer = DenseLayer(curr_input_channel,bottleneck_size,growth_rate).cuda()
self.layers.append(layer)
def forward(self,init_features):
features = [init_features]
for layer in self.layers:
layer_out = layer(*features) #注意参数是*features不是features
features.append(layer_out)

## TransitionLayer

```class TransitionLayer(nn.Sequential):
def __init__(self, in_channels, out_channels):
super(TransitionLayer, self).__init__()
self.add_module('conv', nn.Conv2d(in_channels, out_channels,kernel_size=1, stride=1, bias=False))

## Dense Net

dense net的基本组件我们已经实现了.下面就可以实现dense net了.

```class DenseNet(nn.Module):
def __init__(self,in_channels,num_classes,block_config):
super(DenseNet,self).__init__()
self.conv1 = nn.Sequential(
nn.BatchNorm2d(64),
nn.ReLU(inplace=True)
)

self.dense_block_layers = nn.Sequential()
block_in_channels = in_channels
growth_rate = 32
for i,layers_counts in enumerate(block_config):
block = DenseBlock(in_channels=block_in_channels,layer_counts=layers_counts,growth_rate=growth_rate)
block_out_channels = block_in_channels + layers_counts*growth_rate
transition = TransitionLayer(block_out_channels,block_out_channels//2)
if i != len(block_config): #最后一个dense block后没有transition layer
block_in_channels = block_out_channels // 2 #更新下一个dense block的in_channels

self.fc = nn.Linear(block_in_channels,num_classes)

def forward(self,x):
out = self.conv1(x)
out = self.pool1(x)
for layer in self.dense_block_layers:
out = layer(out)
# print(out.shape)
out = self.avg_pool(out)
out = torch.flatten(out,start_dim=1) #相当于out = out.view((x.shape[0],-1))
out = self.fc(out)
return out```

```X=torch.randn(1,3,224,224).cuda()
block_config = [6,12,24,16]
net = DenseNet(3,10,block_config)
net = net.cuda()
out = net(X)
print(out.shape)```

```torch.Size([1, 195, 112, 112])
torch.Size([1, 97, 56, 56])
torch.Size([1, 481, 56, 56])
torch.Size([1, 240, 28, 28])
torch.Size([1, 1008, 28, 28])
torch.Size([1, 504, 14, 14])
torch.Size([1, 1016, 14, 14])
torch.Size([1, 508, 7, 7])
torch.Size([1, 10])```