Project Code

def conv(in_channels, out_channels, kernel_size=3, padding=1, bn=True, dilation=1, stride=1, relu=True, bias=True):
    modules = [nn.Conv2d(in_channels, out_channels, kernel_size, stride, padding, dilation, bias=bias)]
    if bn:
        modules.append(nn.BatchNorm2d(out_channels))
    if relu:
        modules.append(nn.ReLU(inplace=True))
    return nn.Sequential(*modules)

def conv_dw(in_channels, out_channels, kernel_size=3, padding=1, stride=1, dilation=1):
    return nn.Sequential(
        nn.Conv2d(in_channels, in_channels, kernel_size, stride, padding, dilation=dilation, groups=in_channels, bias=False),
        nn.BatchNorm2d(in_channels),
        nn.ReLU(inplace=True),

        nn.Conv2d(in_channels, out_channels, 1, 1, 0, bias=False),
        nn.BatchNorm2d(out_channels),
        nn.ReLU(inplace=True),
    )

def conv_dw_no_bn(in_channels, out_channels, kernel_size=3, padding=1, stride=1, dilation=1):
    return nn.Sequential(
        nn.Conv2d(in_channels, in_channels, kernel_size, stride, padding, dilation=dilation, groups=in_channels, bias=False),
        nn.ELU(inplace=True),

        nn.Conv2d(in_channels, out_channels, 1, 1, 0, bias=False),
        nn.ELU(inplace=True),
    )

These functions define different types of convolutional layers in PyTorch, each with specific characteristics:

1. conv

• Purpose: A standard convolutional layer with optional batch normalization (bn) and ReLU activation (relu).
• Key Features:
  • Uses a single nn.Conv2d layer.
  • Batch normalization (nn.BatchNorm2d) is applied if bn=True.
  • ReLU activation (nn.ReLU) is applied if relu=True.
  • Configurable parameters like kernel size, padding, stride, dilation, and bias.
• Use Case: General-purpose convolutional layers for feature extraction.

2. conv_dw

• Purpose: A depthwise separable convolution layer.
• Key Features:
  • First, performs a depthwise convolution using groups=in_channels in nn.Conv2d. This applies a separate convolutional filter to each input channel.
  • Then, applies batch normalization (nn.BatchNorm2d) and ReLU activation (nn.ReLU).
  • Finally, performs a pointwise convolution (1x1 kernel) to combine the outputs of the depthwise convolution.
  • Includes batch normalization and ReLU activation after both convolutions.
• Use Case: Efficient convolutional layers that reduce computational cost and parameters, commonly used in lightweight models like MobileNet.

3. conv_dw_no_bn

• Purpose: A depthwise separable convolution layer without batch normalization, using ELU activation instead of ReLU.
• Key Features:
  • Similar to conv_dw, but:
    • Does not include batch normalization layers.
    • Uses nn.ELU (Exponential Linear Unit) activation instead of nn.ReLU.
  • Designed for scenarios where batch normalization is not desired or necessary.
• Use Case: Lightweight convolutional layers for specific tasks where ELU activation is preferred, or batch normalization is avoided (e.g., memory-constrained environments).

Summary of Differences: