Project Code

def get_parameters(model, predicate):
    for module in model.modules():
        for param_name, param in module.named_parameters():
            if predicate(module, param_name):
                yield param

def get_parameters_conv(model, name):
    return get_parameters(model, lambda m, p: isinstance(m, nn.Conv2d) and m.groups == 1 and p == name)

def get_parameters_conv_depthwise(model, name):
    return get_parameters(model, lambda m, p: isinstance(m, nn.Conv2d)
                                              and m.groups == m.in_channels
                                              and m.in_channels == m.out_channels
                                              and p == name)

def get_parameters_bn(model, name):
    return get_parameters(model, lambda m, p: isinstance(m, nn.BatchNorm2d) and p == name)

The provided code defines three utility functions (get_parameters_conv, get_parameters_conv_depthwise, and get_parameters_bn) that are used to filter and retrieve specific parameters from a PyTorch model based on certain conditions. These functions rely on the generic get_parameters function, which iterates over all modules and their parameters in a model.

1. get_parameters

def get_parameters(model, predicate):
    for module in model.modules():
        for param_name, param in module.named_parameters():
            if predicate(module, param_name):
                yield param

• Purpose: This is a generic function that iterates over all modules and their parameters in a PyTorch model and yields parameters that satisfy a given condition (predicate).
• Parameters:
  • model: The PyTorch model to search through.
  • predicate: A function that takes a module and a parameter name as input and returns True if the parameter should be selected.
• How It Works:
  1. Iterates over all modules in the model using model.modules().
  2. For each module, iterates over its named parameters using module.named_parameters().
  3. Applies the predicate function to determine if the parameter should be selected.
  4. If the condition is met, the parameter is yielded.

2. get_parameters_conv

def get_parameters_conv(model, name):
    return get_parameters(model, lambda m, p: isinstance(m, nn.Conv2d) and m.groups == 1 and p == name)

• Purpose: Retrieves parameters from standard convolutional layers (nn.Conv2d) in the model.
• Parameters:
  • model: The PyTorch model to search through.
  • name: The name of the parameter to filter (e.g., "weight" or "bias").
• How It Works:
  • Uses get_parameters with a predicate that checks:
    1. The module is an instance of nn.Conv2d.
    2. The convolution is a standard convolution (i.e., groups == 1).
    3. The parameter name matches the given name.
• Use Case: Selects parameters (e.g., weights or biases) from standard convolutional layers.

3. get_parameters_conv_depthwise

def get_parameters_conv_depthwise(model, name):
    return get_parameters(model, lambda m, p: isinstance(m, nn.Conv2d)
                                              and m.groups == m.in_channels
                                              and m.in_channels == m.out_channels
                                              and p == name)