torch.nn.ModuleList
class ModuleList extends Modulenew ModuleList(options?: ModuleListOptions)
- readonly
length(number) - – Get the number of modules in the list.
Holds submodules in a list with automatic parameter registration.
ModuleList stores modules in a Python list-like container. Unlike building a model
by composing modules as attributes (e.g., this.layer1 = Linear(...)), ModuleList
allows dynamic construction and modification of layer sequences. Essential for:
- Stacking variable numbers of layers (determined at runtime)
- Sequential processing of layer sequences (encoder stacks, decoder stacks)
- Conditional layer selection based on input or configuration
- Dynamic model construction (pruning, adaptation, curriculum learning)
- Iterating over layers and applying the same operation to all
Key difference from array of modules: ModuleList automatically registers submodules for proper parameter tracking, gradient computation, device movement, and training/eval mode propagation. Plain Python/TypeScript arrays won't track parameters correctly.
When to use ModuleList:
- Variable-depth networks (number of layers determined at runtime)
- Sequential stacks where each layer processes previous layer's output
- Encoder/decoder with many similar layers (transformers, ResNets with many blocks)
- Dynamic architectures that add/remove layers during execution
- Simpler than ModuleDict when you don't need named access
When NOT to use:
- Fixed layer counts - use attributes (this.fc1, this.fc2, etc.)
- Complex interconnection patterns - might be clearer with attributes
- Named module access preferred - use ModuleDict instead
- Parameter tracking: All modules in list are automatically tracked for parameters/buffers
- Device movement: .to(device) applies to all submodules
- Training mode: .train()/.eval() propagates to all submodules
- Iteration: Use for...of to iterate over modules
- No forward method: Must manually implement forward() - ModuleList doesn't have it
- Index-based access: get(i) or index operator (if supported)
- Dynamic append: Use append() to add modules during construction
Examples
// Build a ResNet-style stack with variable depth
class ResNetWithModuleList extends torch.nn.Module {
layers: torch.nn.ModuleList;
constructor(num_blocks: number, num_features: number) {
super();
this.layers = new torch.nn.ModuleList();
for (let i = 0; i < num_blocks; i++) {
const block = new ResNetBlock(num_features);
this.layers.append(block);
}
}
forward(x: torch.Tensor): torch.Tensor {
for (const layer of this.layers) {
x = (layer).forward(x);
}
return x;
}
}
const model = new ResNetWithModuleList(50, 64); // 50 blocks// Transformer encoder stack
class TransformerEncoder extends torch.nn.Module {
layers: torch.nn.ModuleList;
norm: torch.nn.LayerNorm;
constructor(num_layers: number, d_model: number, nhead: number) {
super();
this.layers = new torch.nn.ModuleList();
for (let i = 0; i < num_layers; i++) {
const layer = new torch.nn.TransformerEncoderLayer(
d_model,
nhead,
2048,
0.1,
'relu'
);
this.layers.append(layer);
}
this.norm = new torch.nn.LayerNorm(d_model);
}
forward(x: torch.Tensor): torch.Tensor {
for (const layer of this.layers) {
x = (layer).forward(x);
}
return this.norm.forward(x);
}
}// MLP with dynamic width and depth
class DynamicMLP extends torch.nn.Module {
layers: torch.nn.ModuleList;
constructor(input_size: number, hidden_sizes: number[]) {
super();
this.layers = new torch.nn.ModuleList();
let in_size = input_size;
for (const hidden_size of hidden_sizes) {
this.layers.append(new torch.nn.Linear(in_size, hidden_size));
in_size = hidden_size;
}
}
forward(x: torch.Tensor): torch.Tensor {
for (let i = 0; i < this.layers.length; i++) {
x = (this.layers.get(i)).forward(x);
if (i < this.layers.length - 1) {
x = torch.nn.functional.relu(x);
}
}
return x;
}
}
const mlp = new DynamicMLP(784, [512, 256, 128, 10]);