torch.nn.functional.dropout1d
function dropout1d<S extends Shape, D extends DType = DType, Dev extends DeviceType = DeviceType>(input: Tensor<S, D, Dev>, options?: DropoutFunctionalOptions): Tensor<S, D, Dev>function dropout1d<S extends Shape, D extends DType = DType, Dev extends DeviceType = DeviceType>(input: Tensor<S, D, Dev>, p: number, training: boolean, inplace: boolean, options?: DropoutFunctionalOptions): Tensor<S, D, Dev>Randomly zeros out entire channels in 1D input (sequences).
Applies channel-wise dropout specifically designed for 1D sequential data. Instead of dropping individual elements, dropout1d drops entire channels (feature maps) consistently across all time steps. This preserves temporal coherence in sequence data and is more effective for regularizing RNNs, CNNs on 1D sequences, and Transformers. The same channel mask is applied to all positions in the sequence, preventing feature adaptation across time without corresponding temporal relationships.
Common use cases:
- 1D CNNs for sequences: Drop entire filters consistently across all time steps
- RNN regularization: Prevent channel co-adaptation in recurrent networks
- Sequential processing: Dropout that respects temporal structure (audio, time series)
- Transformer layers: Regularize attention outputs maintaining temporal dependencies
- Time series classification: Feature-level regularization for sequence models
- Signal processing networks: Dropout on frequency channels across time
- Sequence-to-sequence models: Regularization in encoder-decoder architectures
The key difference from element-wise dropout:
- Element-wise: Each value independently dropped (breaks temporal coherence)
- Channel-wise: Entire channels dropped consistently (preserves temporal relationships)
- Temporal consistency: Same features are dropped at all time steps for a given channel
Dimensionality handling:
- 1D input (L,): Element-wise dropout applied
- 2D input (C, L): Channel dropout per channel, shared across sequence
- 3D input (N, C, L): Batch-wise channel dropout (different mask per batch)
- Higher dims: Falls back to element-wise dropout
- Channel-wise consistency: The same channel is either dropped or kept across all sequence positions. This is fundamentally different from element-wise dropout
- Temporal preservation: Maintains temporal coherence by applying identical masks across time. Critical for sequence models where temporal relationships matter
- Scaling factor: Kept channels are automatically scaled by 1/(1-p) to maintain expected value. This is inverted dropout, preventing rescaling at inference
- Different per batch: With 3D input (N, C, L), each batch element gets an independent random channel mask, improving regularization effectiveness
- Higher dims fallback: Tensors with more than 3 spatial dimensions fall back to element-wise dropout. Use reshape/view if you need strict channel dropout
- Deterministic eval: Always set training=false during inference for reproducible results
- Don't mix with batch_norm: Using both channel dropout and batch norm can cause training instability. Consider using one or the other, not both
- Scaling is automatic: The 1/(1-p) scaling is applied automatically during training. Don't manually scale outputs - that would cause double-scaling
- RNN state dropout: For RNNs with hidden states, apply dropout after the RNN, not within it, to avoid disrupting state propagation across time steps
- Incompatible with layer normalization: Be careful combining with layer_norm as both operate on channels. Test combinations thoroughly for your architecture
- Channel dimension position: This function assumes channels are at dimension 1. Use permute/transpose if your data has channels in a different position
Parameters
inputTensor<S, D, Dev>- Input tensor. Typical shapes: - (N, C, L): Batch of sequences [batch, channels, length] - (C, L): Unbatched sequence [channels, length] - (L,): 1D sequence [length] Where: N=batch size, C=channels/features, L=sequence length
optionsDropoutFunctionalOptionsoptional
Returns
Tensor<S, D, Dev>– Tensor with same shape as input, with channels dropped and scaledExamples
// Basic 1D channel dropout for sequences
const sequences = torch.randn(32, 64, 100); // Batch of 32, 64 channels, length 100
const output = torch.nn.functional.dropout1d(sequences, 0.2, true);
// ~20% of the 64 channels are dropped entirely// 1D CNN for time series classification
const input = torch.randn(16, 32, 128); // 16 samples, 32 feature channels, 128 time steps
const conv_out = new torch.nn.Conv1d(32, 64, 3, 1, 1).forward(input);
const regularized = torch.nn.functional.dropout1d(conv_out, 0.3, model.training);
const pooled = torch.nn.functional.avg_pool1d(regularized, 2);
// Channel dropout applied consistently across all time steps// RNN layer with channel dropout
const x = torch.randn(10, 20, 128); // 10 batch, 20 time steps, 128 hidden units
const rnn_out = torch.randn(10, 20, 256); // RNN output
const dropped = torch.nn.functional.dropout1d(rnn_out, 0.5, true);
const final = new torch.nn.Linear(256, 10).forward(dropped.squeeze(1));
// Same channels dropped across all time steps// Unbatched sequence (2D input)
const sequence = torch.randn(64, 100); // 64 channels, 100 time steps
const output = torch.nn.functional.dropout1d(sequence, 0.1, true);
// output shape: [64, 100] with ~10% of channels masked// Training vs evaluation
const data = torch.randn(8, 32, 50);
const train_output = torch.nn.functional.dropout1d(data, 0.2, true);
// During training: 20% channels dropped, values scaled by 1.25
const eval_output = torch.nn.functional.dropout1d(data, 0.2, false);
// During evaluation: No dropout, returns input unchanged// Time series with different dropout rates
const timeseries = torch.randn(5, 128, 256); // 5 samples, 128 features, 256 time steps
const light = torch.nn.functional.dropout1d(timeseries, 0.05, true); // 5% drop rate
const moderate = torch.nn.functional.dropout1d(timeseries, 0.2, true); // 20% drop rate
const heavy = torch.nn.functional.dropout1d(timeseries, 0.4, true); // 40% drop rate
// Higher drop rates provide stronger regularization// Attention-based sequence model
const batch_size = 16, seq_len = 50, d_model = 512;
const attention_output = torch.randn(batch_size, d_model, seq_len);
const after_dropout = torch.nn.functional.dropout1d(attention_output, 0.1, true);
const layer_norm = new torch.nn.LayerNorm([d_model]).forward(
after_dropout.transpose(1, 2)
);
// Channel dropout preserves temporal dependencies in attentionSee Also
- PyTorch torch.nn.functional.dropout1d
- dropout2d - Channel dropout for 2D spatial data (height, width)
- dropout3d - Channel dropout for 3D volumetric data
- dropout - Element-wise dropout (not channel-wise)
- feature_alpha_dropout - SELU-compatible channel dropout
- batch_norm - Alternative regularization for sequences
- layer_norm - Per-channel normalization