torch.bernoulli
function bernoulli<S extends Shape, D extends DType = DType, Dev extends DeviceType = DeviceType>(input: Tensor<S, D, Dev>): Tensor<S, D, Dev>Draws independent binary random numbers from a Bernoulli distribution.
For each element p in the input tensor, returns 1 with probability p and 0 with probability 1-p. Each element is sampled independently. Essential for:
- Stochastic regularization: Dropout and other noise-based techniques
- Probabilistic modeling: Generating binary data and binary masks
- Data augmentation: Randomly masking or selecting features
- Monte Carlo simulation: Simulating coin flips and binary events
- Feature selection: Randomly enabling/disabling neurons or features
- Binary matrix generation: Creating random binary masks for attention
Implementation: For each probability p, samples u ~ Uniform(0,1) and returns 1 if u < p, else 0. Guarantees: Each sample is independent, results are exactly 0 or 1 (no intermediate values).
- Probability range: Input values must be in [0, 1]. Values outside this range may produce unexpected results
- Independence: Each element is sampled independently
- Exact binary: Output is exactly 0 or 1 (no rounding or interpolation)
- Shape preservation: Output has same shape as input
- Expected value: E[Bernoulli(p)] = p; useful for importance weighting
- GPU support: Works on both CPU and GPU backends
- Dropout connection: Related to dropout; can implement dropout with bernoulli + scaling
- Probability validation: Values 1 or 0 produce undefined behavior
- No gradient flow: Like all sampling operations, bernoulli is non-differentiable
- Random seed: Results depend on global random seed; use manual_seed for reproducibility
- Batch independence: Samples across batch dimension are independent
Parameters
inputTensor<S, D, Dev>- Tensor of probabilities with shape (...) and values in [0, 1]
Returns
Tensor<S, D, Dev>– Tensor with same shape as input, containing binary values (0 or 1)Examples
// Simple binary sampling
const probs = torch.tensor([0.3, 0.7, 0.5]);
const samples = torch.bernoulli(probs); // e.g., [0, 1, 1]
// Dropout: randomly mask activations
const activations = torch.randn(batch_size, hidden_dim);
const dropout_rate = 0.5;
const keep_prob = torch.ones(batch_size, hidden_dim).mul(1 - dropout_rate);
const mask = torch.bernoulli(keep_prob);
const dropped = activations.mul(mask).div(1 - dropout_rate); // Scale to maintain expectation
// Binary feature selection
const features = torch.randn(100, 50); // 100 samples, 50 features
const feature_select_prob = 0.8; // Keep each feature with 80% probability
const feature_mask = torch.bernoulli(torch.ones(50).mul(feature_select_prob));
const selected_features = features.mul(feature_mask); // Zero out unselected features
// Stochastic data augmentation
const image = torch.randn(3, 32, 32); // Image tensor
const flip_prob = torch.full([1], 0.5); // 50% chance to flip
const should_flip = torch.bernoulli(flip_prob).bool();
const augmented = should_flip ? image.flip(-1) : image; // Horizontal flip
// 2D binary matrix (random edges in graph)
const edge_prob = 0.3; // 30% chance of edge between any two nodes
const adj_matrix = torch.bernoulli(torch.full([100, 100], edge_prob)); // Random 100×100 graph
// Batched binary sampling
const batch_probs = torch.randn(32, 10).sigmoid(); // 32 samples, 10 features each
const binary_samples = torch.bernoulli(batch_probs); // [32, 10] of 0s and 1sSee Also
- PyTorch torch.bernoulli()
- randint - Generate random integers
- rand - Generate uniform random numbers in [0, 1)
- randn - Generate normally distributed random numbers
- poisson - Sample from Poisson distribution
- multinomial - Sample from multinomial distribution