torch.addcmul
function addcmul(input: unknown, tensor1: unknown, tensor2: unknown, valueOrOptions: unknown, options: unknown): TensorPerforms element-wise multiplication then adds to input: result = input + value * (tensor1 * tensor2).
Efficiently combines scaled multiplication with accumulation. Essential for:
- Optimizer implementations: SGD with momentum, Adam (m update), gradient accumulation
- Momentum/velocity: Exponential moving averages in optimizers
- Gate mechanisms: Applying learned gates (attention, multiplicative attention)
- Fused operation: Combined multiplication+addition for efficiency
- Memory efficient: No intermediate tensor from multiplication
- Commutative: tensor1 * tensor2 = tensor2 * tensor1
Parameters
inputunknown- The input tensor (accumulator)
tensor1unknown- The first tensor to multiply
tensor2unknown- The second tensor to multiply
valueOrOptionsunknown- Scaling factor (default: 1) or options object
optionsunknown- Optional settings including
outandvalueparameters
Returns
Tensor– Tensor with shape = broadcast(input, tensor1*tensor2)Examples
// Basic usage
const input = torch.tensor([10, 20, 30]);
const t1 = torch.tensor([2, 3, 4]);
const t2 = torch.tensor([5, 2, 1]);
torch.addcmul(input, t1, t2); // [20, 26, 34]
// Adam first moment update
const m = torch.zeros([100]);
const grad = torch.randn([100]);
const beta1 = 0.9;
m.mul_(beta1);
torch.addcmul(m, grad, torch.ones_like(grad), 1 - beta1);See Also
- PyTorch torch.addcmul()
- addcdiv - Similar with division instead of multiplication
- mul - Simple multiplication