Skip to main content
torch.js has not been released yet.
torch.js logotorch.js logotorch.js
PlaygroundContact
Login
Documentation
IntroductionType SafetyTensor ExpressionsTensor IndexingEinsumEinopsAutogradTraining a ModelProfiling & MemoryPyTorch MigrationBest PracticesRuntimesPerformancePyTorch CompatibilityBenchmarksDType Coverage
absabs_acosacos_acoshacosh_addadd_addbmmaddbmm_addcdivaddcdiv_addcmuladdcmul_addmmaddmm_addmvaddmv_addraddr_allamaxaminangleanyAnyTensorarccosarccos_arccosharccosh_arcsinarcsin_arcsinharcsinh_arctanarctan_arctan2arctanharctanh_argmaxargminargsortargwhereas_stridedasinasin_asinhasinh_atatanatan_atan2atan2_atanhatanh_backwardbaddbmmbaddbmm_bernoullibernoulli_bitwise_andbitwise_and_bitwise_left_shiftbitwise_left_shift_bitwise_notbitwise_not_bitwise_orbitwise_or_bitwise_right_shiftbitwise_right_shift_bitwise_xorbitwise_xor_bmmboolbytecauchy_ceilceil_charcholeskycholesky_inversecholesky_solvechunkclampclamp_clipclip_cloneconjconj_physicalcontiguouscopy_copysign_coscos_coshcosh_count_nonzerocpucrosscudacummaxcummincumprod_cumsum_cumulative_trapezoidcustom')]deg2raddeg2rad_destroydetdetachdetach_diagdiag_embeddiagflatdiagonaldiagonal_scatterdigamma_dimdivdiv_dividedivide_dotdoubleeigeigheigvalseigvalshelement_sizeeluelu_eqeq_erferf_erfcerfc_erfinverfinv_expexp_exp2exp2_expandexpand_asexpm1expm1_exponential_fftfill_fix_flattenflipfliplrflipudfloatfloat_powerfloorfloor_floor_divide_fmod_frac_frexpgathergcdgcd_gege_gelugelu_geometric_gergetCPUStorageglugreatergreater_equalgtgt_halfhardsigmoidhardsigmoid_hardswishhardswish_heavisideheaviside_hypothypot_i0_ifftindex_addindex_add_index_copyindex_copy_index_fillindex_fill_index_putindex_put_index_reduceindex_selectinnerintinverseis_complexis_contiguousis_floating_pointis_inferenceis_pinnedis_sharedis_signedisCPUisfiniteisinisinfisnanisneginfisposinfisrealisWebGPUitemkronkthvaluelcmlcm_ldexpldexp_lele_leaky_reluleaky_relu_lerplerp_lessless_equallgamma_loglog_log_normal_log10log10_log1plog1p_log2log2_logaddexplogaddexp2logcumsumexplogdetlogical_andlogical_and_logical_notlogical_not_logical_orlogical_or_logical_xorlogical_xor_logitlogit_logsumexplonglstsqltlt_lulu_solvemasked_fillmasked_fill_masked_scattermasked_scatter_masked_selectmasked_select_asyncmatmulmatrix_expmatrix_powermatrix_rankmaxmaximummeanmedianminminimummishmish_mmmodemsortmulmul_multiplymultiply_mvnan_to_numnanmeannanmediannanquantilenansumnarrow_copyndimensionnene_neg_negative_newnew_emptynew_fullnew_onesnew_tensornew_zerosnextafternextafter_nonzeronormnormal_not_equalnumelnumpyones_likeouterpermutepinversepolygammapositivepowpow_prodputput_qrquantilerad2degrad2deg_random_reciprocalreciprocal_register_hookregister_post_accumulate_grad_hookrelurelu_remainder_repeat_interleavereshapereshape_asretain_gradrfftrot90roundround_rsqrtrsqrt_scatterscatter_scatter_addscatter_add_scatter_reducescatter_reduce_selectselect_scatterseluselu_sgnsgn_shortsigmoidsigmoid_signsign_signbitsilusilu_sinsin_sincsinc_sinhsinh_sizesliceslice_scatterslogdetsoftplussoftplus_softsignsoftsign_solvesortsplitsqrtsqrt_squaresquare_squeezesqueeze_stdstorage_offsetstridesubsub_subtractsubtract_sumsvdtt_take_along_dimtantan_tanhtanh_tanhshrinktanhshrink_Tensortensor_splitTensorInternalsthresholdthreshold_totolisttopktoStringtracetransposetranspose_trapezoidtriangular_solvetrigammatriltril_triutriu_true_dividetrue_divide_trunc_typetype_asunflattenuniform_uniqueunique_consecutiveunsqueezeunsqueeze_varvdotviewview_aswebgpuwherexlogyxlogy_zero_zeros_like
absacosacoshAdaptivePool1dShapeAdaptivePool2dShapeaddaddbmmAddbmmOptionsaddcdivAddcdivOptionsaddcmulAddcmulOptionsaddmmAddmmOptionsaddmvAddmvOptionsaddrAddrOptionsadjointallallcloseAllcloseOptionsAlphaBetaOptionsamaxaminaminmaxAminmaxOptionsangleanyapplyOutarangeare_deterministic_algorithms_enabledargmaxargminargsortargwhereas_stridedas_tensorasinasinhAssertNoShapeErrorAssertNotErrorAsStridedOptionsAtat_error_index_out_of_boundsatanatan2atanhatleast_1datleast_2datleast_3dAtShapeautocast_decrement_nestingautocast_increment_nestingautograd_gradient_mismatch_errorautograd_not_registered_errorAutogradConfigAutogradDeviceAutogradDTypeAutogradEntryAutogradHandleAutogradHandleImplAxesRecordBackwardFnbaddbmmBaddbmmOptionsbartlett_windowBaseKernelConfigbatch_dimensions_do_not_match_errorbernoulliBernoulliOptionsBinaryBackwardFnBinaryBroadcastResultBinaryDTypeBinaryKernelConfigCPUBinaryKernelCPUBinaryOpConfigBinaryOpNamesBinaryOpSchemaBinaryOptionsbincountBincountOptionsbitwise_andbitwise_left_shiftbitwise_notbitwise_orbitwise_right_shiftbitwise_xorblackman_windowblock_diagbmmBooleanDTypeRulebroadcast_error_incompatible_dimensionsbroadcast_shapesbroadcast_tensorsbroadcast_toBroadcastShapeBroadcastShapeRulebroadcastShapesbucketizeBucketizeOptionsBufferUsagebuildEinopsErrorbuildErrorMessagecanBroadcastTocartesian_prodcatCatOptionsCatShapeCauchyOptionscdistCdistOptionsceilceluCeluFunctionalOptionschain_matmulCheckShapeErrorCholeskyShapechunkchunk_error_dim_out_of_rangeChunkOptionsclampClampOptionsclear_autocast_cacheclearEinopsCacheclearEinsumCacheclonecolumn_stackcombinationsCombinationsOptionscompiled_with_cxx11_abicomplexconjconj_physicalcontiguousConv1dShapeConv2dShapeConv3dShapeConvTranspose2dShapecopysigncorrcoefcoscoshcount_nonzeroCountNonzeroOptionscovcoverage_reportcoverageReportCoverageReportCovOptionsCPUForwardFnCPUKernelConfigCPUKernelEntryCPUOnlyResultCPUTensorDatacreateCumExtremeResultcreateTorchCreationOpSchemaCumExtremeResultcummaxcummincumprodCumShapecumsumcumulative_trapezoidCumulativeOptionsCumulativeOptionsWithDimdeg2raddetachDeterministicOptionsDetShapeDevicedevice_error_requiresDeviceBufferDeviceCapabilitiesDeviceCheckedResultDeviceConfigDeviceContextDeviceEntryDeviceHandleDeviceInputDeviceOptionsDeviceRegistryDeviceTypediagdiag_embedDiagEmbedOptionsdiagflatDiagflatOptionsDiagFlatOptionsdiagonal_scatterDiagonalOptionsDiagonalScatterOptionsDiagOptionsDiagShapediffDiffOptionsdigammadimension_error_out_of_rangeDispatchConfigdistDistOptionsdivdotDotShapeRuleDoubleDoubleDimdropoutDropoutFunctionalOptionsdsplitdstackDTypedtype_already_registered_errordtype_components_mismatch_errordtype_not_found_errorDTypeComponentsDTypeConfigDTypeCoverageReportDTypeDisplayConfigDTypeEntryDTypeHandleDTypeHandleImplDTypeInfoDTypeRegistryDTypeRuleDTypeSerializationConfigDynamicShapeEigShapeeinops_error_ambiguous_decompositioneinops_error_anonymous_in_outputeinops_error_dimension_mismatcheinops_error_invalid_patterneinops_error_reduce_undefined_outputeinops_error_repeat_missing_sizeeinops_error_undefined_axiseinsumeinsum_error_dimension_mismatcheinsum_error_index_out_of_rangeeinsum_error_invalid_equationeinsum_error_invalid_sublist_elementeinsum_error_operand_count_mismatcheinsum_error_subscript_rank_mismatcheinsum_error_unknown_output_indexEinsumOptionsEinsumOutputShapeEllipsiseluelu_EluFunctionalOptionsembedding_bag_error_requires_2d_inputemptyempty_cacheempty_likeeqequalerferfcerfinvexpexp2expandexpand_asexpand_error_incompatibleExpandShapeexpm1ExponentialOptionseyeEyeOptionsfftFFTOptionsfindKernelWithPredicatefindSimilarPatternsflattenFlattenOptionsFlattenShapeflipflip_error_dim_out_of_rangefliplrFlipShapeflipudfloat_powerFloatDTypeRulefloorfloor_dividefmaxfminfmodformatEquationErrorformatShapefracfrexpfrombufferfullfull_likefunction_already_registered_errorFunctionConfigFunctionEntryFunctionHandlegathergather_error_dim_out_of_rangeGatherShapegcdgegeluGeometricOptionsget_autocast_cpu_dtypeget_autocast_gpu_dtypeget_autocast_ipu_dtypeget_autocast_xla_dtypeget_default_deviceget_default_dtypeget_deterministic_debug_modeget_device_configget_device_contextget_device_moduleget_dtype_infoget_file_pathget_float32_matmul_precisionget_num_interop_threadsget_num_threadsget_op_infoget_printoptionsget_real_dtypeget_rng_stategetAutogradgetDTypegetEinopsCacheSizegetEinsumCacheSizegetFunctiongetKernelgetMethodgetOpInfoGetOpKindGetOpSchemagetScalarKernelgluGluFunctionalOptionsGradContextGradFnGradientsForgtHalfHalfDimhamming_windowhann_windowhardshrinkhardsigmoidhardswishhardtanhhardtanh_HardtanhFunctionalOptionshas_autogradhas_devicehas_dtypehas_kernelhasAutogradhasDTypehasFunctionhasKernelhasMethodhasScalarKernelHasShapeErrorheavisidehistcHistcOptionshistogramHistogramOptionsHistogramResulthsplithstackhypoti0IdentityShapeifftimagindex_addindex_copyindex_fillindex_putindex_reduceindex_selectindex_select_error_dim_out_of_rangeIndexPutOptionsIndexSelectShapeIndexSpecIndicesOptionsIndicesSpecinitialize_deviceInputsForInsertDiminvalid_config_errorinverseInverseShapeirfftis_anomaly_check_nan_enabledis_anomaly_enabledis_autocast_cache_enabledis_autocast_cpu_enabledis_autocast_ipu_enabledis_autocast_xla_enabledis_complexis_complex_dtypeis_cpu_only_modeis_deterministic_algorithms_warn_only_enabledis_floating_pointis_floating_point_dtypeis_inference_mode_enabledis_nonzerois_tensoris_warn_always_enabledis_webgpu_availableIs2DIsAtLeast1DIsBinaryOpIsBinaryOpNameiscloseIscloseOptionsisfiniteisinisinfisnanisneginfisposinfisrealIsReductionOpIsReductionOpNameIsRegistryErrorIsShapeErroristftISTFTOptionsIsUnaryOpIsUnaryOpNameitem_error_not_scalarItemResultkaiser_windowKaiserWindowOptionskernel_not_registered_errorkernel_signature_mismatch_errorKernelConfigKernelConfigWebGPUKernelEntryKernelHandleKernelInfoKernelPredicateKernelRegistryKernelWebGPUkronkthvalueKthvalueOptionslcmldexpleleaky_reluleaky_relu_LeakyReluFunctionalOptionslerplevenshteinDistancelgammalinalg_error_not_square_matrixlinalg_error_requires_2dlinalg_error_requires_at_least_2dlinearlinspacelist_custom_deviceslist_custom_dtypeslist_deviceslist_dtypeslist_functionslist_kernelslist_methodslist_opslistCustomDTypeslistDTypeslistFunctionslistKernelsListKernelsOptionslistMethodslistOpsListOpsOptionsloglog_softmaxlog10log1plog2logaddexplogaddexp2logcumsumexplogical_andlogical_notlogical_orlogical_xorLogitOptionsLogNormalOptionsLogOptionslogsigmoidlogspacelogsumexpLogsumexpOptionsltLUShapeLuSolveOptionsmasked_fillmasked_selectmasked_select_asyncMaskSpecmatmulmatmul_error_inner_dimensions_do_not_matchMatmul2DShapeMatmulShapeMatmulShapeRuleMatrixTransposeShapemaxmaximummeanmedianmemory_statsmemory_summarymeshgridmethod_already_registered_errormethod_dtype_not_supported_errorMethodConfigMethodEntryMethodHandleminminimummishmmMMShapeRulemodemovedimmsortmulmultinomialmultinomial_asyncMultinomialAsyncOptionsMultinomialOptionsMultiplyBymvMVShapeRulenan_to_numnanmeannanmediannanquantileNanReductionOptionsnansumNanToNumOptionsnarrownarrow_copynarrow_error_length_exceeds_boundsnarrow_error_start_out_of_boundsNarrowShapeneneedsBroadcastnegNegativeDimnextafternonzeroNonzeroOptionsnormnormalNormalOptionsNormOptionsnumelonesones_likeop_kind_mismatch_errorop_not_found_errorOpCoverageEntryOpInfoOpKindOpNameOpSchemaOpSchemasouterOuterShapepackPackShapepermutepermute_error_dimension_count_mismatchPermuteShapepoissonpolarPool1dShapePool2dShapePool3dShapepositivepowpreluPrintOptionsprodprofiler_allow_cudagraph_cupti_lazy_reinit_cuda12promote_typesPromoteDTypeRulePutOptionsquantileQuantileOptionsrad2degrandrand_likerandintrandint_likeRandintLikeOptionsRandintOptionsrandnrandn_likeRandomLikeOptionsRandomOptionsrandpermRangeSpecRankravelrealrearrangeRearrangeOptionsRearrangeShapereciprocalreduceReduceOperationReduceOptionsReduceShapeReductionKernelConfigCPUReductionKernelCPUReductionOpNamesReductionOpSchemaReductionOptionsReductionShapeRuleregister_backwardregister_deviceregister_dtyperegister_forwardregister_functionregister_methodregister_scalar_forwardregisterAutogradRegisterBackwardOptionsregisterBinaryOpregisterDTypeRegisterDTypeOptionsRegisteredDTyperegisterFunctionRegisterFunctionOptionsregisterKernelRegisterKernelOptionsregisterMethodRegisterMethodOptionsregisterScalarKernelregisterUnaryOpregistration_failed_errorrelurelu_relu6ReluFunctionalOptionsremainderRemoveDimrepeatrepeat_interleaveRepeatInterleaveOptionsRepeatOptionsRepeatShapeReplaceDimrequireWebGPUreset_peak_memory_statsreshapeReshapeShaperesult_typerfftrollRollOptionsrot90Rot90Optionsroundrrelurrelu_RreluFunctionalOptionsrsqrtSafeExpandShapeSameDTypeRuleSameShapeRuleSaveForBackwardScalarCPUForwardFnScalarCPUKernelConfigScalarKernelEntryScalarKernelHandleScalarWebGPUKernelConfigScaleDimscatterscatter_addscatter_add_scatter_error_dim_out_of_rangescatter_reducescatter_reduce_ScatterReduceOptionsScatterShapesearchsortedSearchSortedOptionsselectselect_error_index_out_of_boundsselect_scatterSelectShapeseluset_default_deviceset_default_tensor_typeset_deterministic_debug_modeset_float32_matmul_precisionset_printoptionsset_warn_alwaysSetupContextFnShapeShapeCheckedResultShapedTensorShapeErrorMessageShapeOpSchemaShapeRulesigmoidsignsignbitsilusinsincsinhSizeOptionsslice_error_out_of_boundsslice_scatterSliceOptionsSliceScatterOptionsSliceShapeSliceSpecsoftmaxsoftmax_error_dim_out_of_rangeSoftmaxShapesoftminSoftminFunctionalOptionssoftplusSoftplusFunctionalOptionssoftshrinksoftsignsortSortOptionssplitsplit_error_dim_out_of_rangeSplitOptionssqrtsquaresqueezeSqueezeOptionsSqueezeShapestackStackOptionsStackShapestdstd_meanStdVarMeanOptionsStdVarOptionsstftSTFTOptionsStrideOptionssubSublistSublistElementSubscriptIndexsumSVDShapeswapaxessym_floatsym_intsym_notttaketake_along_dimTakeAlongDimOptionstantanhtanhshrinktensortensor_splitTensorCreatorTensorDatatensordotTensordotOptionsTensorLikeTensorMetaTensorOptionsTensorStoragethresholdthreshold_tileTileShapeToOptionstopkTopkOptionsTorchtraceTraceShapetransposetranspose_dims_error_out_of_rangetranspose_error_requires_2d_tensorTransposeDimsShapeTransposeDimsShapeCheckedTransposeShapetrapezoidTrapezoidOptionsTriangularOptionstriltril_indicesTriOptionsTripletriutriu_indicestrue_dividetruncTupleOfLengthTypedArrayTypedArrayForTypedStorageTypeOptionsUnaryBackwardFnUnaryDTypeUnaryKernelConfigCPUUnaryKernelCPUUnaryOpConfigUnaryOpFnUnaryOpNamesUnaryOpParamsUnaryOpSchemaUnaryOptionsunbindunbind_error_dim_out_of_rangeUnbindOptionsunflattenUniformOptionsuniqueunique_consecutiveUniqueConsecutiveOptionsUniqueOptionsunpackUnpackShapeunravel_indexunregister_deviceunsqueezeUnsqueezeOptionsUnsqueezeShapeuse_deterministic_algorithmsValidateBatchedSquareMatrixValidateChunkDimValidatedEinsumShapevalidateDeviceValidateDeviceValidatedRearrangeShapeValidatedReduceShapeValidatedRepeatShapevalidateDTypeValidateEinsumValidateOperandCountValidateRanksValidateScalarValidateSplitDimValidateSquareMatrixValidateUnbindDimValueOptionsvar_var_meanvdotviewview_as_complexview_as_realvmapvsplitvstackWebGPUKernelConfigWebGPUOnlyResultWebGPUTensorDatawhereWindowOptionsxlogyzeroszeros_like
torch.js· 2026
LegalTerms of UsePrivacy Policy
/
/
  1. docs
  3. torch.js
  5. torch
  7. Tensor
  9. Tensor
  11. hardswish

torch.Tensor.Tensor.hardswish

Tensor.hardswish(): Tensor<S, D, Dev>

Hard Swish activation function.

Efficient approximation of SiLU (Swish) using hard sigmoid instead of smooth sigmoid. Replaces the expensive exp(x) in sigmoid with piecewise linear clipping. Originally introduced in MobileNetV3 to speed up inference on mobile devices.

Definition: HardSwish(x) = x * HardSigmoid(x), where HardSigmoid uses efficient clipping. Combines the benefits of SiLU's smooth gating with sigmoid's computational efficiency.

Use Cases:

  • Mobile inference (MobileNetV3, EfficientNet)
  • Edge devices requiring sub-millisecond latency
  • Real-time inference in resource-constrained environments
  • On-device ML (phones, IoT, embedded systems)
  • Gating mechanisms when speed is critical
HardSwish(x)=x⋅HardSigmoid(x)=x⋅{0if x≤−3x6+12if −3<x<31if x≥3Simplified: HardSwish(x)={0if x≤−3x26+x2if −3<x<3xif x≥3\begin{aligned} \text{HardSwish}(x) = x \cdot \text{HardSigmoid}(x) = x \cdot \begin{cases} 0 & \text{if } x \le -3 \\ \frac{x}{6} + \frac{1}{2} & \text{if } -3 < x < 3 \\ 1 & \text{if } x \ge 3 \end{cases} \\ \text{Simplified: } \text{HardSwish}(x) = \begin{cases} 0 & \text{if } x \le -3 \\ \frac{x^2}{6} + \frac{x}{2} & \text{if } -3 < x < 3 \\ x & \text{if } x \ge 3 \end{cases} \end{aligned}HardSwish(x)=x⋅HardSigmoid(x)=x⋅⎩⎨⎧​06x​+21​1​if x≤−3if −3<x<3if x≥3​Simplified: HardSwish(x)=⎩⎨⎧​06x2​+2x​x​if x≤−3if −3<x<3if x≥3​​
  • 10-100x faster than SiLU (no exponential computation).
  • Self-gating: Like Swish, gates input by activation value.
  • Output range: [-3, +∞) - unbounded like SiLU.
  • Negative values: Allows negative outputs unlike hardsigmoid.
  • MobileNet: Official activation for MobileNetV3 and derivatives.
  • Smooth behavior: Provides smooth activation within [-3, 3] range.
  • Gradient is zero for x ≤ -3 (dead neurons possible).
  • Large positive values not saturated (unlike sigmoid).
  • Not recommended for server-side deep models (use SiLU instead).

Returns

Tensor<S, D, Dev>– Tensor with same shape as input

Examples

// Basic usage - efficient SiLU approximation
const x = torch.tensor([-4, -2, 0, 2, 4]);
x.hardswish();  // [0, ~-0.333, 0, ~0.667, 4]

// MobileNetV3 backbone - ultra-fast inference
const mobilenet_v3 = MobileNetV3();  // Uses hardswish internally
const image = torch.randn(1, 224, 224, 3);
const features = mobilenet_v3.forward(image);

// Edge device latency-critical inference
const tflite_model = torch.load('model.tflite');  // Converted to hardswish
const latency_critical = true;  // Must respond in < 50ms

// Gating with speed requirements
const input = torch.randn(16, 512);
const gate = input.hardswish();  // Fast gating without exp

// Speed comparison
const silu_time = benchmark(() => input.silu());     // ~100µs
const hardswish_time = benchmark(() => input.hardswish()); // ~10µs (10x faster)

See Also

  • PyTorch torch.nn.functional.hardswish()
  • silu - Smooth version (Swish): x * sigmoid(x)
  • hardsigmoid - Hard sigmoid (used internally)
  • relu - Simpler alternative
  • gelu - Better quality (slower)
Previous
Tensor.hardsigmoid_
Next
Tensor.hardswish_