Q/jddlZddlmZmZmZmZmZmZmZddl m Z ddl Z ddl m Z ddl mZmZmZgdZee jee jfZdZe dded ed ed ed eed e jf dZe de dded ed ed ed eed e jf dZe ddeded eed dfdZy)N)castDictIterableListOptionalTupleUnion) deprecated)Tensor)_device_has_foreach_support"_group_tensors_by_device_and_dtype_has_foreach_support)clip_grad_norm_clip_grad_normclip_grad_value_c>fd}tj||S)z This wrapper is needed to avoid a circular import when using @torch.no_grad on the exposed functions clip_grad_norm_ and clip_grad_value_ themselves. chtj5|i|cdddS#1swYyxYwN)torchno_grad)argskwargsfuncs [/root/repos/pocs/video-evals/.venv/lib/python3.12/site-packages/torch/nn/utils/clip_grad.py_no_grad_wrapperz"_no_grad.._no_grad_wrappers. ]]_ )(( ) ) )s(1) functoolsupdate_wrapper)rrs` r_no_gradrs! )-t4  parametersmax_norm norm_typeerror_if_nonfiniteforeachreturnc vt|tjr|g}|Dcgc]}|j|j}}t |}t |}t |dk(rtj dS|dj}t|g}g} |jD]\\} } \\} } | t| | s |r1t| r&| jtj| |O|rtd| jd| j| D cgc]"} tj j#| |$c} tj j#tj$| Dcgc]}|j'|c}|}|rAtj(|j+|j-rtd|d||dzz }tj.|d }|jD]\\} } \\} } | t| | s |r1t| r&tj0| |j'| O|rtd| jd|j'| }| D]} | j3||Scc}wcc} wcc}w) aClip the gradient norm of an iterable of parameters. The norm is computed over the norms of the individual gradients of all parameters, as if the norms of the individual gradients were concatenated into a single vector. Gradients are modified in-place. Args: parameters (Iterable[Tensor] or Tensor): an iterable of Tensors or a single Tensor that will have gradients normalized max_norm (float): max norm of the gradients norm_type (float): type of the used p-norm. Can be ``'inf'`` for infinity norm. error_if_nonfinite (bool): if True, an error is thrown if the total norm of the gradients from :attr:`parameters` is ``nan``, ``inf``, or ``-inf``. Default: False (will switch to True in the future) foreach (bool): use the faster foreach-based implementation. If ``None``, use the foreach implementation for CUDA and CPU native tensors and silently fall back to the slow implementation for other device types. Default: ``None`` Returns: Total norm of the parameter gradients (viewed as a single vector). rg:foreach=True was passed, but can't use the foreach API on  tensorszThe total norm of order z for gradients from `parameters` is non-finite, so it cannot be clipped. To disable this error and scale the gradients by the non-finite norm anyway, set `error_if_nonfinite=False`gư>g?)max) isinstancerr gradfloatlentensordevicer itemsrr extend _foreach_norm RuntimeErrortypelinalg vector_normstackto logical_orisnanisinfclamp _foreach_mul_mul_)r r!r"r#r$pgrads first_device grouped_gradsnormsr/_ device_gradsgnorm total_norm clip_coefclip_coef_clampedclip_coef_clamped_devices rrr$s>*ell+ \ ' >166+=QVV >E >XHi I 5zQ||C  8??L +   E,9,?,?,A Y( (n|a O 4\6 J 3F; LL,,\9E F LV[[MYab  LL,WQ%,,221i@W X Y)) u=tTWW\*=> Je..z/?/?/A:CSCSCUV&yk2- -  J-.I I37,9,?,?,A 1( (n|a O 4\6 J 3F;    .?.B.B6.J K LV[[MYab (9';';F'C $! 1/0 1 1 o ?.X>sJ,J,#'J1 ?J6 z_`torch.nn.utils.clip_grad_norm` is now deprecated in favor of `torch.nn.utils.clip_grad_norm_`.)categoryc t|||||S)zClip the gradient norm of an iterable of parameters. .. warning:: This method is now deprecated in favor of :func:`torch.nn.utils.clip_grad_norm_`. )r)r r!r"r#r$s rrrs$ :x166+=QVV >E >6w?M%2%8%8%: K! \guq O$T$v,%>vN5f=  % %d4<&?* M  % %d4<&? L LV[[MYab  KVT"))zkz)J K K ?s E E )g@FNr)rtypingrrrrrrr typing_extensionsr rr torch.utils._foreach_utilsr r r__all___tensor_or_tensorsrr,boolr FutureWarningrrrrr\suEEE(  D5<<%,,)??@  $" W"WWW W d^ W  \\ W Wt 4 $" Y" Y Y Y Y d^ Y  \\ Y  Y  #(K"(K(Kd^(K (K (Kr