Skip to content

gradient-based saliency maps to support different activations #6057

Description

@wyli
class GuidedBackpropGrad(VanillaGrad):
   

    def __call__(self, x: torch.Tensor, index: torch.Tensor | int | None = None, **kwargs) -> torch.Tensor:
        with replace_modules_temp(self.model, "relu", _GradReLU(), strict_match=False):
            return super().__call__(x, index, **kwargs)

I think the method tries to look for "relu" layers and replace it with a grad-hooked customised one. This assumingly might not work with other activation functions. this is a feature request to support different activation functions, SiLU for instance.

Originally posted by @trinhdhk in #6012 (reply in thread)

Activity

  1. AnaghaV17 commented on Nov 15, 2026

    @AnaghaV17

    @wyli @KumoLiu @Nic-Ma @ericspod,

    Request you to kindly review this PR: #9283 intended to resolve this issue.

    It generalizes the guided-backpropagation:

    • A single _AutoGradActivation autograd Function that works for any activation module (ReLU, LeakyReLU, SiLU, GELU) by delegating forward to the module and computing the local derivative via autograd — no per-activation hardcoded math.

    • Wires guided backprop into both GuidedBackpropGrad and GuidedBackpropSmoothGrad via a reusable _GuidedActivationsTemp context manager.

    • Fixes two real bugs PR 6057guided backprop silu #7070 didn't address: in-place activation aliasing (calling module(x) directly while saving a pre-activation clone) and shared module instances (named_modules(remove_duplicate=False) so each binding is wrapped/restored).

    Please review and let me know your thoughts. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions