Hi @rogertrullo,
Thanks for sharing this. While computing the receptive field of a network with a sigmoid in the forward pass, the gradient does not flow as expected, probably because the gradient vanishes. I think it may then be useful to add something like "Turn off / Remove sigmoid activations" as a requirement. Or did I interpret it wrongly? What do you think?
Hi @rogertrullo,
Thanks for sharing this. While computing the receptive field of a network with a sigmoid in the forward pass, the gradient does not flow as expected, probably because the gradient vanishes. I think it may then be useful to add something like "Turn off / Remove sigmoid activations" as a requirement. Or did I interpret it wrongly? What do you think?