ReLU before MaxPool

[mhmd97z]

Hi there!

I could not quite understand why reluifying a maxpool operation can happen only when it comes after relu.
I'm referring to this line of code.

[dlshriver]

Great question. Honestly, I don't remember. It looks like that simplification is applied to MaxPool operations that occur either before OR after ReLU operations. I assume I made some assumption about the outputs being non-negative, but I don't remember why that was necessary or whether it actually was necessary. It seems pretty common to have both of those operations next to each other. Do you have a network with a MaxPool operation that isn't preceded or succeeded by a ReLU? If we can prove that the ReLU is not needed, we can certainly remove that check.