How to get the index of the pruned channel(s) when using High-level Pruners?

[YuxuanDuan]

I want to know which channels are pruned during the process of L2-norm pruning , but i don't know how

[VainF]

Hello @YuxuanDuan, thank you for bringing up this discussion. I was wondering if you could share your thoughts on a possible solution. Specifically, I was thinking about allowing the pruner.step() to return a dictionary that contains both the pruned nn.Module and their corresponding indices. What do you think about this approach? Just like the following:

{
    nn.Conv2d(....): [0,2,6],
    nn.BatchNorm(...): [0,2,6],
}

Or, what do you think about a more complicated output with group structures:

[
    {  # group 0
            nn.Conv2d(....): [0,2,6],
            nn.BatchNorm(...): [0,2,6],
            #...
     },

    { # group 1
     # ...
    }
]

[YuxuanDuan]

I am not quite sure what the group 0 / group 1 means, but approach 1 is an excellent implemetation!

[VainF]

Hi @YuxuanDuan, all pruners now support interactive pruning. You can use pruner.step(interactive=True) to get all groups as follows:

for i in range(iterative_steps):
    for group in pruner.step(interactive=True):
        print(group) 
        # do whatever you like with the group 
        # ...
        group.prune() # you should manually call the group.prune()
    macs, nparams = tp.utils.count_ops_and_params(model, example_inputs)
    # finetune your model here
    # finetune(model)
    # ...

[YuxuanDuan]

Yes,I will try it later! If there’s any problem, I will feedback to you. Many thanks!