Reproduction results on MVTec LOCO differ from reported paper results

[zhiqing0205]

Hello,

First of all, thank you very much for sharing this excellent work and the codebase. I carefully followed the instructions in the README to reproduce the experiments, using the compressed dataset provided in the link you shared.

However, I noticed that when evaluating on the MVTec LOCO dataset, my reproduced results are consistently lower than those reported in the paper. Below are the comparisons:

Reproduced results:

class	logical	struct	mean
breakfast_box	90.79	78.69	84.74
juice_bottle	99.02	97.91	98.46
pushpins	99.91	96.41	98.16
screw_bag	90.65	95.60	93.13
splicing_connectors	95.14	97.89	96.52
avg	95.10	93.30	94.20

Reported in the paper:

class	logical	struct	mean
breakfast_box	92.9	85.7	89.3
juice_bottle	99.7	99.6	99.7
pushpins	100.0	98.8	99.4
screw_bag	93.9	96.0	95.0
splicing_connectors	96.0	98.6	97.3
avg	96.5	95.7	96.1

I understand that some variation is expected, but the gap in certain categories (e.g., breakfast_box) seems larger than I anticipated.

Would you kindly let me know if there are any additional steps, preprocessing details, or settings I might have overlooked? I would really appreciate your guidance whenever you have time.

Thank you again for your work and for making the resources available.

Best regards,
zhiqing

[MaticFuc]

I will look into it when I have time. I hope this will be next week. Based on other issues, it seems that I've accidentally introduced some bugs in the release version. I expect this to bring a larger variation.

I will provide more details when I look into it.