Hello @MathGaron ,
Firstly thanks for sharing your great work. The performance looks very promising. I'm trying to reproduce the result in the paper but got big translation error for the real data.
I trained a model for the clock. The training and validation loss curve seems fine. The final training and validation MSE loss are both around 0.012. Then I adapted the inference script from https://github.com/lvsn/deeptracking and use it to test the images inside directory clock_occlusion_0. Following the evaluation procedure in the paper, I got a big translation error of 23mm. The rotation error seems fine, it's 2.7 degree. I found the big translation error mainly comes from Tz. Then I check the training MSE loss for Tz and found it's 0.044, while training MSE loss for Tx and Ty are just 0.003. I'm wondering if that's the normal case and have a few questions:
- Could you please share what training and validation MSE loss is reasonable? Is 0.012 a good enough value for the training?
- Do you also get much bigger Tz MSE loss than for Tx and Ty?
- The network architecture inside deeptrack_net.py seems different from the one in the paper. filter_size_1 is 64 and not 96. Could that cause the performance gap?
- Do you have any ideas on what are possible reasons for the big Tz error?
Looking forward to your reply and thanks in advance!
Hello @MathGaron ,
Firstly thanks for sharing your great work. The performance looks very promising. I'm trying to reproduce the result in the paper but got big translation error for the real data.
I trained a model for the clock. The training and validation loss curve seems fine. The final training and validation MSE loss are both around 0.012. Then I adapted the inference script from https://github.com/lvsn/deeptracking and use it to test the images inside directory clock_occlusion_0. Following the evaluation procedure in the paper, I got a big translation error of 23mm. The rotation error seems fine, it's 2.7 degree. I found the big translation error mainly comes from Tz. Then I check the training MSE loss for Tz and found it's 0.044, while training MSE loss for Tx and Ty are just 0.003. I'm wondering if that's the normal case and have a few questions:
Looking forward to your reply and thanks in advance!