[Question]: when outputting pixel coordinates, does robot always look down

[BaoBao0926]

Question

I see in appendix that, if the model outputs look_down, then the next step is model will output pixel coordinates. However, I want to know, if model output look_down, does it always first look_down, and then output a pixel?

Greate job! Thanks!

[kellyiss]

Because we design the data to be look_down action followed by a pixel goal, it's easy for the model to learn such behavior.