Support Stream Generation

[xbyym]

Thank you for your great work!

I have a coding question.
Does the code support stream generation, chunk by chunk? In other words, can it take one chunk (as a window) as input and output the corresponding 3D structure, rather than first processing the whole video chunk and then internally splitting it into multiple windows? I think this would be more helpful for streaming video world models.

[Junyi42]

Hi,

Thanks for the question! Yes, LoGeR is designed for chunk-wise streaming.

You can find the implementation of our chunk-wise streaming processing logic here: loger/models/pi3.py#L667.

The model is causal and allows you to process one chunk at a time as a temporal window. To achieve this in a true streaming setup, you simply need to maintain and pass the memory states (TTT weights and SWA cache) from the previous chunk to the next one.

Thanks