out of memory issue

[skunkwerk]

Hi,
thanks for this great repo. I was trying to run with a small video (attached) on an nVidia GPU with 24 GB VRAM but ran into CUDA memory issues:

Running configs/prompts/SampleConfig.yaml ...
/home/me/environments/ca/lib/python3.10/site-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
  warnings.warn(
20240109_022113_013548
Frames to be processed: 15.0
Option to (record or transcode stop time) cannot be applied to input url /home/me/bean.mp4 -- you are trying to apply an input option to an output file or vice versa. Move this option before the file it belongs to.
Error parsing options for input file /home/me/bean.mp4.
Error opening input files: Invalid argument
loaded temporal unet's pretrained weights from models/StableDiffusion/stable-diffusion-v1-5/unet ...
### missing keys: 668; 
### unexpected keys: 0;
### Temporal Module Parameters: 1312 -> 453.20928 M
Traceback (most recent call last):
...
  File "/home/me/environments/ca/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1143, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: CUDA error: out of memory

I have attached my YAML file (as a .txt as Github doesn't support YAML uploads)

Any ideas what could be causing this?

bean.mp4

SampleConfig.txt

[intellerce]

Hello,

You shouldn't have any issues with these settings especially with 24GB of VRAM - if anything your resolution is too low!

Have you tracked your VRAM usage from the beginning using nvidia-smi (e.g., watch nvidia-smi)?

Looking more closely it seems that the issue might be possibly arising from your FFMPEG not being able to read your input file correctly and throwing an error. You might want to reinstall FFMPEG -- I should add that sometimes it can be a bit difficult to get it right -- I have included a couple of guides in the README. Btw, you can set your FFMPEG path at the end of the config file under 'ffmpeg_path' in case you are installing it inside your conda/python environment.

If the issue persists after fixing the FFMPEG error, please let me know.

Thanks