2025-09-22 LMCache Dynamo example needs cleanup

[ACodingfreak]

Hi,

Thanks for sharing the detailed instructions for 2025-09-22 LMCache Dynamo example

https://github.com/Quick-AI-tutorials/AI-Infra/tree/b30ff196dbed6dd3a27e54bb0220ab7ac59de36e/2025-09-22%20LMCache%20Dynamo

With the latest version of Dynamo, build.sh is no more available in respective directory

# Build the container
./container/build.sh --framework vllm

[ACodingfreak]

Well looks like below commands are the right order with the latest dynamo

# Build runtime image
container/render.py --framework vllm --target runtime --output-short-filename
sudo docker build -t dynamo:latest-vllm-runtime -f container/rendered.Dockerfile .

[ACodingfreak]

Now I am ending up with new issue.

I am having 24g vram on my GPU and eventhough I have 2/3rd vram available on running offloading script inside vllm-runtime container I am seeing it is out of memory

dynamo@mltr01:/workspace/AI-Infra/2025-09-22 LMCache Dynamo$ python cpu-offloading.py
Traceback (most recent call last):
  File "/workspace/AI-Infra/2025-09-22 LMCache Dynamo/cpu-offloading.py", line 161, in <module>
    main()
  File "/workspace/AI-Infra/2025-09-22 LMCache Dynamo/cpu-offloading.py", line 139, in main
    llm = initialize_llm(enable_lmcache=args.enable_lmcache)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/AI-Infra/2025-09-22 LMCache Dynamo/cpu-offloading.py", line 102, in initialize_llm
    gpu_memory_utilization=calculate_gpu_utilization()
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/AI-Infra/2025-09-22 LMCache Dynamo/cpu-offloading.py", line 61, in calculate_gpu_utilization
    raise RuntimeError(f"GPU memory ({total_memory:.1f}GB) is less than required memory ({target_memory_gb}GB)")
RuntimeError: GPU memory (23.5GB) is less than required memory (24GB)
dynamo@mltr01:/workspace/AI-Infra/2025-09-22 LMCache Dynamo$
dynamo@mltr01:/workspace/AI-Infra/2025-09-22 LMCache Dynamo$
dynamo@mltr01:/workspace/AI-Infra/2025-09-22 LMCache Dynamo$ nvidia-smi
Sun Mar 22 19:21:58 2026
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.288.01             Driver Version: 535.288.01   CUDA Version: 12.9     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA A30                     Off | 00000000:9B:00.0 Off |                    0 |
| N/A   34C    P0              35W / 165W |   8102MiB / 24576MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
+---------------------------------------------------------------------------------------+