Include documented TPU recipes#477
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds TPU support documentation and verification status for the Qwen3-Coder-480B-A35B-Instruct and Llama-3.3-70B-Instruct models on Ironwood and Trillium hardware. A review comment suggests replacing the ':latest' tag in the Docker example with a placeholder to ensure consistency with the instructions and improve reproducibility.
| docker run -itd --name llama33-tpu \ | ||
| --privileged --network host --shm-size 16G \ | ||
| -v /dev/shm:/dev/shm -e HF_TOKEN=$HF_TOKEN \ | ||
| vllm/vllm-tpu:latest \ |
There was a problem hiding this comment.
The documentation on line 106 correctly advises to 'Pull the tag specified by the upstream Trillium recipe'. However, the example docker run command on line 111 uses the vllm/vllm-tpu:latest tag. Using :latest can lead to non-reproducible behavior and might break if there are changes in the upstream image. To align with the documentation's recommendation and avoid potential issues for users, please consider using a placeholder to indicate that a specific tag from the recipe should be used.
vllm/vllm-tpu:<tag-from-recipe> \
Signed-off-by: Mateusz Sokół <mat646@gmail.com>
1 participant
Hi Team,
This PR adds two more TPU configurations for models documented in https://github.com/AI-Hypercomputer/tpu-recipes repository.