-
Notifications
You must be signed in to change notification settings - Fork 61
Update README.md #242
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sahithyamurthy
wants to merge
1
commit into
AI-Hypercomputer:main
Choose a base branch
from
sahithyamurthy:patch-1
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+64
−10
Open
Update README.md #242
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,21 +1,75 @@ | ||
| # Cloud TPU performance recipes | ||
|
|
||
| This repository provides the necessary instructions to reproduce a | ||
| specific workload on Google Cloud TPUs. The focus is on reliably achieving | ||
| a performance metric (e.g. throughput) that demonstrates the combined hardware | ||
| and software stack on TPUs. | ||
| This repository provides the instructions necessary to reproduce specific | ||
| workload performance measurements, which are part of a confidential | ||
| benchmarking program. This repository focuses on helping you reliably | ||
| achieve performance metrics, for example, throughput that demonstrates | ||
| the combined hardware and software stack on TPUs. | ||
|
|
||
| ## Organization | ||
| **Note:** The content in this repository is not designed as a set of | ||
| general-purpose code samples or tutorials for using Compute Engine-based products. | ||
|
|
||
| - `./training`: instructions to reproduce the training performance of | ||
| ## Intended audience | ||
|
|
||
| This content is for you if you are a customer or partner who needs to: | ||
|
|
||
| - validate hardware performance with your suppliers | ||
| - inform purchasing decisions using the benchmarking data | ||
| - reproduce optimal performance scenarios before you customize workflows | ||
| for your own requirements | ||
|
|
||
| ## How to use these recipes | ||
|
|
||
| To reproduce a benchmark, follow these steps: | ||
|
|
||
| - **Identify your requirements:** determine the model, TPU type, workload, and | ||
| framework (JAX or PyTorch) you are interested in. | ||
| - **Select a recipe:** navigate to the appropriate directory, for example, | ||
| `./training` or `./inference`, to find a recipe that meets your needs. | ||
| - **Follow the procedure:** Each recipe guides you through preparing your | ||
| environment, running the benchmark, and analyzing the results (including detailed logs). | ||
|
|
||
| ## Repository organization | ||
|
|
||
| - `./training`: Use these instructions to reproduce the training performance of | ||
| popular LLMs, diffusion, and other models with PyTorch and JAX. | ||
| - `./inference`: Use these instructions to reproduce inference performance. | ||
| - `./microbenchmarks`: Use these instructions for low-level TPU benchmarks | ||
| such as matrix multiplication performance and memory bandwidth. | ||
| - `./utils`: Find utility scripts here for your cluster and resource | ||
| management, for example, Ironwood for GKE TPU v7. | ||
|
|
||
| ## Repository scope | ||
|
|
||
| This repository provides the steps that you can use to reproduce a specific | ||
| benchmark. The actual performance measurements or the complete, confidential | ||
| benchmark report are not included. | ||
|
|
||
| ## Methodology | ||
|
|
||
| Performance benchmarks measure the performance of various workloads on the | ||
| platform. These benchmarks are primarily used to validate performance with | ||
| hardware suppliers and to provide you with data for purchasing decisions. | ||
|
|
||
| ### Maintenance policy | ||
|
|
||
| Benchmark data is considered a point-in-time measurement and completed | ||
| benchmarks are not repeated. As such, there is no intent to maintain or | ||
| update the reproducibility steps provided in this repository. | ||
|
|
||
| ## Resources | ||
|
|
||
| If you are looking for general guidance on how to get started using | ||
| Compute products, refer to the official documentation and tutorials: | ||
|
|
||
| - `./inference`: instructions to reproduce inference performance. | ||
| - [Official Compute Engine tutorials and samples](https://docs.cloud.google.com/compute/docs/overview) | ||
| - [Cloud TPU documentation](https://docs.cloud.google.com/tpu/docs) | ||
| - [AI Hypercomputer documentation](https://docs.cloud.google.com/ai-hypercomputer/docs) | ||
|
|
||
| - `./microbenchmarks`: instructions for low-level TPU benchmarks such as | ||
| matrix multiplication performance and memory bandwidth. | ||
| ## Getting help | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can also add to reach out to the account team here. |
||
|
|
||
| - `./utils`: utility scripts for cluster and resource management (e.g., [Ironwood](utils/ironwood/README.md) for GKE TPU v7). | ||
| If you have any questions or if you encounter any problems with this repository, | ||
| report them through https://github.com/AI-Hypercomputer/tpu-recipes/issues. | ||
|
|
||
| ## Contributor notes | ||
|
|
||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do periodically update, but it is best effort.