Dont quantize the current token for attention#5715

Merged

facebook-github-bot merged 10 commits into

gh/kimishpatel/114/basefrom

gh/kimishpatel/114/head

Dec 2, 2024

kimishpatel commented Sep 26, 2024 •

edited

Loading

Contributor

Stack from ghstack (oldest at bottom):

Differential Revision: D63497872


          Dont quantize the current token for attention

8b5cd86

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

[ghstack-poisoned]

pytorch-bot Bot commented Sep 26, 2024 •

edited

Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/5715

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit c7c9ffe with merge base c726a9b ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot added the CLA Signed label

kimishpatel mentioned this pull request

[ExecuTorch] Add quantized kv cache to llama #5664

Closed

facebook-github-bot commented Sep 26, 2024

Contributor

This pull request was exported from Phabricator. Differential Revision: D63497872

kimishpatel mentioned this pull request

Refactor custom SDPA op to separate kv cache update from the custom sdpa op #5665

Closed

facebook-github-bot added the fb-exported label

This was referenced Sep 26, 2024

Add update_quantized_cache op #5527

Closed

[Executorch][llama] Update SDPA op to use quantized kv cache #5666

Closed

[Executorch][llama] Refactoring sdpa #5667

Closed

[Executorch] Update EXECUTORCH_LIBRARY macro #5668

Closed

[Executorch][llama] Add custom_sdpa and use that instead of sdpa_with_kv_cache #5669

Closed

[Executorch][quant] Optimize per channel dequantize #5670

Merged

kimishpatel added a commit that referenced this pull request


          Dont quantize the current token for attention

9f43ab7

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

ghstack-source-id: 244978598
Pull Request resolved: #5715


          Update on "Dont quantize the current token for attention"

48e045e

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

[ghstack-poisoned]

facebook-github-bot commented Sep 30, 2024

Contributor

This pull request was exported from Phabricator. Differential Revision: D63497872

kimishpatel added a commit that referenced this pull request


          Dont quantize the current token for attention

4c0ece4

Pull Request resolved: #5715


ghstack-source-id: 245478028
@exported-using-ghexport

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

digantdesai approved these changes

View reviewed changes


          Update on "Dont quantize the current token for attention"

d9986d4

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

[ghstack-poisoned]

facebook-github-bot commented Oct 1, 2024

Contributor

This pull request was exported from Phabricator. Differential Revision: D63497872

kimishpatel added a commit that referenced this pull request


          Dont quantize the current token for attention

a572c04

Pull Request resolved: #5715


ghstack-source-id: 245703478
@exported-using-ghexport

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)


          Update on "Dont quantize the current token for attention"

9ed141f

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

[ghstack-poisoned]

facebook-github-bot commented Oct 1, 2024

Contributor

This pull request was exported from Phabricator. Differential Revision: D63497872

kimishpatel added a commit that referenced this pull request


          Dont quantize the current token for attention

f237471

Pull Request resolved: #5715


ghstack-source-id: 245718144
@exported-using-ghexport

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)


          Update on "Dont quantize the current token for attention"

ae88ec7

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

[ghstack-poisoned]

facebook-github-bot commented Oct 1, 2024

Contributor

This pull request was exported from Phabricator. Differential Revision: D63497872

kimishpatel added a commit that referenced this pull request


          Dont quantize the current token for attention

adfc140

Pull Request resolved: #5715


ghstack-source-id: 245751541
@exported-using-ghexport

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)


          Update on "Dont quantize the current token for attention"

b54362e

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

[ghstack-poisoned]

kimishpatel added a commit that referenced this pull request


          Dont quantize the current token for attention

6f57124

Pull Request resolved: #5715


ghstack-source-id: 246121068
@exported-using-ghexport

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

facebook-github-bot commented Oct 3, 2024

Contributor

This pull request was exported from Phabricator. Differential Revision: D63497872


          Update on "Dont quantize the current token for attention"

a25f5bf

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

[ghstack-poisoned]

kimishpatel mentioned this pull request

[Executorch][llama] Rename update_quantized_cache to update_cache #6914

Merged

facebook-github-bot commented Nov 16, 2024

Contributor

This pull request was exported from Phabricator. Differential Revision: D63497872

This was referenced Nov 20, 2024

[Executorch][BE] Rename sdpa_with_kv_cache.py to custom_ops.py #6996

Merged

[Executorch] Add quantized kv cache to oss ci #6997

Merged


          Update on "Dont quantize the current token for attention"

622e928

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

[ghstack-poisoned]

facebook-github-bot commented Nov 21, 2024

Contributor

This pull request was exported from Phabricator. Differential Revision: D63497872

kimishpatel added the topic: not user facing label


          Update on "Dont quantize the current token for attention"

f187574

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

[ghstack-poisoned]

facebook-github-bot commented Nov 22, 2024

Contributor

This pull request was exported from Phabricator. Differential Revision: D63497872

This was referenced Nov 22, 2024

[Executorch][custom ops] Change lib loading logic to account for package dir #7038

Merged

[Executorch][CI] Fix qnn runner ci job scripts #7049

Closed


          Update on "Dont quantize the current token for attention"

c7c9ffe

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

[ghstack-poisoned]

facebook-github-bot commented Nov 28, 2024

Contributor

This pull request was exported from Phabricator. Differential Revision: D63497872

facebook-github-bot merged commit 3b4a931 into gh/kimishpatel/114/base

facebook-github-bot deleted the gh/kimishpatel/114/head branch

December 2, 2024 16:19

facebook-github-bot temporarily deployed to cherry-pick-bot

December 2, 2024 16:19

— with

GitHub Actions Inactive

pytorchbot mentioned this pull request

Dont quantize the current token for attention #7140

Closed

kirklandsign pushed a commit that referenced this pull request


          Dont quantize the current token for attention

418ede3

Pull Request resolved: #5715


ghstack-source-id: 255730816
@exported-using-ghexport

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

kirklandsign mentioned this pull request

Dont quantize the current token for attention #7164

Merged

kirklandsign added a commit that referenced this pull request


          Dont quantize the current token for attention

73b823b

Pull Request resolved: #5715


ghstack-source-id: 255730816
@exported-using-ghexport

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

Co-authored-by: Kimish Patel <kimishpatel@fb.com>

kedarnath03 pushed a commit to kedarnath03/executorch that referenced this pull request


          Dont quantize the current token for attention

93fc7d5

Pull Request resolved: pytorch/executorch#5715


ghstack-source-id: 254987268
@exported-using-ghexport

Differential Revision: [D63497872](https://our.internmc.facebook.com/intern/diff/D63497872/)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed fb-exported topic: not user facing