fix(server): honor prompt cache byte limit#1392

agisilaos wants to merge 1 commit into

ml-explore:mainml-explore/mlx-lm:mainfrom

agisilaos:fix/issue-aborts-metal-oom-after-prompt-cache-growsagisilaos/mlx-lm:fix/issue-aborts-metal-oom-after-prompt-cache-growsCopy head branch name to clipboard