feat(PGVector): implement validation for embedding dimensions when creating a vector index for ANN search in pgvector

[r3v5]

What does this PR do?

This PR is responsible for implementing a validation for embedding dimensions when creating a pgvector index for Approximate Nearest Neighbor (ANN) search allowing users to successfully create a vector store in pgvector with higher dimensions without receiving 500 Internal Server Error and the runtime error. Though index won't be created due to limitations, users still will be able to use vector store in pgvector via Llama Stack.

Closes #4731

Test Plan

1. Create vector stores with different embedding dimensions:

curl -X GET "http://localhost:8321/v1/vector_stores"

{"object":"list","data":[{"id":"vs_868c6ef3-a713-4b1b-a7ab-f1f267f08f5c","object":"vector_store","created_at":1769435063,"name":"indexed_vector_store","usage_bytes":0,"file_counts":{"completed":0,"cancelled":0,"failed":0,"in_progress":0,"total":0},"status":"completed","expires_after":null,"expires_at":null,"last_active_at":1769435063,"metadata":{"provider_id":"pgvector","provider_vector_store_id":"vs_868c6ef3-a713-4b1b-a7ab-f1f267f08f5c","embedding_model":"sentence-transformers/nomic-ai/nomic-embed-text-v1.5","embedding_dimension":"768"}},{"id":"vs_b9bcd0dc-4a72-4b5f-954e-3f02b1a8569e","object":"vector_store","created_at":1769434929,"name":"indexed_vector_store","usage_bytes":0,"file_counts":{"completed":0,"cancelled":0,"failed":0,"in_progress":0,"total":0},"status":"completed","expires_after":null,"expires_at":null,"last_active_at":1769434929,"metadata":{"provider_id":"pgvector","provider_vector_store_id":"vs_b9bcd0dc-4a72-4b5f-954e-3f02b1a8569e","embedding_model":"openai/text-embedding-3-large","embedding_dimension":"3072"}},{"id":"vs_0babba17-4840-4552-b379-77752fc86149","object":"vector_store","created_at":1769434691,"name":"invoices_store","usage_bytes":0,"file_counts":{"completed":0,"cancelled":0,"failed":0,"in_progress":0,"total":0},"status":"completed","expires_after":null,"expires_at":null,"last_active_at":1769434691,"metadata":{"provider_id":"pgvector","provider_vector_store_id":"vs_0babba17-4840-4552-b379-77752fc86149","embedding_model":"openai/text-embedding-3-large","embedding_dimension":"3072"}}],"first_id":"vs_868c6ef3-a713-4b1b-a7ab-f1f267f08f5c","last_id":"vs_0babba17-4840-4552-b379-77752fc86149","has_more":false}%

2. Check that HNSW index was created only on vector store with embedding_dimension = 768:

psql (18.1 (Debian 18.1-1.pgdg13+2))
Type "help" for help.


testvectordb=# \z
                                               Access privileges
 Schema |                    Name                    | Type  | Access privileges | Column privileges | Policies
--------+--------------------------------------------+-------+-------------------+-------------------+----------
 public | metadata_store                             | table |                   |                   |
 public | vs_vs_0babba17_4840_4552_b379_77752fc86149 | table |                   |                   |
 public | vs_vs_868c6ef3_a713_4b1b_a7ab_f1f267f08f5c | table |                   |                   |
 public | vs_vs_b9bcd0dc_4a72_4b5f_954e_3f02b1a8569e | table |                   |                   |
(4 rows)


testvectordb=# SELECT indexname, indexdef
FROM pg_indexes
WHERE indexdef LIKE '%hnsw%';
                      indexname                      |                                                                                              indexdef
-----------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 vs_vs_868c6ef3_a713_4b1b_a7ab_f1f267f08f5c_hnsw_idx | CREATE INDEX vs_vs_868c6ef3_a713_4b1b_a7ab_f1f267f08f5c_hnsw_idx ON public.vs_vs_868c6ef3_a713_4b1b_a7ab_f1f267f08f5c USING hnsw (embedding vector_cosine_ops) WITH (m='16', ef_construction='64')
(1 row)


testvectordb=#

3. No runtime error and 500 Internal Server Error:

INFO     2026-01-26 13:37:04,011 uvicorn.error:216 uncategorized: Uvicorn running on http://['::', '0.0.0.0']:8321
         (Press CTRL+C to quit)
INFO     2026-01-26 13:37:05,543 llama_stack.providers.utils.inference.openai_mixin:475 providers::utils:
         OpenAIInferenceAdapter.list_provider_model_ids() returned 121 models
INFO     2026-01-26 13:38:11,088 uvicorn.access:476 uncategorized: ::1:55130 - "POST /v1/vector_stores HTTP/1.1" 200
INFO     2026-01-26 13:42:09,411 uvicorn.access:476 uncategorized: ::1:55275 - "POST /v1/vector_stores HTTP/1.1" 200
INFO     2026-01-26 13:42:22,442 uvicorn.access:476 uncategorized: ::1:55281 - "GET /v1/vector_stores HTTP/1.1" 200
INFO     2026-01-26 13:44:23,801 uvicorn.access:476 uncategorized: ::1:55330 - "POST /v1/vector_stores HTTP/1.1" 200
INFO     2026-01-26 13:44:33,288 uvicorn.access:476 uncategorized: ::1:55336 - "GET /v1/vector_stores HTTP/1.1" 200

[franciscojavierarceo]

probably we should log something for the else condition

[r3v5]

Yeah, I am planning to do it along with adding support for IVFFlat after #4714 lands.

[r3v5]

We can log specific information regarding what index user created/attempted to create i.e. HNSW or IVFFlat

[r3v5]

Logging for indexes will be implemented in PR for this issue #4745

[mergify]

This pull request has merge conflicts that must be resolved before it can be merged. @r3v5 please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[r3v5]

I am solving merge conflict. Almost there :)

[r3v5]

It would be great to land current PR as I need it for #4745.

[r3v5]

I will rebase PR tomorrow and solve merge conflict

[mergify]

This pull request has merge conflicts that must be resolved before it can be merged. @r3v5 please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork