[python] add typeddict models-mode for Python HTTP client emitter#10439
Open
iscai-msft wants to merge 48 commits into
Open
[python] add typeddict models-mode for Python HTTP client emitter#10439iscai-msft wants to merge 48 commits into
models-mode for Python HTTP client emitter#10439iscai-msft wants to merge 48 commits into
Conversation
Add a new 'typeddict' value for the models-mode option that generates Python TypedDict classes instead of DPG model classes. Key features: - TypedDict classes with Required[T]/NotRequired[T] annotations - TypedDict inheritance for non-discriminated models - Discriminated models: Union of leaf TypedDicts, no abstract base class - Input-only: operations accept TypedDict input, return dict output - Wire names used as TypedDict keys - _model_base.py still generated for serialization utilities Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
iscai-msft
requested review from
ChenxiJiang333,
catalinaperalta,
laiapat,
lmazuel,
msyyc,
swathipil,
tadelesh and
timotheeguerin
as code owners
microsoft-github-policy-service
Bot
added
the
emitter:client:python
commit: |
Contributor
|
All changed packages have been documented.
Show changes
|
Collaborator
|
You can try these changes here
|
…ypespec into python/addTypedDict
…hon/addTypedDict
- TypedDictModelType returns 'JSON' for response type annotations - Response.type_annotation/docstring passes is_response=True - Typeddict deserialization uses response.json() directly - Removed NotRequired from TypedDictModelSerializer (total=False handles it) - Updated mock API tests to verify JSON dict responses Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add client/naming typeddict variant to regenerate-common.ts - Create test_client_naming_typeddict.py with 11 tests verifying TypedDict uses wire names (defaultName, wireName) not client names - Tests cover: ClientNameModel, LanguageClientNameModel, ClientNameAndJsonEncodedNameModel, ClientModel, PythonModel Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TypedDict is already JSON, so the MutableMapping[str, Any] overload is unnecessary. Only keep TypedDict model + IO[bytes] overloads. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Typeddict mode uses response.json() directly, so _deserialize is never called. Skip importing it to avoid W0611 unused-import lint warning. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove unused MutableMapping/Any imports from TypedDictModelType.imports() - Skip _deserialize import in paging_operation.py for typeddict mode Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ypespec into python/addTypedDict
TypedDictModelType returns 'JSON' for response type annotations but never defined the JSON = MutableMapping[str, Any] type alias, causing NameError at runtime. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
iscai-msft
force-pushed
the
python/addTypedDict
branch
from
34e0cda to
95db199
Compare
- Skip rest_field/rest_discriminator imports for TYPES_FILE context - Use bare model/enum names in types.py instead of _models.Name prefix to avoid pyright reportInvalidTypeForm with dotted forward refs - Import model/enum names directly from .models under TYPE_CHECKING - Fix unions file to properly import and reference discriminated subtypes - Fix typed-dict-only import style to use 'from .. import types' module Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…hon/addTypedDict
…hon/addTypedDict # Conflicts: # packages/http-client-python/eng/scripts/ci/regenerate.ts
Discriminated model unions now reference TypedDict subtypes instead of _Model classes. Union aliases are generated in types.py alongside the TypedDict definitions, and _unions.py is only generated when named unions exist. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ensures nested discriminated bases (e.g. Shark) are defined before their parents (e.g. Fish = Union[Salmon, Shark]). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
All TypedDict models are defined in the same types.py file, so cross- namespace model imports also cause redefinition errors. Remove the same_namespace check so any model that will be a TypedDict is never imported from its models module. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Discriminated bases are now Union aliases in types.py, so they must also be excluded from TYPE_CHECKING imports to avoid shadowing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When a TypedDict field wire_name matches a Python builtin type name (e.g. 'int'), it shadows that builtin in subsequent annotations. Move such properties to the end of the class to prevent this. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Use builtins.X qualification when a TypedDict field wire_name shadows a Python builtin type name (e.g. int, str, list) that appears in type annotations within the same class - Add functional TypedDict form for models with Python keyword wire_names (e.g. and, class, for) since keywords can't be identifiers in class bodies - Only import builtins when actually needed (when shadowed builtins appear in annotations) - Remove the previous sorting workaround Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Previously types.py was only generated once at the root namespace with all models. Now it's generated per namespace alongside the models/ folder, using only that namespace's models. This ensures sub-namespaces like modelproperties/ get their own types.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Pass client_namespace to BaseSerializer in TypesSerializer so serialize_namespace resolves relative to the correct namespace - Distinguish same-namespace vs cross-namespace models in TYPES_FILE import logic: same-namespace non-json models are skipped (defined locally), cross-namespace models import from sibling types module, same-namespace json models import from .models Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Merge regenerate-common.ts into regenerate.ts (single file) - Add typeddict variant packages for not-discriminated and single-discriminator - Add typeddictonly variant package for usage (typed-dict-only-models: all) - Support 'all' value for typed-dict-only-models option in model_type.py Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TypedDicts don't differentiate input/output. The parent classes don't use is_response either, so the pop overrides were just noise. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The typeddict test should pass plain dicts with wire names, not model class constructors which expect client names. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ict-only models - Skip importing types module from itself when model is in same namespace - Skip _deserialize import when response type is typed-dict-only (uses response.json() instead) - Fix paging operation _deserialize import for typed-dict-only item types Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Internal models in types.py should use bare names like all other models, since they're imported by name under TYPE_CHECKING. Also fix import path for internal models to use .models._models instead of .models. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…anges into regenerate.ts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds alpha support for generating/using Python TypedDict-style model shapes in the HTTP client Python emitter, alongside restructuring named-union output into a dedicated _unions.py module and updating tests/regeneration config to cover the new behaviors.
Changes:
- Introduces
models-mode=typeddict,types.pyTypedDict generation, andtyped-dict-only-modelsbehavior hooks in codegen. - Moves named union aliases out of the old types file flow into a new
_unions.pyfile with a dedicated serializer/template. - Adds unit + mock API tests and updates regeneration script/config to produce typeddict variants.
Reviewed changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/http-client-python/tests/unit/test_typeddict.py | Adds unit coverage for types.py TypedDict output and typed-dict-only behaviors. |
| packages/http-client-python/tests/mock_api/shared/test_typetest_model_usage_typeddictonly.py | Adds mock API validation for typed-dict-only input/output behavior. |
| packages/http-client-python/tests/mock_api/shared/test_typetest_model_inheritance_single_discriminator_typeddict.py | Adds mock API coverage for discriminated models in typeddict scenario. |
| packages/http-client-python/tests/mock_api/shared/test_typetest_model_inheritance_not_discriminated_typeddict.py | Adds mock API coverage for non-discriminated model inheritance in typeddict scenario. |
| packages/http-client-python/tests/mock_api/azure/test_client_naming_typeddict.py | Adds mock API coverage to ensure wire-name keys are used for TypedDict interactions. |
| packages/http-client-python/generator/pygen/codegen/templates/unions.py.jinja2 | New template for _unions.py named union aliases. |
| packages/http-client-python/generator/pygen/codegen/templates/types.py.jinja2 | Reworked types.py template to emit TypedDicts and discriminated unions. |
| packages/http-client-python/generator/pygen/codegen/templates/model_typeddict.py.jinja2 | Adds a (currently unused) TypedDict model template. |
| packages/http-client-python/generator/pygen/codegen/templates/model_container.py.jinja2 | Routes typeddict base models through the shared model template include. |
| packages/http-client-python/generator/pygen/codegen/serializers/unions_serializer.py | New serializer to render _unions.py with correct imports. |
| packages/http-client-python/generator/pygen/codegen/serializers/types_serializer.py | Implements TypedDict rendering/import logic for types.py including discriminated-base unions. |
| packages/http-client-python/generator/pygen/codegen/serializers/model_serializer.py | Adds TypedDictModelSerializer for generating TypedDict-shaped “models” output. |
| packages/http-client-python/generator/pygen/codegen/serializers/builder_serializer.py | Skips deserialization and returns raw JSON for typed-dict-only responses/items. |
| packages/http-client-python/generator/pygen/codegen/serializers/init.py | Writes types.py per namespace; writes _unions.py at top-level; filters typed-dict-only models out of model classes. |
| packages/http-client-python/generator/pygen/codegen/models/utils.py | Adds NamespaceType.UNIONS_FILE to support _unions.py import behavior. |
| packages/http-client-python/generator/pygen/codegen/models/response.py | Switches named-union imports to _unions and threads is_response flag through annotation/docstring paths. |
| packages/http-client-python/generator/pygen/codegen/models/property.py | Avoids importing rest_field/rest_discriminator when rendering properties for types.py. |
| packages/http-client-python/generator/pygen/codegen/models/parameter.py | Switches named-union imports to _unions. |
| packages/http-client-python/generator/pygen/codegen/models/paging_operation.py | Avoids importing _deserialize when paging item type is typed-dict-only. |
| packages/http-client-python/generator/pygen/codegen/models/operation.py | Avoids importing _deserialize when a response type is typed-dict-only. |
| packages/http-client-python/generator/pygen/codegen/models/model_type.py | Adds is_typed_dict_only and adjusts typing/import logic for types.py and typed-dict-only references. |
| packages/http-client-python/generator/pygen/codegen/models/enum_type.py | Adjusts enum type annotations/imports to behave better within types.py. |
| packages/http-client-python/generator/pygen/codegen/models/combined_type.py | Redirects named-union references/imports to _unions. |
| packages/http-client-python/generator/pygen/codegen/models/code_model.py | Excludes typed-dict-only models from public_model_types. |
| packages/http-client-python/generator/pygen/init.py | Extends option validation to allow models-mode=typeddict and normalizes typed-dict-only-models. |
| packages/http-client-python/eng/scripts/ci/regenerate.ts | Inlines regeneration-common logic and adds/updates emitter options for typeddict-related generated packages. |
| packages/http-client-python/eng/scripts/ci/regenerate-common.ts | Removes shared regeneration helper module (logic inlined into regenerate.ts). |
| .chronus/changes/python-addTypedDict-2026-3-21-17-47-3.md | Adds Chronus entry describing the feature. |
Comment on lines
+152
to
+161
| for parent in model.parents: | ||
| if parent.client_namespace != model.client_namespace and not parent.discriminated_subtypes: | ||
| file_import.add_submodule_import( | ||
| self.code_model.get_relative_import_path( | ||
| self.serialize_namespace, | ||
| self.code_model.get_imported_namespace_for_model(parent.client_namespace), | ||
| ), | ||
| parent.name, | ||
| ImportType.LOCAL, | ||
| ) |
Comment on lines
+224
to
+233
| def declare_property(self, prop: Property, shadowed_builtins: frozenset[str]) -> str: | ||
| type_annotation = prop.type_annotation( | ||
| serialize_namespace=self.serialize_namespace, | ||
| serialize_namespace_type=NamespaceType.TYPES_FILE, | ||
| ) | ||
| type_annotation = _qualify_shadowed_builtins(type_annotation, shadowed_builtins) | ||
| is_optional = prop.optional or prop.client_default_value is not None | ||
| if is_optional: | ||
| return f"{prop.wire_name}: {type_annotation}" | ||
| return f"{prop.wire_name}: Required[{type_annotation}]" |
Comment on lines
+315
to
+319
| models_mode = self.code_model.options["models-mode"] | ||
| if models_mode in ("dpg", "typeddict"): | ||
| serializer = DpgModelSerializer | ||
| else: | ||
| serializer = MsrestModelSerializer |
| - "@typespec/http-client-python" | ||
| --- | ||
|
|
||
| [python] Always generate `TypedDict` typing hints for input models in the `_types.py` file |
…hon/addTypedDict
…of literals instead of enums when td only
- Fix cross-namespace parent imports in types.py to import from types module - Add isidentifier() check for wire names (not just iskeyword) for functional TypedDict form - Use __doc__ assignment for functional-form TypedDict docstrings - Add missing op_tools import in model_typeddict.py.jinja2 - Fix test docstring to match actual behavior - Update changelog entry with correct file names (types.py, _unions.py) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #8800
Add a new 'typeddict' value for the
models-modeoption that generates PythonTypedDictclasses instead of DPG model classes. Key features:_model_base.pystill generated for serialization utilities