update changes

Author: iam404

CHANGELOG.md

@@ -5,14 +5,24 @@
 ### Relationships
 * Added a lossless JSON:API escape hatch. **Every resource model** now derives from the new `pytfe.models.TFEModel` base and exposes `model.relationships`, `model.included`, `model.included_by(type, id)`, `model.related(name)`, and the `model.has_relationships` / `model.has_included` presence flags (distinguishing "absent on the wire" from "present but empty"). The raw blocks are private attributes — excluded from `model_dump()` **and from equality** — so this is additive and non-breaking; they complement `extra="allow"`, which only retains unknown *attributes*.
   * **`relationships` capture** is wired broadly across the resources whose models are built through a dedicated parser (workspaces, runs, projects, teams, policies, policy sets, stacks, registry, no-code modules, comments, state versions, variable sets, oauth clients, notification configs, org memberships, query runs, admin orgs/runs/users/workspaces, and more), so the raw relationship references are always reachable.
-  * **`included` hydration** (typed relations filled from the document's top-level `included`, and a populated `model.included`) currently applies to the single-resource reads that thread it — `workspaces.read*`, `runs.read*`, `no_code_modules.read_variables`. Other single reads and **all list endpoints** capture `relationships` but not yet `included`; threading `included` through the remaining reads and list pagination is an in-progress follow-up.
+  * **`included` hydration** now comes in two forms, both purely additive:
+    * **Typed hydration** — declared relationship fields are filled from the document's top-level `included` array (and `model.included` is populated). Applies to the single-resource reads of `workspaces`, `runs`, `agent_pools`, `stack_configuration`, `teams`, `task_stages`, `policy_set`, `organization_membership`, `variable_set`, `run_event`, and `no_code_modules.read_variables`. The rule is uniform: **wherever a resource models a relation as a typed field, `?include=<relation>` fills that field** (e.g. `policy_set.current_version`, `organization_membership.user`, `run_event.actor`).
+    * **Raw capture** — `model.included` is populated and `model.related(name)` / `model.included_by(type, id)` resolve to the full related bodies. Applies to the single-resource reads whose includable relations are **not** modelled as typed fields, so there is no typed field to fill: `state_versions`, `agents`, `configuration_version`, `oauth_client`, `organizations`, `projects`, `query_run`, `registry_provider`, and `run_task` reads. (Capturing the raw blocks only populates the private escape hatch — no typed field changes, so this is non-breaking.)
+    * **Not yet wired** — `registry_module`, `run_trigger`, and `policy_check` accept `?include=` only on their *list* endpoints, and **all list endpoints** across the SDK still capture `relationships` but not `included` (the shared top-level `included` array is not yet threaded through list pagination — an in-progress follow-up).
 
-  See [docs/related-resources.md](docs/related-resources.md).
+  See [docs/related-resources.md](docs/related-resources.md) for the per-resource coverage table and a "typed field vs raw accessor" guide.
+
+* Added `?include=` support to three single-resource reads that previously exposed no include option, matching the HCP Terraform API (verified against go-tfe's OpenAPI spec and the live API):
+  * `teams.read(team_id, TeamReadOptions(include=[...]))` — `users`, `organization-memberships` (typed hydration).
+  * `task_stages.read(task_stage_id, TaskStageReadOptions(include=[...]))` — `run`, `run.workspace`, `task-results`, `policy-evaluations` (typed hydration).
+  * `organizations.read(name, OrganizationReadOptions(include=[...]))` — `subscription` (raw capture). The new `options` argument is optional, so existing positional calls are unchanged.
 
 ## Bug Fixes
 
 ### Relationships
 * Fixed `workspaces.read*(..., include=[WorkspaceIncludeOpt.OUTPUTS])` returning outputs with `None` name/value/type. Workspace `outputs` is now hydrated from the JSON:API `included` array through the shared relationship parser (matching go-tfe's `relation,outputs`), instead of a broken special case that read attributes off the id-only relationship references. [#134](https://github.com/hashicorp/python-tfe/issues/134) (the related project-include case, [#74](https://github.com/hashicorp/python-tfe/issues/74), was already resolved by the relationship refactor and is verified covered.)
+* `PolicySetVersion` is now exported from `pytfe.models` and its forward reference to `PolicySet` is resolved via `model_rebuild()`. Previously it was never fully defined, so `policy_set.read*(include=[current_version|newest_version])` silently fell back to an id-only stub instead of hydrating the version's `source`/`created_at`/`status`.
+* `variable_set.read` no longer fabricates placeholder relation values (e.g. `name="workspace-<id>"`, `key="var-<id>"`, `category="terraform"`) for `workspaces`/`projects`/`vars`. Those relations are now id-only stubs by default and hydrate from `included` when requested via `?include=`, like every other typed relation.
 
 # Released
 # v1.1.0

docs/related-resources.md

@@ -60,6 +60,53 @@ pyTFE handles this on two levels:
    print(list(ws.relationships))   # e.g. ['organization', 'project', 'outputs', ...]
    ```
 
+## Which should I use — the typed field or the raw accessor?
+
+**The one rule:** a typed relationship field always carries **at least the `id`**.
+Pass `?include=<relation>` to fill in the rest.
+
+```python
+from pytfe.models.policy_set import PolicySetReadOptions, PolicySetIncludeOpt
+
+ps = client.policy_sets.read("polset-abc")
+ps.current_version.id        # always present (id-only stub)
+ps.current_version.source    # None — you didn't ask for it
+
+ps = client.policy_sets.read_with_options(
+    "polset-abc",
+    PolicySetReadOptions(include=[PolicySetIncludeOpt.POLICY_SET_CURRENT_VERSION]),
+)
+ps.current_version.source    # now hydrated from `included`
+```
+
+* **Prefer the typed field** (`ps.current_version`, `ws.outputs`, `team.users`,
+  `org_membership.user`, `run_event.actor`) whenever the relation is modelled — it's
+  type-checked and stable, and `?include=<relation>` fills it. This works the *same
+  way for every resource that models the relation*: there are no resources where a
+  typed field silently stays a stub after you `?include=` it.
+* **Use the raw accessors** (`model.related(name)`, `model.included_by(type, id)`)
+  only for relations the SDK does **not** model as a typed field — e.g. an
+  organization's `subscription`, or a workspace `readme`. The data is still returned
+  by `?include=`, just untyped.
+
+You never need both for the same relation: if a typed field exists, `?include=` fills
+it; if it doesn't, the raw accessors are the way in.
+
+## Per-resource coverage
+
+`?include=` support by single-resource `read*` (see each resource's `*IncludeOpt`):
+
+| Behaviour | Resources |
+|---|---|
+| **Typed hydration** — `include` fills the typed field | `workspaces`, `runs`, `agent_pools`, `stack_configuration`, `teams`, `task_stages`, `policy_set`, `organization_membership`, `variable_set`, `run_event`, `no_code_modules.read_variables` |
+| **Raw capture** — relation not modelled as a typed field; reach it via `related()` / `included_by()` | `organizations` (`subscription`), `state_versions`, `agents`, `configuration_version`, `oauth_client`, `projects`, `query_run`, `registry_provider`, `run_task` |
+| **List-only** — `?include=` exists only on the `list` endpoint | `registry_module`, `run_trigger`, `policy_check` |
+
+In every case the **`relationships`** block and the four raw accessors are populated,
+so unmodelled relations are never lost. **List endpoints** currently capture
+`relationships` but not `included` (the page-level `included` array is not yet threaded
+through pagination — in progress).
+
 ## Notes
 
 - The raw blocks are **private attributes**, so they never appear in
@@ -72,6 +119,8 @@ pyTFE handles this on two levels:
 - Accessors are provided by `pytfe.models.TFEModel`, which **every
   resource model** now derives from — so `.relationships` / `.included` /
   `.included_by` / `.related` are available everywhere. They're *populated* on
-  resources parsed through a dedicated parser; other resources expose the
-  accessors but return them empty until their parser is wired to capture the
-  raw blocks.
+  single-resource `read*` calls: the `relationships` block on reads that go
+  through a relationship-capturing parser, and the `included` array whenever you
+  pass `?include=`. **List endpoints** currently populate `relationships` but not
+  `included` — the shared top-level `included` array is not yet threaded through
+  pagination (in progress).

examples/related_resources.py

@@ -9,24 +9,27 @@
   * `included`      — full bodies of relations you ask for with ?include=
 
 pyTFE hydrates the relations it models into typed fields, AND keeps both raw
-blocks so nothing is ever lost. This example shows both.
+blocks so nothing is ever lost. The one rule: a typed relationship field always
+carries at least the `id`; pass ?include= to fill in the rest. This example shows
+both a workspace and a team.
 
 Prerequisites:
     export TFE_TOKEN=...          # your API token
+    export TFE_ORG=...            # org to look up a team in (optional)
     python examples/related_resources.py ws-abc123     # a workspace id
 """
 
 from __future__ import annotations
 
+import os
 import sys
 
 from pytfe import TFEClient
+from pytfe.models.team import TeamIncludeOpt, TeamReadOptions
 from pytfe.models.workspace import WorkspaceIncludeOpt, WorkspaceReadOptions
 
 
-def main(workspace_id: str) -> None:
-    client = TFEClient()
-
+def workspace_demo(client: TFEClient, workspace_id: str) -> None:
     # Ask the API to include the workspace's outputs and project.
     ws = client.workspaces.read_by_id_with_options(
         workspace_id,
@@ -60,6 +63,42 @@ def main(workspace_id: str) -> None:
     assert "relationships" not in ws.model_dump()
 
 
+def team_demo(client: TFEClient, org: str) -> None:
+    # Grab any team in the org, then read it back asking for its users.
+    teams = list(client.teams.list(org))
+    if not teams:
+        print(f"\n(no teams in {org} to demo)")
+        return
+
+    team = client.teams.read(
+        teams[0].id,
+        TeamReadOptions(include=[TeamIncludeOpt.TEAM_USERS]),
+    )
+
+    # Typed hydration: team.users carries the full user bodies, not just ids.
+    print(f"\nteam: {team.name}  ({team.user_count} members)")
+    for user in team.users or []:
+        # Without include=users this would be an id-only stub (username == None).
+        print(f"  user (hydrated): {user.id}  {user.username}")
+
+    # The raw escape hatch is populated too, for relations not modelled as fields.
+    print(
+        f"  has_included={team.has_included}  relationships={sorted(team.relationships)}"
+    )
+    assert "included" not in team.model_dump()
+
+
+def main(workspace_id: str) -> None:
+    client = TFEClient()
+    workspace_demo(client, workspace_id)
+
+    org = os.environ.get("TFE_ORG")
+    if org:
+        team_demo(client, org)
+    else:
+        print("\n(set TFE_ORG to also run the team include demo)")
+
+
 if __name__ == "__main__":
     if len(sys.argv) != 2:
         print("usage: python examples/related_resources.py <workspace-id>")

src/pytfe/models/__init__.py

@@ -211,6 +211,8 @@
     OrganizationCreateOptions,
     OrganizationDefaultSettings,
     OrganizationDefaultSettingsUpdateOptions,
+    OrganizationIncludeOpt,
+    OrganizationReadOptions,
     OrganizationUpdateOptions,
     ReadRunQueueOptions,
     RunQueue,
@@ -298,6 +300,7 @@
     PolicySetParameterListOptions,
     PolicySetParameterUpdateOptions,
 )
+from .policy_set_version import PolicySetVersion
 from .policy_types import (
     EnforcementLevel,
     PolicyKind,
@@ -503,14 +506,15 @@
     TaskResultStatus,
     TaskResultStatusTimestamps,
 )
-from .task_stage import TaskStage
+from .task_stage import TaskStage, TaskStageIncludeOpt, TaskStageReadOptions
 from .team import (
     OrganizationAccess,
     Team,
     TeamCreateOptions,
     TeamIncludeOpt,
     TeamListOptions,
     TeamPermissions,
+    TeamReadOptions,
     TeamUpdateOptions,
 )
 from .team_project_access import (
@@ -837,6 +841,8 @@
     "OrganizationCreateOptions",
     "OrganizationDefaultSettings",
     "OrganizationDefaultSettingsUpdateOptions",
+    "OrganizationIncludeOpt",
+    "OrganizationReadOptions",
     "OrganizationUpdateOptions",
     # Org-token TTL policy
     "DEFAULT_MAX_TTL_MS",
@@ -869,6 +875,7 @@
     "TeamCreateOptions",
     "TeamIncludeOpt",
     "TeamListOptions",
+    "TeamReadOptions",
     "TeamUpdateOptions",
     # Team Tokens
     "CreatedByChoice",
@@ -972,6 +979,8 @@
     "RunEventReadOptions",
     # Task Stage & Task Result
     "TaskStage",
+    "TaskStageIncludeOpt",
+    "TaskStageReadOptions",
     "TaskResult",
     # Comments
     "Comment",
@@ -1037,6 +1046,7 @@
     "PolicySet",
     "PolicySetIncludeOpt",
     "PolicySetList",
+    "PolicySetVersion",
     "PolicySetAddPoliciesOptions",
     "PolicySetAddProjectsOptions",
     "PolicySetAddWorkspacesOptions",
@@ -1118,6 +1128,10 @@
 
 # Rebuild models with forward references after all models are loaded
 PolicyCheck.model_rebuild()
+PolicySetVersion.model_rebuild(
+    raise_errors=False,
+    _types_namespace={"PolicySet": PolicySet},
+)
 RegistryProvider.model_rebuild()
 RegistryProviderVersion.model_rebuild()
 RegistryProviderPlatform.model_rebuild()

src/pytfe/models/organization.py

@@ -88,6 +88,20 @@ class OrganizationCreateOptions(BaseModel):
     data_retention_policy_choice: dict | None = None
 
 
+class OrganizationIncludeOpt(str, Enum):
+    """Available include options for reading an organization."""
+
+    ORGANIZATION_SUBSCRIPTION = "subscription"
+
+
+class OrganizationReadOptions(BaseModel):
+    """Options for reading a single organization."""
+
+    model_config = ConfigDict(populate_by_name=True)
+
+    include: list[OrganizationIncludeOpt] | None = Field(None, alias="include")
+
+
 class ExecutionMode(str, Enum):
     REMOTE = "remote"
     AGENT = "agent"

src/pytfe/models/task_stage.py

@@ -84,3 +84,20 @@ class TaskStageListOptions(BaseModel):
     model_config = ConfigDict(populate_by_name=True)
 
     page_size: int | None = Field(None, alias="page[size]")
+
+
+class TaskStageIncludeOpt(str, Enum):
+    """Available include options for reading a task stage."""
+
+    TASK_STAGE_RUN = "run"
+    TASK_STAGE_RUN_WORKSPACE = "run.workspace"
+    TASK_STAGE_TASK_RESULTS = "task-results"
+    TASK_STAGE_POLICY_EVALUATIONS = "policy-evaluations"
+
+
+class TaskStageReadOptions(BaseModel):
+    """Options for reading a single task stage."""
+
+    model_config = ConfigDict(populate_by_name=True)
+
+    include: list[TaskStageIncludeOpt] | None = Field(None, alias="include")

src/pytfe/models/team.py

@@ -100,6 +100,14 @@ def valid(self) -> TeamListOptions:
         return self
 
 
+class TeamReadOptions(BaseModel):
+    """Options for reading a single team."""
+
+    model_config = ConfigDict(populate_by_name=True)
+
+    include: list[TeamIncludeOpt] | None = Field(None, alias="include")
+
+
 class OrganizationAccessOptions(BaseModel):
     model_config = ConfigDict(populate_by_name=True)
 

src/pytfe/resources/agents.py

@@ -12,6 +12,7 @@
 from collections.abc import Iterator
 from typing import Any, cast
 
+from .._jsonapi import attach_jsonapi
 from ..models.agent import (
     Agent,
     AgentListOptions,
@@ -139,7 +140,8 @@ def read(self, agent_id: str, options: AgentReadOptions | None = None) -> Agent:
         else:
             response = self.t.request("GET", path)
 
-        data = response.json()["data"]
+        payload = response.json()
+        data = payload["data"]
 
         # Extract agent data from response
         attr = data.get("attributes", {}) or {}
@@ -162,13 +164,17 @@ def read(self, agent_id: str, options: AgentReadOptions | None = None) -> Agent:
             "ip_address": _safe_str(attr.get("ip-address")),
         }
 
-        return Agent(
-            id=_safe_str(agent_data["id"]) or "",
-            name=agent_data["name"],
-            status=_safe_agent_status(agent_data["status"]),
-            version=agent_data["version"],
-            last_ping_at=cast(Any, agent_data["last_ping_at"]),
-            ip_address=agent_data["ip_address"],
+        return attach_jsonapi(
+            Agent(
+                id=_safe_str(agent_data["id"]) or "",
+                name=agent_data["name"],
+                status=_safe_agent_status(agent_data["status"]),
+                version=agent_data["version"],
+                last_ping_at=cast(Any, agent_data["last_ping_at"]),
+                ip_address=agent_data["ip_address"],
+            ),
+            data,
+            payload.get("included"),
         )
 
     def delete(self, agent_id: str) -> None:

src/pytfe/resources/configuration_version.py

@@ -3,6 +3,7 @@
 
 from __future__ import annotations
 
+import builtins
 import io
 from collections.abc import Iterator
 from typing import Any
@@ -124,7 +125,9 @@ def read_with_options(
 
         response = self.t.request("GET", path, params=params)
         response_data = response.json()
-        return self._parse_configuration_version(response_data["data"])
+        return self._parse_configuration_version(
+            response_data["data"], response_data.get("included")
+        )
 
     def upload(self, upload_url: str, path: str) -> None:
         """Upload configuration files from a directory path."""
@@ -254,7 +257,9 @@ def _manage_backing_data(self, cv_id: str, action: str) -> None:
         self.t.request("POST", path)
 
     def _parse_configuration_version(
-        self, data: dict[str, Any]
+        self,
+        data: dict[str, Any],
+        included: builtins.list[dict[str, Any]] | None = None,
     ) -> ConfigurationVersion:
         """Parse a configuration version from API response data."""
         if data is None:
@@ -287,4 +292,4 @@ def _parse_configuration_version(
             "links": data.get("links"),
         }
 
-        return attach_jsonapi(ConfigurationVersion(**cv_data), data)
+        return attach_jsonapi(ConfigurationVersion(**cv_data), data, included)