Build documentation with `mkdocs` by smokestacklightnin · Pull Request #38 · google/struct2tensor

smokestacklightnin · 2024-09-25T08:17:04Z

This PR builds the documentation using mkdocs.

The following is included in this PR:

Add mkdocs.yml configuration along with associated javascript and stylesheets
Automatically render API docs using mkdocstrings
Remove deprecated API docs stored as markdown files in favor of automatically generated API docs. This is advantageous for several reasons, including that developers no longer need to update both docstrings and separate markdown files to document their code.
Add modules that should be included to __all__ list in appropriate __init__.py files
Move example notebooks to documentation directory so they are rendered in docs
Add a deployment workflow for the repository
Fix formatting of code and buttons
Fix some broken links
Use README.md as homepage by creating a softlink to it from the documentation folder

In order to deploy the documentation, a maintainer will need to enable publishing from a branch on this repo for the gh-pages branch after this PR is merged.

Here is a preview of the new documentation

This is just a first step to including the correct items in the api docs

peytondmurray

A few changes mentioned below, but also:

Can we have a workflow that also builds (but doesn't deploy) the docs trigger when a PR is made? With the way it currently is currently building on push to master, contributors can easily break documentation without knowing it.

peytondmurray · 2024-09-25T20:47:36Z

.github/workflows/cd-docs.yml

+            mkdocs-material-
+
+      - name: Install Dependencies
+        run: pip install mkdocs mkdocs-material mkdocstrings[python] griffe-inherited-docstrings mkdocs-autorefs mkdocs-jupyter mkdocs-caption markdown-grid-tables


Can these be moved into a dedicated optional dependency? pip install '.[docs]' or similar so that the dependencies can actually be tracked.

.github/workflows/cd-docs.yml

smokestacklightnin · 2024-10-02T22:07:28Z

Can we have a workflow that also builds (but doesn't deploy) the docs trigger when a PR is made? With the way it currently is currently building on push to master, contributors can easily break documentation without knowing it.

Made this change, mirroring what we've done for the other repos we're working on.

peytondmurray · 2024-10-22T21:22:42Z

Hi @hassler-google, is there anything else I can do to help push this forward? It would be great to get this merged.

peytondmurray

Wow, this one was a lot of work. Thanks for this!

One request: can the docs_dir be the default docs/? Or is there some reason to have them be under g3doc?

smokestacklightnin · 2025-08-12T01:02:22Z

One request: can the docs_dir be the default docs/?

@peytondmurray Done

* Struct2Tensor 0.48.0 Release PiperOrigin-RevId: 743712449 * Build documentation with `mkdocs` (#38) * Add minimum viable mkdocs configuration * Add examples section * Add links to api docs and overviews * Get api docs up and running * Add `__all__` to `__init__.py` files to include correct classes This is just a first step to including the correct items in the api docs * Exclude test modules from docs * Use correct module * Add docs deployment workflow * Include all correct modules/classes/functions * Fix button rendering * Fix formatting, including admonitions * Fix broken link * Use README.md as homepage for docs * Fix links * Remove debug triggers from docs deployment workflow * Remove deprecated api docs * Fix triggers * Move docs requirements to common location * Only trigger on pushes to master * Remove unnecessary newline characters * Move docs location to default `docs` directory * only internal visibility change, no external change PiperOrigin-RevId: 755419474 * Automated Code Change PiperOrigin-RevId: 796979132 * Automated Code Change PiperOrigin-RevId: 803375108 * no-op PiperOrigin-RevId: 809038483 * Added pytype None checks to calculate.py PiperOrigin-RevId: 845226580 * Replace unicode escaped characters in ipynb files PiperOrigin-RevId: 856192266 * Fixes that needs to be available in the main branch in order to cherry-pick into Release branch (#49) * Update WORKSPACE * Update workspace.bzl * Migrate to Protobuf 4.23.4: custom Bazel rule for Python proto generation Protobuf 4.x removed py_proto_library and changed proto_library behavior. This commit: - Creates _py_proto_library_rule: a custom Bazel rule that accepts ProtoInfo or PyInfo, runs protoc to generate Python _pb2.py files, and provides PyInfo for Python deps - Replaces deprecated py_proto_library calls with custom rule implementation - Configures protoc proto_path to include workspace dirs and external dependencies - Adds local any.proto copies to bypass sandbox limitations in proto imports - Applies compatibility patches to TensorFlow and TensorFlow Metadata dependencies - Updates all s2t_proto_library_py calls to work with new implementation Fixes: Protobuf 4.23.4 compatibility for struct2tensor build system * Add compatibility patches for older Abseil and Protobuf 4.x This patch extends tensorflow.patch with changes to support: 1. Abseil backward compatibility: - Add absl_base_prefetch.h compatibility shim for older Abseil versions that lack absl/base/prefetch.h - Update prefetch includes across TensorFlow/TSL to use the shim - Add inline implementation of NullTerminatedMessage for older Abseil versions without StatusMessageAsCStr 2. Protobuf 4.x support: - Refactor cc_proto_library to use native proto_library and cc_proto_library rules instead of custom proto_gen - Implement custom _tsl_py_proto_library_rule to replace the built-in py_proto_library removed in Protobuf 4.x - Update proto library generation to depend on generated cc_proto_library targets 3. Build cleanup: - Remove unused absl/strings:string_view dependency - Update BUILD files to reflect new dependency structure These changes enable building struct2tensor with older Abseil versions while supporting Protobuf 4.x, improving compatibility across different dependency versions. * chore: upgrade Protobuf dependency from 4.23.4 to 4.25.6 * Move TFMD patch to tfmd.patch; Protobuf 4.x compatibility Remove vendored google/protobuf/any.proto and obsolete third_party/README.md * fix: Use native prefix for Bazel rules in macros Prefix cc_binary and cc_library with native. in struct2tensor.bzl * Update Dockerfile * Fix struct2tensor integration in TensorFlow Serving Docker build Apply rules_cc patch before adding struct2tensor local repository to ensure proper Bazel configuration. * Update RELEASE.md to sync with main branch * Update version.py --------- Co-authored-by: Gagandeep Singh <gdp.1807@gmail.com> * docs(release): Update RELEASE.md and version for 0.48.0 * Update Dockerfile to build with serving version 2.17.1 * Build and stage dynamic op libraries before wheel creation (#51) Add explicit build and copy steps for dynamic op libraries (.so files) in build_common.sh. This ensures all six dynamic library targets are built and copied to struct2tensor/ops/ before the wheel is packaged. Previously, the stamp_wheel step in build_manylinux.sh would fail with patchelf errors ("No such file or directory") because the .so files were expected in struct2tensor/ops/ but were not being copied from bazel-bin. This fix: - Explicitly builds each dynamic library target - Copies the built .so files from bazel-bin to the staging directory - Runs before bazel run :build_pip_package to ensure files are present Fixes the missing _decode_proto_map_op.so, _decode_proto_sparse_op.so and related dynamic library warnings during wheel packaging. * Enable OSS Bazel builds by fixing dependencies and proto paths (#50) - Add py_proto_library macro for OSS compatibility - Replace gunit_main with googletest gtest_main - Comment out Google-internal dependencies and missing test files - Fix proto import to use workspace-relative path - Remove invalid third_party load statements Fixes build errors when running `bazel build //...` in OSS environment. Co-authored-by: Madhur Karampudi <142544288+vkarampudi@users.noreply.github.com> --------- Co-authored-by: struct2tensor-team <tensorflow-extended-nonhuman@googlegroups.com> Co-authored-by: Gagandeep Singh <gdp.1807@gmail.com>

smokestacklightnin added 16 commits September 24, 2024 17:28

Add minimum viable mkdocs configuration

89c36bd

Add examples section

952f431

Add links to api docs and overviews

b44c44f

Get api docs up and running

660b153

Add __all__ to __init__.py files to include correct classes

5ee610f

This is just a first step to including the correct items in the api docs

Exclude test modules from docs

4d3fef0

Use correct module

9dd8dc2

Add docs deployment workflow

fed0cf8

Include all correct modules/classes/functions

beb9c37

Fix button rendering

7dc3f4b

Fix formatting, including admonitions

557ab21

Fix broken link

a31f0b2

Use README.md as homepage for docs

1c7d161

Fix links

6790d80

Remove debug triggers from docs deployment workflow

8360a97

Remove deprecated api docs

00d22bc

peytondmurray requested changes Sep 25, 2024

View reviewed changes

Fix triggers

920f56b

smokestacklightnin added 3 commits October 2, 2024 15:16

Move docs requirements to common location

5baf266

Only trigger on pushes to master

043aee3

Remove unnecessary newline characters

6fb6fbc

vkarampudi requested a review from hassler-google October 16, 2024 23:32

hassler-google mentioned this pull request Oct 24, 2024

Apply fixes for linting issues #39

Draft

Merge remote-tracking branch 'upstream/master' into ci/docs/use-mkdocs

213c53f

peytondmurray approved these changes Aug 9, 2025

View reviewed changes

Move docs location to default docs directory

85345a6

smokestacklightnin force-pushed the ci/docs/use-mkdocs branch from 00ed1d7 to 85345a6 Compare August 12, 2025 01:00

smokestacklightnin requested a review from peytondmurray August 12, 2025 01:02

peytondmurray approved these changes Aug 12, 2025

View reviewed changes

peytondmurray merged commit 83f9b1f into google:master Aug 12, 2025
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build documentation with `mkdocs`#38

Build documentation with `mkdocs`#38
peytondmurray merged 22 commits intogoogle:masterfrom
smokestacklightnin:ci/docs/use-mkdocs

smokestacklightnin commented Sep 25, 2024 •

edited

Loading

Uh oh!

peytondmurray left a comment

Uh oh!

peytondmurray Sep 25, 2024

Uh oh!

smokestacklightnin Oct 2, 2024

Uh oh!

Uh oh!

smokestacklightnin commented Oct 2, 2024

Uh oh!

peytondmurray commented Oct 22, 2024

Uh oh!

peytondmurray left a comment

Uh oh!

smokestacklightnin commented Aug 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

smokestacklightnin commented Sep 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

peytondmurray left a comment

Choose a reason for hiding this comment

Uh oh!

peytondmurray Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

smokestacklightnin Oct 2, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

smokestacklightnin commented Oct 2, 2024

Uh oh!

peytondmurray commented Oct 22, 2024

Uh oh!

peytondmurray left a comment

Choose a reason for hiding this comment

Uh oh!

smokestacklightnin commented Aug 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

smokestacklightnin commented Sep 25, 2024 •

edited

Loading