STG-5076: восстановить App Store авторизацию и выпустить CLI 2026.9.1

.dockerignore

@@ -153,3 +153,8 @@ cython_debug/
 /mdast_cli/downloaded_apps/
 /README_DOCKER.md
 /mdast_cli.egg-info/
+
+# Local credentials and cross-platform binaries are not part of the build context.
+.git
+appstore_sessions/
+mdast_cli/distribution_systems/appstore_client/bin/

.github/workflows/python-publish.yml

@@ -56,6 +56,14 @@ jobs:
         build
         --user
 
+    - uses: actions/setup-go@v6
+      with:
+        go-version: '1.27.1'
+        cache: false
+
+    - name: Build SAP helpers for release platforms
+      run: python tools/sap/build.py --target all
+
     - name: Build a binary wheel and a source tarball
       run: >-
         python -m
@@ -65,6 +73,9 @@ jobs:
         --outdir dist/
         .
 
+    - name: Verify packaged SAP helpers
+      run: python tools/sap/verify_artifacts.py
+
     - name: Publish distribution 📦 to Test PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:

.github/workflows/tests.yml

@@ -19,6 +19,13 @@ jobs:
       - name: Install dependencies
         run: pip install -r requirements.txt -r requirements-dev.txt
 
+      - uses: actions/setup-go@v6
+        with:
+          go-version: '1.27.1'
+          cache: false
+      - name: Build native SAP helper
+        run: python tools/sap/build.py --target host
+
       - name: Run full test suite
         run: python -m pytest -q
 

.gitignore

@@ -90,4 +90,6 @@ mdast_cli.egg-info/
 Thumbs.db
 
 # Application sessions (may contain sensitive data)
-appstore_sessions/
+appstore_sessions/
+# Reproducible native SAP helpers are built by CI, not committed.
+mdast_cli/distribution_systems/appstore_client/bin/

Dockerfile

@@ -1,14 +1,20 @@
-FROM python:3.9-slim
+FROM golang:1.27.1-bookworm AS go-toolchain
+FROM python:3.12-slim-bookworm AS sap-builder
+COPY --from=go-toolchain /usr/local/go /usr/local/go
+ENV PATH="/usr/local/go/bin:${PATH}"
+WORKDIR /src
+COPY tools/sap tools/sap
+RUN mkdir -p mdast_cli/distribution_systems/appstore_client/bin \
+    && python tools/sap/build.py --target host
 
+FROM python:3.12-slim-bookworm
 WORKDIR /mdast_cli
-
 COPY ./ /mdast_cli
-
-# Make apkeep_linux executable (if it exists)
+COPY --from=sap-builder /src/mdast_cli/distribution_systems/appstore_client/bin/ \
+    /mdast_cli/mdast_cli/distribution_systems/appstore_client/bin/
+COPY --from=sap-builder /src/mdast_cli/distribution_systems/appstore_client/IPATOOL-LICENSE \
+    /mdast_cli/mdast_cli/distribution_systems/appstore_client/IPATOOL-LICENSE
 RUN if [ -f /mdast_cli/apkeep_linux ]; then chmod +x /mdast_cli/apkeep_linux; fi
-
-RUN pip install -r requirements.txt
-
-ENV PYTHONPATH "${PYTHONPATH}:/mdast_cli"
-
-ENTRYPOINT ["python3", "mdast_cli/mdast_scan.py"]
+RUN pip install --no-cache-dir -r requirements.txt
+ENV PYTHONPATH="/mdast_cli"
+ENTRYPOINT ["python3", "mdast_cli/mdast_scan.py"]

MANIFEST.in

@@ -0,0 +1,5 @@
+include tools/sap/main.go tools/sap/build.py tools/sap/upstream.json
+include docs/STG-5076-appstore-sap.md
+recursive-include mdast_cli/distribution_systems/appstore_client/bin mdast-sap-*
+include mdast_cli/distribution_systems/appstore_client/IPATOOL-LICENSE
+include tools/sap/verify_artifacts.py

README.md

@@ -1179,3 +1179,22 @@ See LICENSE file for details.
 For issues, questions, or contributions, please visit the GitHub repository or contact support.
 
 **Note:** This documentation is maintained alongside the codebase. For the latest information, always refer to the version-specific documentation or the `--help` command output.
+
+### App Store authentication (2026.9.1)
+
+App Store login uses SAP-signed requests. The PyPI wheel includes native helpers
+for Linux, macOS and Windows (x86-64 and ARM64); Go is not required at runtime.
+On first login the signer downloads checksum-pinned Unicorn and Apple runtime
+assets. Allow outbound HTTPS to Apple, `swcdn.apple.com`, `s.mzstatic.com` and
+`files.pythonhosted.org` (Windows also uses the upstream runtime download hosts).
+The runtime cache is under the OS user cache directory in `ipatool/sap` and
+`ipatool/unicorn`. It contains runtime assets, not account sessions. A writable
+user cache directory is required. Alpine/musl is supported through its system
+loader. Offline first login is not supported; warm runtime assets avoid repeating
+these downloads. Account session caching continues to use `appstore_sessions`.
+
+For development from a source checkout, install Go 1.27.1 and run
+`python tools/sap/build.py --target host`. Release builds use `--target all`
+before building the wheel. See `docs/STG-5076-appstore-sap.md` for the protocol,
+source pin and acceptance criteria. The Apple frameworks are fetched directly
+from Apple at runtime and are not included in the package.

docs/STG-5076-appstore-sap.md

@@ -0,0 +1,69 @@
+# STG-5076: App Store SAP authentication
+
+## Problem and acceptance
+
+Fresh App Store authentication currently exhausts unsigned endpoint retries. A real
+CLI run from main 28c0b18 failed after 17 POSTs and 769 seconds. PyPI 2026.8.6 also
+fails and predates PR #149. Upstream ipatool v2.5.0 uses SAP-signed authentication.
+
+Implement locally first. Publishing a PR, merging and releasing are conditional on
+successful cold authentication and a valid IPA through the actual CLI entry point.
+A second process must download using its saved session. Then integrate the proven
+implementation into Markea and the monolith, test on dev and finish STG-5071 pod
+replacement acceptance. Do not automatically promote staging or production.
+
+## Design
+
+- Retain Python StoreClient and download/session interfaces.
+- Fetch Apple's bag over verified TLS and validate its authentication and SAP URLs.
+- Use a small subprocess adapter around the unmodified SAP signer from pinned
+  ipatool v2.5.0 (d5d0b56faf64e3fdef885d49e7928b390aadb6c7).
+- Build the adapter inside a verified temporary upstream source tree. Bundle
+  compiled helpers for Linux and macOS on amd64/arm64 in release artifacts, with
+  Windows support where build checks permit. No Go toolchain is required at runtime.
+  Keep the adapter source and reproducible build instructions in this repository.
+- The helper receives configuration and exact request bytes as bounded JSON lines
+  through stdin, returns signatures through stdout and never logs payloads. No
+  account credentials in process arguments. Initialize once per login, sign each
+  POST, close in finally. Enforce setup and sign deadlines and reap failed helpers.
+- Use X-Apple-ActionSignature (base64) for the exact serialized plist bytes. GUID
+  is the hex encoding of the hardware bytes passed to the signer.
+- Use only the bag authentication endpoint; validate every redirect before
+  forwarding credentials. Preserve body/attempt across pod redirects and transient
+  response retries. Limit redirects, protocol retries and HTTP retries explicitly.
+- No unsigned fallback. Classify missing/invalid SAP configuration, signature
+  failures, temporary Apple responses and authentication failures separately.
+- Upstream downloads its pinned, checksum-verified Unicorn runtime and Apple
+  framework assets on the first use. Document/cache these separately from account
+  sessions. Do not redistribute Apple frameworks in our package.
+
+## Checks
+
+- Unit coverage: exact signed bytes, bag validation, malicious redirects, malformed
+  and oversized helper responses, timeout/crash cleanup, 2FA/provider failures,
+  bounded retry behavior and unchanged successful download flow.
+- Helper build and protocol tests; complete existing CLI test suite.
+- Actual cold and warm CLI download with isolated cache and private credentials.
+- Verify published wheel and Docker version/content only after local acceptance.
+- Preserve Markea JSON session persistence and database locking.
+
+## Sources
+
+- https://tracker.yandex.ru/STG-5076
+- https://github.com/majd/ipatool/pull/525
+- https://github.com/majd/ipatool/releases/tag/v2.5.0
+
+## Local acceptance, 2026-09-07
+
+- Actual CLI cold login with empty account cache: signed POSTs 302 -> 200,
+  valid IPA 113,364,008 bytes, exit 0 in 35.13 seconds including first SAP setup.
+- Second CLI process: saved session restored, zero auth POSTs, valid IPA,
+  exit 0 in 7.13 seconds.
+- Installed wheel 2026.9.1, separate empty account cache: signed POSTs 302 -> 200,
+  valid IPA 113,364,009 bytes, exit 0 in 10.16 seconds (runtime assets already cached).
+- 241 Python tests pass. All six native targets build from the pinned Go module.
+- Linux amd64 Debian helper starts; Alpine amd64 completes real anonymous SAP
+  setup and creates a 501-byte signature in 25 seconds. Alpine uses its installed
+  musl loader explicitly because purego's default ELF interpreter names glibc.
+- Docker base aligned to Python 3.12, already required by the Python package.
+  Apple assets remain runtime downloads and are not bundled in released images.

mdast_cli/__init__.py

@@ -1 +1 @@
-__version__ = '2026.8.6'
+__version__ = '2026.9.1'

mdast_cli/distribution_systems/appstore.py

@@ -143,9 +143,7 @@ def login(self, force=False):
                 pickle.dump(self.store, file)
                 logger.info(f'Dumped session for {self.apple_id}')
         except StoreException as e:
-            raise RuntimeError(f'Failed to log into iTunes. This is either wrong credentials / an expired 2FA '
-                               f'code, or Apple refusing the request from this host. '
-                               f'Message: {e.req} {e.err_msg} {e.err_type}')
+            raise RuntimeError(f'Failed to log into iTunes: {e.err_msg} (type: {e.err_type})') from e
 
     def get_app_info(self, app_id=None, bundle_id=None, country='US'):
         if not app_id and not bundle_id:

mdast_cli/distribution_systems/appstore_client/IPATOOL-LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2021 Majd Alfhaily
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.