Consolidate launchpad interaction scripts into single launchpad_copy.py

[rtibblesbot]

Phase 1: Create consolidated script with shared infrastructure

Phase 1: Create Consolidated Script with Shared Infrastructure

Goal

Create scripts/launchpad_copy.py with the shared LaunchpadWrapper class, argparse skeleton with two subcommands, and shared utilities. This phase builds the foundation that phases 2 and 3 plug into.

---

Task 1: Create the shared infrastructure module

Files:

• Create: scripts/launchpad_copy.py

Step 1: Write the script skeleton with imports, constants, and argparse

Create scripts/launchpad_copy.py with:

#!/usr/bin/env python3
"""Consolidated Launchpad PPA copy tool.

Subcommands:
  copy-to-series  Copy packages from source series to all other supported series within a PPA.
  promote         Copy all published packages from one PPA to another.
"""

import argparse
import functools
import logging
import subprocess
import sys
import time
from collections import defaultdict

from launchpadlib.launchpad import Launchpad
import lazr.restfulclient.errors as lre

# --- Constants ---
PPA_OWNER = "learningequality"
PROPOSED_PPA_NAME = "kolibri-proposed"
RELEASE_PPA_NAME = "kolibri"
PACKAGE_WHITELIST = ["kolibri-server"]
SOURCE_SERIES = "jammy"
FALLBACK_SERIESES = ["plucky", "noble", "jammy", "focal"]
POCKET = "Release"
APP_NAME = "ppa-kolibri-server-copy-packages"

log = logging.getLogger(APP_NAME)

STARTUP_TIME = LAST_LOG_TIME = time.time()
REQUESTS = LAST_REQUESTS = 0

Step 2: Add the caching decorators (preserved from copy_to_other_distributions.py)

# --- Caching decorators ---

class once:
    """A @property computed only once per instance (aka @reify / @Lazy)."""

    def __init__(self, fn):
        self.fn = fn

    def __get__(self, obj, type=None):
        value = self.fn(obj)
        setattr(obj, self.fn.__name__, value)
        return value


def cache(fn):
    """Trivial memoization decorator."""
    _cache = fn.cache = {}

    @functools.wraps(fn)
    def inner(*args):
        try:
            return _cache[args]
        except KeyError:
            value = _cache[args] = fn(*args)
            return value
        except TypeError:
            raise TypeError(
                "%s argument types preclude caching: %s" % (fn.__name__, repr(args))
            )

    return inner

Step 3: Add the LaunchpadWrapper class with shared auth, PPA lookup, package filtering

This class is adapted from copy_to_other_distributions.py but generalized to support looking up both PPAs:

# --- LaunchpadWrapper ---

class LaunchpadWrapper:
    """Cached wrapper around the Launchpad API."""

    def __init__(self):
        self.queue = defaultdict(set)

    @once
    def lp(self):
        log.debug("Logging in...")
        return Launchpad.login_with(
            application_name=APP_NAME,
            service_root="production",
        )

    @once
    def owner(self):
        lp = self.lp
        log.debug("Getting the owner...")
        return lp.people[PPA_OWNER]

    def get_ppa(self, name):
        owner = self.owner
        log.debug("Getting PPA: %s...", name)
        return owner.getPPAByName(name=name)

    @once
    def proposed_ppa(self):
        return self.get_ppa(PROPOSED_PPA_NAME)

    @once
    def release_ppa(self):
        return self.get_ppa(RELEASE_PPA_NAME)

    @cache
    def get_series(self, name):
        ppa = self.proposed_ppa
        log.debug("Locating the series: %s...", name)
        return ppa.distribution.getSeries(name_or_version=name)

    @cache
    def get_published_sources(self, ppa, series_name=None, status=None):
        kwargs = {}
        if series_name:
            kwargs["distro_series"] = self.get_series(series_name)
        if status:
            kwargs["status"] = status
        kwargs["order_by_date"] = True
        log.debug("Listing source packages...")
        return ppa.getPublishedSources(**kwargs)

    @cache
    def get_builds_for_source(self, source):
        log.debug(
            "Listing %s builds for %s %s...",
            source.distro_series_link.rpartition("/")[-1],
            source.source_package_name,
            source.source_package_version,
        )
        return source.getBuilds()

    @cache
    def get_source_packages(self, ppa, series_name, package_names=None):
        """Return {package_name: {version: source, ...}, ...}"""
        res = defaultdict(dict)
        for source in self.get_published_sources(ppa, series_name):
            name = source.source_package_name
            if package_names is not None and name not in package_names:
                continue
            res[name][source.source_package_version] = source
        return res

    def get_source_for(self, ppa, name, version, series_name):
        sources = self.get_source_packages(ppa, series_name)
        return sources.get(name, {}).get(version)

    def is_missing(self, ppa, name, version, series_name):
        return self.get_source_for(ppa, name, version, series_name) is None

    def get_builds_for(self, ppa, name, version, series_name):
        source = self.get_source_for(ppa, name, version, series_name)
        if not source:
            return None
        return self.get_builds_for_source(source)

    def has_published_binaries(self, ppa, name, version, series_name):
        builds = self.get_builds_for(ppa, name, version, series_name)
        return not builds or builds[0].buildstate == "Successfully built"

    @cache
    def get_usable_sources(self, ppa, package_names, series_name):
        res = []
        for source in self.get_published_sources(ppa, series_name):
            name = source.source_package_name
            if name not in package_names:
                continue
            version = source.source_package_version
            if source.status in ("Superseded", "Deleted", "Obsolete"):
                log.info(
                    "%s %s is %s in %s", name, version, source.status.lower(), series_name
                )
                continue
            if source.status != "Published":
                log.warning(
                    "%s %s is %s in %s", name, version, source.status.lower(), series_name
                )
                continue
            res.append((name, version))
        return res

    def queue_copy(self, name, source_series, target_series, pocket):
        self.queue[source_series, target_series, pocket].add(name)

    def perform_queued_copies(self, ppa):
        first = True
        for (source_series, target_series, pocket), names in self.queue.items():
            if not names:
                continue
            if first:
                log.info("")
                first = False
            log.info("Copying %s to %s", ", ".join(sorted(names)), target_series)
            ppa.syncSources(
                from_archive=ppa,
                to_series=target_series,
                to_pocket=pocket,
                include_binaries=True,
                source_names=sorted(names),
            )

Step 4: Add utility functions (logging, series discovery)

# --- Utilities ---

class DebugFormatter(logging.Formatter):
    def format(self, record):
        global LAST_LOG_TIME, LAST_REQUESTS
        msg = super().format(record)
        if msg.startswith("  "):
            return msg
        now = time.time()
        elapsed = now - STARTUP_TIME
        delta = now - LAST_LOG_TIME
        LAST_LOG_TIME = now
        delta_requests = REQUESTS - LAST_REQUESTS
        LAST_REQUESTS = REQUESTS
        return "\n%.3fs (%+.3fs) [%d/+%d] %s" % (
            elapsed, delta, REQUESTS, delta_requests, msg
        )


def enable_http_debugging():
    import httplib2
    httplib2.debuglevel = 1


def install_request_counter():
    import httplib2
    orig = httplib2.Http.request

    @functools.wraps(orig)
    def wrapper(*args, **kw):
        global REQUESTS
        REQUESTS += 1
        return orig(*args, **kw)

    httplib2.Http.request = wrapper


def set_up_logging(level=logging.INFO):
    handler = logging.StreamHandler(sys.stdout)
    if level == logging.DEBUG:
        handler.setFormatter(DebugFormatter())
    log.addHandler(handler)
    log.setLevel(level)


def get_supported_series(source_series):
    """Discover supported Ubuntu series dynamically, with hardcoded fallback."""
    try:
        out = subprocess.check_output(
            ["ubuntu-distro-info", "--supported"], text=True
        ).strip()
        all_series = out.split()
        series = [s for s in all_series if s and s != source_series]
        log.info("Dynamic series discovery:")
        log.info("  Target series (will copy to): %s", ", ".join(series))
        return series
    except Exception as e:
        log.warning("Failed to get dynamic series list: %s", e)
        log.info(
            "Falling back to hardcoded target series: %s", ", ".join(FALLBACK_SERIESES)
        )
        return FALLBACK_SERIESES

Step 5: Add argparse main with subcommand skeleton

# --- CLI ---

def build_parser():
    parser = argparse.ArgumentParser(
        description="Launchpad PPA copy tool for kolibri-server packages."
    )
    parser.add_argument(
        "-v", "--verbose", action="count", default=0,
        help="Increase verbosity (use -vv for debug)."
    )
    parser.add_argument("-q", "--quiet", action="store_true", help="Suppress info output.")
    parser.add_argument("--debug", action="store_true", help="Enable HTTP debug output.")

    subparsers = parser.add_subparsers(dest="command", required=True)

    # copy-to-series subcommand
    subparsers.add_parser(
        "copy-to-series",
        help="Copy packages from source series to all other supported series within a PPA.",
    )

    # promote subcommand
    subparsers.add_parser(
        "promote",
        help="Promote published packages from kolibri-proposed to kolibri PPA.",
    )

    return parser


def configure_logging(args):
    if args.quiet:
        set_up_logging(logging.WARNING)
    elif args.debug:
        enable_http_debugging()
        install_request_counter()
        set_up_logging(logging.DEBUG)
    elif args.verbose > 1:
        install_request_counter()
        set_up_logging(logging.DEBUG)
    else:
        set_up_logging(logging.INFO)


def main():
    parser = build_parser()
    args = parser.parse_args()
    configure_logging(args)

    if args.command == "copy-to-series":
        return cmd_copy_to_series(args)
    elif args.command == "promote":
        return cmd_promote(args)


def cmd_copy_to_series(args):
    """Placeholder — implemented in Phase 2."""
    raise NotImplementedError("copy-to-series not yet implemented")


def cmd_promote(args):
    """Placeholder — implemented in Phase 3."""
    raise NotImplementedError("promote not yet implemented")


if __name__ == "__main__":
    raise SystemExit(main())

Step 6: Verify --help works

Run: python3 scripts/launchpad_copy.py --help
Expected: Shows usage with subcommands, exits 0.

Run: python3 scripts/launchpad_copy.py copy-to-series --help
Expected: Shows copy-to-series help, exits 0.

Run: python3 scripts/launchpad_copy.py promote --help
Expected: Shows promote help, exits 0.

Step 7: Commit

git add scripts/launchpad_copy.py
git commit -m "feat: add consolidated launchpad_copy.py with shared infrastructure

Shared LaunchpadWrapper, caching, logging, series discovery, and argparse
skeleton with copy-to-series and promote subcommands. Fixes verbose default
to 0 to prevent TypeError.

Refs #106"

Phase 2: Implement `copy-to-series` subcommand

Phase 2: Implement copy-to-series Subcommand

Goal

Replace cmd_copy_to_series placeholder with full logic from copy_to_other_distributions.py. This replicates the behavior of iterating usable sources in the source series, checking each target series, queueing copies for missing packages, and performing the copies.

---

Task 2: Implement cmd_copy_to_series

Files:

• Modify: scripts/launchpad_copy.py (replace cmd_copy_to_series placeholder)

Step 1: Replace cmd_copy_to_series with the full implementation

Replace the placeholder function with:

def cmd_copy_to_series(args):
    """Copy packages from source series to all other supported Ubuntu series."""
    log.info(
        "Spinning up the Launchpad API to copy targets in %s",
        ", ".join(PACKAGE_WHITELIST),
    )

    lp = LaunchpadWrapper()
    ppa = lp.proposed_ppa

    for name, version in lp.get_usable_sources(
        ppa, tuple(PACKAGE_WHITELIST), SOURCE_SERIES
    ):
        mentioned = False
        notices = []
        target_series_names = get_supported_series(SOURCE_SERIES)
        for target_series_name in target_series_names:
            source = lp.get_source_for(ppa, name, version, target_series_name)
            if source is None:
                mentioned = True
                log.info("%s %s missing from %s", name, version, target_series_name)
                if lp.has_published_binaries(ppa, name, version, SOURCE_SERIES):
                    lp.queue_copy(name, SOURCE_SERIES, target_series_name, POCKET)
                else:
                    builds = lp.get_builds_for(ppa, name, version, SOURCE_SERIES)
                    if builds:
                        log.info(
                            "  but it isn't built yet (state: %s) - %s",
                            builds[0].buildstate,
                            builds[0].web_link,
                        )
            elif source.status != "Published":
                notices.append(
                    "  but it is %s in %s" % (source.status.lower(), target_series_name)
                )
            elif not lp.has_published_binaries(ppa, name, version, target_series_name):
                builds = lp.get_builds_for(ppa, name, version, target_series_name)
                if builds:
                    notices.append(
                        "  but it isn't built yet for %s (state: %s) - %s"
                        % (target_series_name, builds[0].buildstate, builds[0].web_link)
                    )
        if not mentioned or notices:
            log.info("%s %s", name, version)
            for notice in notices:
                log.info(notice)

    lp.perform_queued_copies(ppa)
    log.debug("All done")
    return 0

Step 2: Verify --help still works

Run: python3 scripts/launchpad_copy.py --help
Expected: exits 0, shows both subcommands.

Step 3: Commit

git add scripts/launchpad_copy.py
git commit -m "feat: implement copy-to-series subcommand

Replicates copy_to_other_distributions.py behavior: iterates usable sources
in jammy, discovers supported series dynamically, queues and performs copies
for missing packages.

Refs #106"

Phase 3: Implement `promote` subcommand

Phase 3: Implement promote Subcommand

Goal

Replace cmd_promote placeholder with logic from copy_package_proposed_to_ppa.py. This copies all published, whitelisted packages from kolibri-proposed to kolibri PPA, handling obsolete series gracefully.

---

Task 3: Implement cmd_promote

Files:

• Modify: scripts/launchpad_copy.py (replace cmd_promote placeholder)

Step 1: Replace cmd_promote with the full implementation

Replace the placeholder function with:

def cmd_promote(args):
    """Promote published packages from kolibri-proposed to kolibri PPA."""
    log.info("Promoting packages from %s to %s", PROPOSED_PPA_NAME, RELEASE_PPA_NAME)

    lp = LaunchpadWrapper()
    source_ppa = lp.proposed_ppa
    dest_ppa = lp.release_ppa

    packages = source_ppa.getPublishedSources(status="Published", order_by_date=True)

    copied_any = False
    for pkg in packages:
        if pkg.source_package_name not in PACKAGE_WHITELIST:
            continue
        try:
            log.info(
                "Copying %s %s (%s) to %s",
                pkg.source_package_name,
                pkg.source_package_version,
                pkg.distro_series_link,
                RELEASE_PPA_NAME,
            )
            dest_ppa.copyPackage(
                from_archive=source_ppa,
                include_binaries=True,
                to_pocket=pkg.pocket,
                source_name=pkg.source_package_name,
                version=pkg.source_package_version,
            )
            copied_any = True
        except lre.BadRequest as e:
            if "is obsolete and will not accept new uploads" in str(e):
                log.info(
                    "Skip obsolete series for %s %s",
                    pkg.source_package_name,
                    pkg.source_package_version,
                )
            else:
                raise

    if not copied_any:
        log.info("No eligible packages to promote.")
    else:
        log.info("Promotion requests submitted.")

    return 0

Step 2: Verify --help still works

Run: python3 scripts/launchpad_copy.py --help
Expected: exits 0.

Step 3: Commit

git add scripts/launchpad_copy.py
git commit -m "feat: implement promote subcommand

Replicates copy_package_proposed_to_ppa.py behavior: copies all published,
whitelisted packages from kolibri-proposed to kolibri PPA with obsolete
series error handling.

Refs #106"

Phase 4: Unit tests with pytest + vcrpy/mocking

Phase 4: Unit Tests with pytest + vcrpy/mocking

Goal

Add comprehensive unit tests for scripts/launchpad_copy.py using pytest, unittest.mock, and vcrpy where appropriate. Since the Launchpad API requires credentials, most tests will use mocking. VCR is set up for potential future use with recorded cassettes.

---

Task 4: Set up test infrastructure

Files:

• Create: tests/conftest.py
• Create: tests/vcr_config.py

Step 1: Create test infrastructure files

tests/conftest.py:

"""Shared pytest fixtures for launchpad_copy tests."""

tests/vcr_config.py (following the pattern from kolibri-installer-debian):

import pytest

try:
    import vcr

    my_vcr = vcr.VCR(
        cassette_library_dir="tests/cassettes",
        record_mode="new_episodes",
        path_transformer=vcr.VCR.ensure_suffix(".yaml"),
        filter_headers=["authorization"],
    )
except ImportError:

    class VCR:
        def use_cassette(self, *args, **kwargs):
            return pytest.mark.skip("vcrpy is not available on this Python version")

    my_vcr = VCR()

Step 2: Create the cassettes directory

mkdir -p tests/cassettes

Step 3: Commit

git add tests/conftest.py tests/vcr_config.py tests/cassettes/
git commit -m "chore: add test infrastructure with pytest and vcrpy config

Refs #106"

---

Task 5: Write unit tests for shared infrastructure

Files:

• Create: tests/test_launchpad_copy.py

Step 1: Write tests for caching decorators, series discovery, logging setup, and CLI parsing

"""Tests for scripts/launchpad_copy.py"""

import argparse
import logging
import subprocess
from unittest import mock

import pytest

# Add scripts dir to path so we can import the module
import sys
import os

sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "scripts"))

import launchpad_copy


class TestOnceDescriptor:
    """Test the @once caching property descriptor."""

    def test_computes_value_once(self):
        call_count = 0

        class MyClass:
            @launchpad_copy.once
            def value(self):
                nonlocal call_count
                call_count += 1
                return 42

        obj = MyClass()
        assert obj.value == 42
        assert obj.value == 42
        assert call_count == 1

    def test_separate_instances_get_separate_values(self):
        class MyClass:
            def __init__(self, val):
                self._val = val

            @launchpad_copy.once
            def value(self):
                return self._val

        a = MyClass(1)
        b = MyClass(2)
        assert a.value == 1
        assert b.value == 2


class TestCacheDecorator:
    """Test the @cache memoization decorator."""

    def test_caches_result(self):
        call_count = 0

        @launchpad_copy.cache
        def add(a, b):
            nonlocal call_count
            call_count += 1
            return a + b

        assert add(1, 2) == 3
        assert add(1, 2) == 3
        assert call_count == 1

    def test_different_args_not_cached(self):
        call_count = 0

        @launchpad_copy.cache
        def add(a, b):
            nonlocal call_count
            call_count += 1
            return a + b

        assert add(1, 2) == 3
        assert add(3, 4) == 7
        assert call_count == 2


class TestGetSupportedSeries:
    """Test dynamic series discovery with fallback."""

    def test_dynamic_discovery(self):
        with mock.patch("subprocess.check_output") as mock_cmd:
            mock_cmd.return_value = "focal jammy noble plucky\n"
            result = launchpad_copy.get_supported_series("jammy")
            assert "jammy" not in result
            assert "focal" in result
            assert "noble" in result
            assert "plucky" in result

    def test_fallback_on_error(self):
        with mock.patch("subprocess.check_output", side_effect=FileNotFoundError):
            result = launchpad_copy.get_supported_series("jammy")
            assert result == launchpad_copy.FALLBACK_SERIESES


class TestBuildParser:
    """Test CLI argument parsing."""

    def test_help_exits_zero(self):
        parser = launchpad_copy.build_parser()
        with pytest.raises(SystemExit) as exc_info:
            parser.parse_args(["--help"])
        assert exc_info.value.code == 0

    def test_copy_to_series_subcommand(self):
        parser = launchpad_copy.build_parser()
        args = parser.parse_args(["copy-to-series"])
        assert args.command == "copy-to-series"

    def test_promote_subcommand(self):
        parser = launchpad_copy.build_parser()
        args = parser.parse_args(["promote"])
        assert args.command == "promote"

    def test_verbose_defaults_to_zero(self):
        parser = launchpad_copy.build_parser()
        args = parser.parse_args(["copy-to-series"])
        assert args.verbose == 0

    def test_verbose_count(self):
        parser = launchpad_copy.build_parser()
        args = parser.parse_args(["-vv", "copy-to-series"])
        assert args.verbose == 2

    def test_no_subcommand_errors(self):
        parser = launchpad_copy.build_parser()
        with pytest.raises(SystemExit) as exc_info:
            parser.parse_args([])
        assert exc_info.value.code != 0


class TestConfigureLogging:
    """Test logging configuration."""

    def test_quiet_sets_warning_level(self):
        parser = launchpad_copy.build_parser()
        args = parser.parse_args(["-q", "copy-to-series"])
        launchpad_copy.configure_logging(args)
        assert launchpad_copy.log.level == logging.WARNING

    def test_default_sets_info_level(self):
        parser = launchpad_copy.build_parser()
        args = parser.parse_args(["copy-to-series"])
        launchpad_copy.configure_logging(args)
        assert launchpad_copy.log.level == logging.INFO

    def test_no_typeerror_without_verbose(self):
        """Regression: --verbose defaulting to None caused TypeError."""
        parser = launchpad_copy.build_parser()
        args = parser.parse_args(["copy-to-series"])
        # Should not raise TypeError
        launchpad_copy.configure_logging(args)

Step 2: Run the tests

Run: python3 -m pytest tests/test_launchpad_copy.py -v
Expected: All tests pass.

Step 3: Commit

git add tests/test_launchpad_copy.py
git commit -m "test: add unit tests for shared infrastructure

Tests for caching decorators, series discovery, CLI parsing, logging
configuration, and verbose default regression fix.

Refs #106"

---

Task 6: Write unit tests for LaunchpadWrapper and subcommands

Files:

• Modify: tests/test_launchpad_copy.py (append new test classes)

Step 1: Add mock-based tests for LaunchpadWrapper

Append to tests/test_launchpad_copy.py:

class TestLaunchpadWrapper:
    """Test LaunchpadWrapper with mocked Launchpad API."""

    def _make_mock_source(self, name, version, status="Published", series="jammy"):
        src = mock.MagicMock()
        src.source_package_name = name
        src.source_package_version = version
        src.status = status
        src.distro_series_link = f"https://api.launchpad.net/devel/ubuntu/{series}"
        src.pocket = "Release"
        return src

    def test_get_usable_sources_filters_by_whitelist(self):
        wrapper = launchpad_copy.LaunchpadWrapper()
        ppa = mock.MagicMock()

        src1 = self._make_mock_source("kolibri-server", "1.0")
        src2 = self._make_mock_source("other-package", "1.0")
        ppa.getPublishedSources.return_value = [src1, src2]

        # Bypass the caching for test isolation
        with mock.patch.object(wrapper, "get_published_sources", return_value=[src1, src2]):
            result = launchpad_copy.LaunchpadWrapper.get_usable_sources.__wrapped__(
                wrapper, ppa, ("kolibri-server",), "jammy"
            )

        assert len(result) == 1
        assert result[0] == ("kolibri-server", "1.0")

    def test_get_usable_sources_skips_superseded(self):
        wrapper = launchpad_copy.LaunchpadWrapper()
        ppa = mock.MagicMock()

        src = self._make_mock_source("kolibri-server", "1.0", status="Superseded")

        with mock.patch.object(wrapper, "get_published_sources", return_value=[src]):
            result = launchpad_copy.LaunchpadWrapper.get_usable_sources.__wrapped__(
                wrapper, ppa, ("kolibri-server",), "jammy"
            )

        assert len(result) == 0


class TestCmdPromote:
    """Test promote subcommand with mocked Launchpad."""

    def test_promote_copies_whitelisted_packages(self):
        with mock.patch.object(launchpad_copy, "LaunchpadWrapper") as MockWrapper:
            instance = MockWrapper.return_value

            pkg = mock.MagicMock()
            pkg.source_package_name = "kolibri-server"
            pkg.source_package_version = "1.0"
            pkg.distro_series_link = "https://api.launchpad.net/devel/ubuntu/jammy"
            pkg.pocket = "Release"

            source_ppa = mock.MagicMock()
            source_ppa.getPublishedSources.return_value = [pkg]
            dest_ppa = mock.MagicMock()

            instance.proposed_ppa = source_ppa
            instance.release_ppa = dest_ppa

            parser = launchpad_copy.build_parser()
            args = parser.parse_args(["promote"])
            result = launchpad_copy.cmd_promote(args)

            dest_ppa.copyPackage.assert_called_once()
            assert result == 0

    def test_promote_skips_non_whitelisted(self):
        with mock.patch.object(launchpad_copy, "LaunchpadWrapper") as MockWrapper:
            instance = MockWrapper.return_value

            pkg = mock.MagicMock()
            pkg.source_package_name = "some-other-pkg"
            pkg.source_package_version = "1.0"

            source_ppa = mock.MagicMock()
            source_ppa.getPublishedSources.return_value = [pkg]
            dest_ppa = mock.MagicMock()

            instance.proposed_ppa = source_ppa
            instance.release_ppa = dest_ppa

            parser = launchpad_copy.build_parser()
            args = parser.parse_args(["promote"])
            result = launchpad_copy.cmd_promote(args)

            dest_ppa.copyPackage.assert_not_called()
            assert result == 0

    def test_promote_handles_obsolete_series(self):
        with mock.patch.object(launchpad_copy, "LaunchpadWrapper") as MockWrapper:
            instance = MockWrapper.return_value

            pkg = mock.MagicMock()
            pkg.source_package_name = "kolibri-server"
            pkg.source_package_version = "1.0"
            pkg.distro_series_link = "https://api.launchpad.net/devel/ubuntu/xenial"
            pkg.pocket = "Release"

            source_ppa = mock.MagicMock()
            source_ppa.getPublishedSources.return_value = [pkg]
            dest_ppa = mock.MagicMock()
            dest_ppa.copyPackage.side_effect = launchpad_copy.lre.BadRequest(
                b"", b"xenial is obsolete and will not accept new uploads"
            )

            instance.proposed_ppa = source_ppa
            instance.release_ppa = dest_ppa

            parser = launchpad_copy.build_parser()
            args = parser.parse_args(["promote"])
            # Should not raise
            result = launchpad_copy.cmd_promote(args)
            assert result == 0

Step 2: Run all tests

Run: python3 -m pytest tests/test_launchpad_copy.py -v
Expected: All tests pass.

Step 3: Commit

git add tests/test_launchpad_copy.py
git commit -m "test: add unit tests for LaunchpadWrapper and promote subcommand

Mock-based tests for package filtering, whitelist enforcement, and
obsolete series error handling.

Refs #106"

Phase 5: CI setup and workflow updates, delete old scripts

Phase 5: CI Setup, Workflow Updates, Delete Old Scripts

Goal

Set up a GitHub Actions workflow for Python tests, update build_debian.yml to use the new script, and delete the old scripts.

---

Task 7: Create CI workflow for Python tests

Files:

• Create: .github/workflows/python_tests.yml

Step 1: Create the workflow file

name: Python Tests
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.10", "3.12"]
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install pytest vcrpy launchpadlib
      - name: Run tests
        run: |
          python -m pytest tests/ -v

Step 2: Verify actionlint passes

Run: pre-commit run actionlint --all-files
Expected: Passes.

Step 3: Commit

git add .github/workflows/python_tests.yml
git commit -m "ci: add GitHub Actions workflow for Python tests

Runs pytest on Python 3.10 and 3.12 on push/PR to main.

Refs #106"

---

Task 8: Update build_debian.yml to use new script

Files:

• Modify: .github/workflows/build_debian.yml

Step 1: Update the copy_to_other_distributions job

Change the script invocation from:

          python3 scripts/copy_to_other_distributions.py

to:

          python3 scripts/launchpad_copy.py copy-to-series

Step 2: Update the copy_package_from_proposed_to_ppa job

Change the script invocation from:

          python3 scripts/copy_package_proposed_to_ppa.py

to:

          python3 scripts/launchpad_copy.py promote

Step 3: Run pre-commit to validate YAML and actionlint

Run: pre-commit run --all-files
Expected: All checks pass.

Step 4: Commit

git add .github/workflows/build_debian.yml
git commit -m "ci: update build_debian.yml to use consolidated launchpad_copy.py

Refs #106"

---

Task 9: Delete old scripts

Files:

• Delete: scripts/copy_to_other_distributions.py
• Delete: scripts/copy_package_proposed_to_ppa.py

Step 1: Delete the old scripts

git rm scripts/copy_to_other_distributions.py scripts/copy_package_proposed_to_ppa.py

Step 2: Verify remaining scripts directory

Run: ls scripts/
Expected: create_lp_creds.py and launchpad_copy.py

Step 3: Run all tests to ensure nothing broke

Run: python3 -m pytest tests/ -v
Expected: All tests pass.

Step 4: Run pre-commit

Run: pre-commit run --all-files
Expected: All checks pass.

Step 5: Commit

git commit -m "chore: delete old copy_to_other_distributions.py and copy_package_proposed_to_ppa.py

Both scripts are replaced by the consolidated launchpad_copy.py.

Closes #106"

Summary

Consolidates copy_to_other_distributions.py and copy_package_proposed_to_ppa.py into a single scripts/launchpad_copy.py with two argparse subcommands:

• copy-to-series — replaces copy_to_other_distributions.py. Copies packages from a source series to all other supported Ubuntu series within a PPA, with dynamic series discovery via ubuntu-distro-info.
• promote — replaces copy_package_proposed_to_ppa.py. Copies all published whitelisted packages from kolibri-proposed PPA to kolibri PPA.

Shared logic (Launchpad auth, PPA lookup, package whitelist filtering, obsolete-series error handling) is unified into a LaunchpadWrapper class with caching. The --verbose default is fixed from None to 0 to prevent TypeError in Python 3.

Also adds 32 unit tests with pytest and vcrpy configuration, a GitHub Actions CI workflow for running tests on PRs, and updates build_debian.yml to use the new script.

References

• Closes Consolidate launchpad interaction scripts #106

Reviewer guidance

• Run python scripts/launchpad_copy.py --help to verify CLI structure
• Run python -m pytest tests/ -v to execute the 32 unit tests
• Review the three commits separately: (1) new consolidated script, (2) tests and CI, (3) workflow update and old script deletion
• The LaunchpadWrapper caching pattern is preserved from the original copy_to_other_distributions.py
• The Once descriptor and cache decorator provide memoization for Launchpad API calls

Test evidence

Test execution (32 tests passing)

$ python -m pytest tests/ -v
tests/test_launchpad_copy.py::TestOnceDescriptor::test_computes_value_on_first_access PASSED
tests/test_launchpad_copy.py::TestOnceDescriptor::test_returns_same_value_on_subsequent_access PASSED
tests/test_launchpad_copy.py::TestOnceDescriptor::test_independent_per_instance PASSED
tests/test_launchpad_copy.py::TestCacheDecorator::test_caches_return_value PASSED
tests/test_launchpad_copy.py::TestCacheDecorator::test_different_args_cached_separately PASSED
tests/test_launchpad_copy.py::TestCacheDecorator::test_raises_type_error_for_unhashable_args PASSED
tests/test_launchpad_copy.py::TestBuildParser::test_verbose_defaults_to_zero PASSED
tests/test_launchpad_copy.py::TestBuildParser::test_verbose_increments_with_v_flags PASSED
tests/test_launchpad_copy.py::TestBuildParser::test_copy_to_series_subcommand_parsed PASSED
tests/test_launchpad_copy.py::TestBuildParser::test_promote_subcommand_parsed PASSED
tests/test_launchpad_copy.py::TestBuildParser::test_subcommand_required PASSED
tests/test_launchpad_copy.py::TestBuildParser::test_quiet_flag PASSED
tests/test_launchpad_copy.py::TestBuildParser::test_debug_flag PASSED
tests/test_launchpad_copy.py::TestGetSupportedSeries::test_returns_series_excluding_source PASSED
tests/test_launchpad_copy.py::TestGetSupportedSeries::test_falls_back_on_subprocess_error PASSED
tests/test_launchpad_copy.py::TestLaunchpadWrapper::test_queue_copy_accumulates_names PASSED
tests/test_launchpad_copy.py::TestLaunchpadWrapper::test_queue_starts_empty PASSED
tests/test_launchpad_copy.py::TestLaunchpadWrapper::test_perform_queued_copies_calls_sync_sources PASSED
tests/test_launchpad_copy.py::TestLaunchpadWrapper::test_perform_queued_copies_skips_empty_queues PASSED
tests/test_launchpad_copy.py::TestLaunchpadWrapper::test_get_usable_sources_filters_by_whitelist PASSED
tests/test_launchpad_copy.py::TestLaunchpadWrapper::test_get_usable_sources_skips_superseded PASSED
tests/test_launchpad_copy.py::TestConfigureLogging::test_default_sets_info_level PASSED
tests/test_launchpad_copy.py::TestConfigureLogging::test_quiet_sets_warning_level PASSED
tests/test_launchpad_copy.py::TestConfigureLogging::test_vv_sets_debug_level PASSED
tests/test_launchpad_copy.py::TestMainDispatch::test_dispatches_to_copy_to_series PASSED
tests/test_launchpad_copy.py::TestMainDispatch::test_dispatches_to_promote PASSED
tests/test_launchpad_copy.py::TestCmdCopyToSeries::test_queues_copy_for_missing_package PASSED
tests/test_launchpad_copy.py::TestCmdCopyToSeries::test_skips_copy_when_not_built_yet PASSED
tests/test_launchpad_copy.py::TestCmdCopyToSeries::test_returns_zero PASSED
tests/test_launchpad_copy.py::TestCmdPromote::test_copies_whitelisted_published_package PASSED
tests/test_launchpad_copy.py::TestCmdPromote::test_skips_non_whitelisted_package PASSED
tests/test_launchpad_copy.py::TestCmdPromote::test_handles_obsolete_series_gracefully PASSED
============================== 32 passed in 0.16s ==============================

CLI help verification

$ python scripts/launchpad_copy.py --help
usage: launchpad_copy.py [-h] [-v] [-q] [--debug] {copy-to-series,promote} ...

Launchpad PPA copy tool for kolibri-server packages.

positional arguments:
  {copy-to-series,promote}
    copy-to-series      Copy packages from source series to all other supported series within a PPA.
    promote             Promote published packages from kolibri-proposed to kolibri PPA.

options:
  -h, --help            show this help message and exit
  -v, --verbose         Increase verbosity (use -vv for debug).
  -q, --quiet           Suppress info output.
  --debug               Enable HTTP debug output.

Pre-commit linting

trim trailing whitespace.................................................Passed
check yaml...............................................................Passed
check for added large files..............................................Passed
fix end of files.........................................................Passed
yamlfmt..................................................................Passed
Lint GitHub Actions workflow files.......................................Passed

---

@rtibblesbot's comments are generated by an LLM, and should be evaluated accordingly

How was this generated?

• Ran pre-flight CI checks (lint, format, tests) and verified all pass
• Rebased onto the target branch and resolved any conflicts
• Reorganized commit history into clean, logical commits
• Audited the diff to ensure only issue-relevant files are changed
• Built PR body from the repository's PR template with evidence blocks

Closes #106

[rtibblesbot]

CI checks failed on this PR:

• Run installation of kolibri-server in docker image (debian:bullseye): cancelled (details)
• Run installation of kolibri-server in docker image (debian:bookworm): cancelled (details)
• Run installation of kolibri-server in docker image (ubuntu:20.04): cancelled (details)
• Run installation of kolibri-server in docker image (ubuntu:22.04): failure (details)

I'll investigate and push a fix.

[rtibblesbot]

CI fix: Removed COPY scripts/copy_to_other_distributions.py from test/Dockerfile — the script was deleted in this PR but the Dockerfile still referenced it, causing the Docker build to fail.

[rtibbles]

This is a great start, but I think we can do some more cleanup.

Also as we are updating Python code here, let's add ruff to the pre-commit config, and add the rule to prevent inline imports. We can copy additional configuration from https://github.com/learningequality/le-skills/blob/main/pyproject.toml

[rtibbles]

I don't think there's a need to guard this, we should just raise an import error and make sure we are documenting dependencies properly (in pyproject.toml if needed) and ensuring we are installing the required dependencies in actions files and documenting this too.

[rtibblesbot]

Fixed — removed import guard, imports now fail loudly. Added distro-info to CI apt deps.

[rtibbles]

I realize this is copied from the original, but I think we should get rid of it and rely on querying for these values based on Ubuntu supported versions plus ELTS versions.

[rtibblesbot]

Fixed — removed FALLBACK_SERIESES. Now relies on ubuntu-distro-info --supported-esm for dynamic discovery including ESM/ELTS.

[rtibbles]

Can you not just use stdlib memoize instead of this?

[rtibblesbot]

Fixed — replaced with functools.cached_property.

[rtibbles]

Again, let's use stdlib?

[rtibblesbot]

Fixed — replaced with functools.cache.

[rtibbles]

Would this include ELTS as well? We want to support as wide a range as we can, and are only really limited by the bundled Python version (must be 3.6 or higher and 3.13 or lower).

[rtibblesbot]

Now uses --supported-esm flag which should include ESM/ELTS versions.

[rtibbles]

Also, we shouldn't mock this in tests, we should actually test the real call here to validate this code.

[rtibblesbot]

Fixed — tests now call real ubuntu-distro-info, skipped on systems where it's not available. Added distro-info to CI apt install.

[rtibbles]

What's the point of these inline imports?

[rtibblesbot]

Fixed — moved httplib2 import to top level.

[rtibbles]

Let's put main last? Organise the code so callers come after their callees.

[rtibblesbot]

Fixed — main() is now last. Code ordered: utilities → wrapper class → CLI → subcommand functions → main.

[rtibbles]

Note, I don't think we need this. We should define the test dependencies, and just assume it exists.

[rtibblesbot]

Fixed — removed try/except guard, imports vcr directly.

[rtibblesbot]

Thanks for the thorough review! Addressed all feedback:

• Added ruff to pre-commit config with config from le-skills pyproject.toml
• Removed all import guards — imports now fail loudly, deps documented in CI workflow
• Replaced custom once/cache with functools.cached_property/functools.cache
• Removed FALLBACK_SERIESES, now uses ubuntu-distro-info --supported-esm for ESM/ELTS coverage
• Fixed bare exception handling (removed try/except fallback entirely)
• Moved inline httplib2 imports to top level
• Moved subcommand logic into LaunchpadWrapper.copy_to_series() and .promote() methods
• Reorganized code: callers after callees, main() last
• Tests now call real ubuntu-distro-info (skipped if not available)

[rtibblesbot]

CI checks failed on this PR:

• All file linting: failure (details)
• test (3.12): cancelled (details)
• test (3.10): failure (details)

I'll investigate and push a fix.

[rtibblesbot]

Added PLC0415 ruff rule to prevent inline imports as requested. Fixed CI lint failures in kolibri_server_setup.py (unsorted imports, unused get_content_dir_path import, formatting) and fixed the broken test assertion in test_queues_copy_for_missing_package.

[rtibbles]

Just a couple more thoughts but this is looking good.

[rtibblesbot]

Addressed the two remaining review comments. Ruff with PLC0415 was already added in the previous round.

[rtibblesbot]

CI checks failed on this PR:

• Run installation of kolibri-server in docker image (debian:bullseye): cancelled (details)
• Run installation of kolibri-server in docker image (debian:bookworm): cancelled (details)
• Run installation of kolibri-server in docker image (ubuntu:20.04): cancelled (details)
• Run installation of kolibri-server in docker image (ubuntu:22.04): failure (details)

I'll investigate and push a fix.

[rtibbles]

Good work, this is good to merge.