diff --git a/.gitignore b/.gitignore index b378a762..deb78538 100644 --- a/.gitignore +++ b/.gitignore @@ -9,6 +9,10 @@ generated .iml .idea/ *.orig +*.war +*.war.asc +*.sha256 venv/ pkg.jenkins.io/ +jenkins.io.key diff --git a/Makefile b/Makefile index 89f00148..f2cd2760 100644 --- a/Makefile +++ b/Makefile @@ -56,11 +56,3 @@ ${CLI}: @mkdir ${TARGET} || true wget -O $@.tmp ${JENKINS_URL}jnlpJars/jenkins-cli.jar mv $@.tmp $@ - - - -test.local.setup: - # start a test Apache server that acts as package server - # we'll refer to this as 'test.pkg.jenkins.io' - @mkdir -p ${TESTDIR} || true - docker run --rm -t -i -p 9200:80 -v ${TESTDIR}:/var/www/html fedora/apache diff --git a/README.md b/README.md index a7d2ccfe..f68fe92e 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,24 @@ The following platforms are currently supported: * RedHat/CentOS/openSUSE RPM: `rpm/` * Debian/Ubuntu DEB: `deb/` -# Pre-requisites +## Pre-requisites + +### Easy Method + +If you are able to run Docker containers with `docker compose` (usually Docker Desktop is enough), +then you can use the same environment as the Jenkins infrastructure production: + +```bash +# Start the setup (2 containers: one for packaging and another for a webserver) +docker compose up -d + +# Spawn an interactive terminal in the packaging environment (with all dependencies) +docker compose exec packaging bash +# You can run all the `make` command and even `prep.sh` script - see sections below +``` + +### Container-less Method (aka "The Hard Way") + Running the main package script requires a Linux environment (currently Ubuntu, see [JENKINS-27744](https://issues.jenkins-ci.org/browse/JENKINS-27744).) Run `make setup` to install (most of the) necessary tools. Alternatively you can manually install the following onto a base install of Ubuntu: * make @@ -24,21 +41,11 @@ Run `make setup` to install (most of the) necessary tools. Alternatively you ca You also need a Jenkins instance with [dist-fork plugin](https://wiki.jenkins-ci.org/display/JENKINS/DistFork+Plugin) installed. URL of this Jenkins can be fed into `make` via the `JENKINS_URL` variable. -This Jenkins needs to have a Windows build agent that has [WiX Toolset](http://wixtoolset.org/) (currently 3.5), msbuild, [cygwin](https://www.cygwin.com/) and .net 2.0. This build agent is used to build MSI packages, which -can be only built on Windows. - -You'll also need a `jenkins.war` file that you are packaging, which comes from the release process. -The location of this file is set via the `WAR` variable. - -Remark: +This Jenkins needs to have a Windows build agent that has [WiX Toolset](http://wixtoolset.org/) (currently 3.5), msbuild, [cygwin](https://www.cygwin.com/) and .net 2.0. +This build agent is used to build MSI packages, which can be only built on Windows. -A docker image is available to run following script +## Generating packages -[![logo](https://img.shields.io/docker/pulls/jenkinsciinfra/packaging?label=jenkinsciinfra%2Fpackaging&logo=docker&logoColor=white)](https://hub.docker.com/r/jenkinsciinfra/packaging) - -Run `docker-compose run --rm packaging bash` to get a shell in the official Docker image for this repository. - -# Generating packages Run `./prep.sh` to perform the preparatory actions of downloading the WAR and importing the GPG key. Run `make package` to build all the native packages. At minimum, you have to specify the `WAR` variable that points to the war file to be packaged and the `BRAND` variable that points to the branding file for licensing, artifact names, and package descriptions. @@ -53,7 +60,7 @@ make package BRAND=./branding/jenkins.mk BUILDENV=./env/test.mk CREDENTIAL=./cre Packages will be placed into `target/` directory. See the definition of the `package` goal for how to build individual packages selectively. -# Running functional tests +## Running functional tests The functional tests require Python 3 and Docker. Having built the packages as described above, run the functional tests with: @@ -66,26 +73,27 @@ molecule test deactivate ``` -# Publishing packages +## Publishing packages + This repository contains scripts for copying packages over to a remote web server to publish them. Run `make publish` to publish all native packages. -See the definition of the `publish` goal for individual package publishment. +See the definition of the `publish` goal for individual package publication. ## Running local tests + These tests install packages from a web server where they are published. So if you want to run tests prior to publishing them, you need to create a temporary web server that you can mess up. The default branding & environment (`branding/test.mk` and `env/test.mk`) are designed to support this scenario. To make local testing work, you also need to have `/etc/hosts` entry that maps -`test.pkg.jenkins.io` hostname to `127.0.0.1`, and your computer has to be running ssh that -lets you login as you. +`test.pkg.jenkins.io` hostname to `127.0.0.1`. -Once you verified the above prerequisites, open another terminal and run `make test.local.setup` -This will run a docker container that acts as your throw-away package web server. When done, Ctrl+C -to kill it. +Once you verified the above prerequisites, open another terminal and run `docker compose up -d pkgserver` +This will run a docker container that acts as your throw-away package web server. + +## Branding -# Branding `branding/` directory contains `*.mk` files that control the branding of the generated packages. It also include text files which are used for large, branded text blocks (license and descriptions). Specify the branding file via the `BRAND` variable. @@ -95,19 +103,18 @@ See [branding readme](branding/README.md) for more details. In the rest of the p these branding parameters are referenced via `@@NAME@@` and get substituted by `bin/branding.py`. To escape a string normally like @@VALUE@@, add an additional two @@ symbols as a prefix: @@@@VALUE@@. -# Environment +## Environment + `env/` directory contains `*.mk` files that control the environment into which you publish packages. Specify the environment file via the `BUILDENV` variable. You can create your own environment definition to customize the package generation process. See [environment readme](env/README.md) for more details. -# Credentials +## Credentials + `credentials/` directory contains `test.mk` file that controls the locations of code-signing keys, their passwords, and certificates. Specify the credentials file via the `CREDENTIAL` variable. For production use, you need to create your own credentials file. See [credentials readme](credentials/README.md) for more details. - -# TODO (mostly note to myself) -* Split resource templates to enable customization diff --git a/bin/indexGenerator.py b/bin/indexGenerator.py index c58ba985..181db123 100755 --- a/bin/indexGenerator.py +++ b/bin/indexGenerator.py @@ -60,6 +60,7 @@ def __init__(self, argv): self.product_name = os.getenv("PRODUCTNAME", "Jenkins") self.distribution = os.getenv("OS_FAMILY", "debian") self.gpg_pub_key_info_file = os.getenv("GPGPUBKEYINFO", ".") + self.gpg_public_key_filename = os.getenv("GPG_PUBLIC_KEY_FILENAME", "jenkins.io.key") self.target_directory = "./target/" + self.distribution try: @@ -107,6 +108,7 @@ def show_information(self): print("Root header generated: " + self.root_header) print("Root footer generated: " + self.root_footer) print("GPG Key Info File: " + self.gpg_pub_key_info_file) + print("GPG Public Key Filename: " + self.gpg_public_key_filename) def generate_root_header(self): @@ -169,6 +171,7 @@ def generate_repository_header(self): "releaseline": self.releaseline, "web_url": self.web_url, "pub_key_info": self.fetch_pubkeyinfo(), + "gpg_public_key_filename": self.gpg_public_key_filename, } env = jinja2.Environment( @@ -192,6 +195,7 @@ def generate_repository_index(self): "releaseline": self.releaseline, "web_url": self.web_url, "pub_key_info": self.fetch_pubkeyinfo(), + "gpg_public_key_filename": self.gpg_public_key_filename, } env = jinja2.Environment( diff --git a/credentials/ssh/id_rsa b/credentials/ssh/id_rsa deleted file mode 100644 index 268c1b9e..00000000 --- a/credentials/ssh/id_rsa +++ /dev/null @@ -1,27 +0,0 @@ ------BEGIN RSA PRIVATE KEY----- -MIIEowIBAAKCAQEAw6bQJN8ceL5T7FaNs2QdSEnGgLKy5zvpuORDJLd0Q/wTm5DV -MVxqJzT8hCtL7a4D1NBPaBm6pAvlxCnKMhdZwSi+e2zuzRokOuivlAHyiV7le9Ln -1K/pKkDchAIz8P3YWLt1YZJERDUI8nXW9MC8ZOEt/IudmLTsZQcSr8lXOFIwNZe4 -b/NSr/9sxvSnYQqw5q6j1ZxdjW/KtXPQapU+PObZ/31jzTH9W966LpoF8XYo0xRA -hJvy3f+NxbSVPhjH422wHCeeyfI/+B/xYlVNC3yYR4ONvXbAXsewB0aRhnOfh+sS -kzotOCDMBZHOjKrg27MLs0dfrmjxGffF2Vq+JQIDAQABAoIBAELefZ9Mfg+qhUZu -YqngWr29MVIFQW4UpRIjOeuPo/YkbpMp0iO3wTQ7QN7vaVkHs5mFxM4AlTDCPDpq -SggKwQtqoIfQuGFzQNS9eFzuuXVH8Mj8UW343Ykqd/PKSPRh3hKdp0W81wY01iUA -L4KhaQJVkAETur5Zf74bx8A64UuG+qWbDUGlRGCRk/pl9xGB1z0FQ7Api/6gQd2A -Tnu6ASmQfoKeDsDDOBVy8sHv7HlkU9msowD9TdOj7Gxx9DoPryX6GlAhgi//+jyA -qUCf03kdey5aCVKFVUIOkxpDxYRI0etW0ef0rww+DJDpL7pT0kWMf4sqxgmxPTly -TaOnl+ECgYEA4GpfL7GTPTFvhatxi78TlaATTzjmzPYLrxOF8EHQM3Tz/nDbDknX -x5YtQdq0cv2TNdR2uZ0AyuzZ06j6axyBCJWKgtErN+SO01+Qxa9rcv+Vw7NtdTGs -GUrMa7CU/X4t8jt6UiObIgGRNbvu93ANeEzjIOFn9S9QKQ0vrtfW/W0CgYEA3zAV -/z5rt/AThnajCRPv5c/o9c2TykKy3DFFUdgNTEwnHE04D+xoqH1eoTMNoNXdVbI0 -1C5WTzCpYLuKmbl/aZEwvidi6ssTpYHcviAAz8iqN/TL9Ys5XmJ3iJViNK3IxxHB -TRLMiBC057tS8ZZLRa756weEZ2TUYRydxFntaJkCgYAV/nbbvsSWb7zlVdsn/g8W -T/z0e7grCEY232v2Ew0rrd+n5Tmi2dvbBL3kwWGED5QY53zHTjrgqHvkwZ/hVYbT -54wOrB9XOABDeQ9AQKQAPkpYRsKIhNjAFdOZDlJb0b0BC5E+cZznpU2s/YE7IPFB -BBASjeTZY8ywaUluEltQtQKBgCi2idy000uLdNRbgeQfCez/Hzzvkl0cC6qVJlMG -uW5Imf3UrDxjYLgTnpaDTKIhQS3nwzFNfpsVgmBN9buTFgX44U5euvGft+bCKLVZ -+yvsK/jnI+mXyxBHoAx/S5nWdcCyoXNg0YSkn4uCJWBCjVqZz6crCOEfiIpqgPEX -gnJJAoGBAKvDah87FijEJRBiaroef5buG1jr9pNCBoXIGbvZ1sFwkGWQGgrH6Y3s -EBD083+BBCcIMvzy2leB692axxGhtdyCxfPRN7KiZgT/YC6cCDL1yzhSHtZ8kamb -N8Qqs+wVE4YIdELB+VgKTho1v4gAyzZNuMJMhne6qH+oxNfGUgER ------END RSA PRIVATE KEY----- diff --git a/credentials/ssh/id_rsa.pub b/credentials/ssh/id_rsa.pub deleted file mode 100644 index 3d3e77dd..00000000 --- a/credentials/ssh/id_rsa.pub +++ /dev/null @@ -1 +0,0 @@ -ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDptAk3xx4vlPsVo2zZB1IScaAsrLnO+m45EMkt3RD/BObkNUxXGonNPyEK0vtrgPU0E9oGbqkC+XEKcoyF1nBKL57bO7NGiQ66K+UAfKJXuV70ufUr+kqQNyEAjPw/dhYu3VhkkRENQjyddb0wLxk4S38i52YtOxlBxKvyVc4UjA1l7hv81Kv/2zG9KdhCrDmrqPVnF2Nb8q1c9BqlT485tn/fWPNMf1b3roumgXxdijTFECEm/Ld/43FtJU+GMfjbbAcJ57J8j/4H/FiVU0LfJhHg429dsBex7AHRpGGc5+H6xKTOi04IMwFkc6MquDbswuzR1+uaPEZ98XZWr4l olblak@winterfell diff --git a/credentials/ssh/known_hosts b/credentials/ssh/known_hosts deleted file mode 100644 index a352863a..00000000 --- a/credentials/ssh/known_hosts +++ /dev/null @@ -1,2 +0,0 @@ -|1|3Ao1unSiaoLJcBH+jj4LxlUJvU8=|9VCo6soeOkBokfDfbckMBvnE/6k= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFnJjx5araEbR3BvluFk5ONHqZSVZW1osdn4NuC/UBFPxwcEkkECK0EHR+WTxfTLGybJCTh3H5hTDady7W0EyIs= -|1|Orovxffw11DXksUZda8iwv3XcME=|OIrRb9oqBvY2esPMc+I0K70HLSs= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFnJjx5araEbR3BvluFk5ONHqZSVZW1osdn4NuC/UBFPxwcEkkECK0EHR+WTxfTLGybJCTh3H5hTDady7W0EyIs= diff --git a/deb/publish/contents/binary/.htaccess b/deb/publish/contents/binary/.htaccess deleted file mode 100644 index 5a928f6d..00000000 --- a/deb/publish/contents/binary/.htaccess +++ /dev/null @@ -1 +0,0 @@ -Options -Indexes diff --git a/deb/publish/publish.sh b/deb/publish/publish.sh index 34d137c3..7e95a6a3 100755 --- a/deb/publish/publish.sh +++ b/deb/publish/publish.sh @@ -8,6 +8,7 @@ set -euxo pipefail : "${DEBDIR:? Require where to put binary files}" : "${DEB_WEBDIR:? Require where to put repository index and other web contents}" : "${DEB_URL:? Require Debian repository Url}" +: "${GPG_PUBLIC_KEY_FILENAME:="${ORGANIZATION}.key"}" # $$ Contains current pid D="$AGENT_WORKDIR/$$" @@ -20,14 +21,14 @@ function clean() { # Generate and publish site content function generateSite() { - cp -R "$bin/contents/." "$D/contents" - - gpg --export -a --output "$D/contents/${ORGANIZATION}.key" "${GPG_KEYNAME}" - gpg --import-options show-only --import "$D/contents/${ORGANIZATION}.key" >"$D/contents/${ORGANIZATION}.key.info" + local gpg_publickey_file="$D/contents/${GPG_PUBLIC_KEY_FILENAME}" + local gpg_publickey_info_file="$D/contents/${GPG_PUBLIC_KEY_FILENAME}.info" + gpg --export -a --output "${gpg_publickey_file}" "${GPG_KEYNAME}" + gpg --import-options show-only --import "${gpg_publickey_file}" > "${gpg_publickey_info_file}" "$BASE/bin/indexGenerator.py" \ --distribution debian \ - --gpg-key-info-file "${D}/contents/${ORGANIZATION}.key.info" \ + --gpg-key-info-file "${gpg_publickey_info_file}" \ --targetDir "$D/html" "$BASE/bin/branding.py" "$D" @@ -59,7 +60,7 @@ function generateSite() { } function init() { - mkdir -p "$D/binary" "$D/contents" "$D/html" \ + mkdir -p "$D/binary" "$D/contents" "$D/html" "$D/contents/binary" \ "$DEBDIR" `# where to put binary files` \ "$DEB_WEBDIR" `# where to put repository index and other web contents` } @@ -87,14 +88,10 @@ function uploadPackageSite() { } function uploadHtmlSite() { - # Html file need to be located in the binary directory rsync --archive \ --verbose \ --progress \ - --include "HEADER.html" \ - --include "FOOTER.html" \ - --exclude "*" \ - "$D/html/" "$DEBDIR/" + "$D/html/" "$DEB_WEBDIR/" } function show() { @@ -103,6 +100,7 @@ function show() { echo "DEBDIR: $DEBDIR" echo "DEB_WEBDIR: $DEB_WEBDIR" echo "GPG_KEYNAME: $GPG_KEYNAME" + echo "GPG_PUBLIC_KEY_FILENAME: $GPG_PUBLIC_KEY_FILENAME" echo "---" } diff --git a/docker-compose.yaml b/docker-compose.yaml index 8db03961..41a6d9ec 100644 --- a/docker-compose.yaml +++ b/docker-compose.yaml @@ -1,12 +1,11 @@ -# docker exec -i -t packaging_packaging_1 gpg --import --batch credentials/sandbox.gpg -version: '3' -volumes: - sshd: - pkgserver: +# docker compose exec packaging gpg --import --batch credentials/sandbox.gpg services: packaging: image: jenkinsciinfra/packaging:latest - command: "sleep 99d" + platform: linux/amd64 + entrypoint: + - "sleep" + - "99d" environment: - "BUILDENV=/srv/releases/jenkins/env/test.mk" - "BRANDING_DIR=/srv/releases/jenkins/branding" @@ -19,25 +18,13 @@ services: - "MSI=/srv/releases/jenkins/jenkins.msi" - "RELEASELINE=-experimental" volumes: - - ".:/srv/releases/jenkins:z" - - "./credentials/ssh:/home/jenkins/.ssh:z" - working_dir: "/srv/releases/jenkins" - - remote: - image: jenkinsciinfra/packaging:latest - user: root # In order to start, sshd needs to be root - command: "/usr/sbin/sshd -D" - ports: - - "2222:22" - volumes: - - "./credentials/ssh/id_rsa.pub:/home/jenkins/.ssh/authorized_keys:ro" - - sshd:/run/sshd - - "pkgserver:/srv/releases/jenkins" + - .:/srv/releases/jenkins:z + working_dir: /srv/releases/jenkins pkgserver: - image: httpd + image: nginx ports: - "80:80" volumes: - - "./pkgConfig/httpd.conf:/usr/local/apache2/conf/httpd.conf" - - "./target:/usr/local/apache2/htdocs" + - ./target:/usr/share/nginx/html:ro + - ./pkgserver/nginx-default-vhost.conf:/etc/nginx/conf.d/default.conf:ro diff --git a/env/test.mk b/env/test.mk index 050b9417..565e86c8 100644 --- a/env/test.mk +++ b/env/test.mk @@ -3,7 +3,7 @@ # # where to put binary files -export TESTDIR=$(realpath .)/pkg.jenkins.io +export TESTDIR=$(realpath .)/target export WARDIR=${TESTDIR}/war${RELEASELINE} export MSIDIR=${TESTDIR}/windows${RELEASELINE} export DEBDIR=${TESTDIR}/debian${RELEASELINE}/binary diff --git a/msi/publish/publish.sh b/msi/publish/publish.sh index 1ab6eea6..ba6e0df5 100755 --- a/msi/publish/publish.sh +++ b/msi/publish/publish.sh @@ -30,23 +30,16 @@ function uploadPackage() { sha256sum "${MSI}" >"${MSI_SHASUM}" cat "${MSI_SHASUM}" - # Update the symlink to point to most recent Windows build - # - # Remove anything in current directory named 'latest' - # This is a safety measure just in case something was left there previously - rm -rf latest - - # Create a local symlink pointing to the MSI file in the VERSION directory. - # Don't need VERSION directory or MSI locally, just the unresolved symlink. - # The jenkins.io page downloads http://mirrors.jenkins-ci.org/windows/latest - # and assumes it points to the most recent MSI file. - ln -s "${VERSION}/$(basename "$MSI")" latest - - # Local rsync --archive \ --verbose \ --progress \ - "${MSI}" "${MSI_SHASUM}" latest "${MSIDIR}/${VERSION}/" + "${MSI}" "${MSI_SHASUM}" "${MSIDIR}/${VERSION}/" + + # Update the symlink to point to most recent MSI directory + pushd "${MSIDIR}" + rm -rf latest # This is a safety measure just in case something was left there previously + ln -s "${VERSION}" latest + popd } # The site need to be located in the binary directory diff --git a/pkgConfig/httpd.conf b/pkgConfig/httpd.conf deleted file mode 100644 index 57935b61..00000000 --- a/pkgConfig/httpd.conf +++ /dev/null @@ -1,487 +0,0 @@ -ServerRoot "/usr/local/apache2" - -Listen 80 - -# -# Dynamic Shared Object (DSO) Support -# -# To be able to use the functionality of a module which was built as a DSO you -# have to place corresponding `LoadModule' lines at this location so the -# directives contained in it are actually available _before_ they are used. -# Statically compiled modules (those listed by `httpd -l') do not need -# to be loaded here. -# -# Example: -# LoadModule foo_module modules/mod_foo.so -# -LoadModule mpm_event_module modules/mod_mpm_event.so -#LoadModule mpm_prefork_module modules/mod_mpm_prefork.so -#LoadModule mpm_worker_module modules/mod_mpm_worker.so -LoadModule authn_file_module modules/mod_authn_file.so -#LoadModule authn_dbm_module modules/mod_authn_dbm.so -#LoadModule authn_anon_module modules/mod_authn_anon.so -#LoadModule authn_dbd_module modules/mod_authn_dbd.so -#LoadModule authn_socache_module modules/mod_authn_socache.so -LoadModule authn_core_module modules/mod_authn_core.so -LoadModule authz_host_module modules/mod_authz_host.so -LoadModule authz_groupfile_module modules/mod_authz_groupfile.so -LoadModule authz_user_module modules/mod_authz_user.so -#LoadModule authz_dbm_module modules/mod_authz_dbm.so -#LoadModule authz_owner_module modules/mod_authz_owner.so -#LoadModule authz_dbd_module modules/mod_authz_dbd.so -LoadModule authz_core_module modules/mod_authz_core.so -#LoadModule authnz_ldap_module modules/mod_authnz_ldap.so -#LoadModule authnz_fcgi_module modules/mod_authnz_fcgi.so -LoadModule access_compat_module modules/mod_access_compat.so -LoadModule auth_basic_module modules/mod_auth_basic.so -#LoadModule auth_form_module modules/mod_auth_form.so -#LoadModule auth_digest_module modules/mod_auth_digest.so -#LoadModule allowmethods_module modules/mod_allowmethods.so -#LoadModule isapi_module modules/mod_isapi.so -#LoadModule file_cache_module modules/mod_file_cache.so -#LoadModule cache_module modules/mod_cache.so -#LoadModule cache_disk_module modules/mod_cache_disk.so -#LoadModule cache_socache_module modules/mod_cache_socache.so -#LoadModule socache_shmcb_module modules/mod_socache_shmcb.so -#LoadModule socache_dbm_module modules/mod_socache_dbm.so -#LoadModule socache_memcache_module modules/mod_socache_memcache.so -#LoadModule socache_redis_module modules/mod_socache_redis.so -#LoadModule watchdog_module modules/mod_watchdog.so -#LoadModule macro_module modules/mod_macro.so -#LoadModule dbd_module modules/mod_dbd.so -#LoadModule bucketeer_module modules/mod_bucketeer.so -#LoadModule dumpio_module modules/mod_dumpio.so -#LoadModule echo_module modules/mod_echo.so -#LoadModule example_hooks_module modules/mod_example_hooks.so -#LoadModule case_filter_module modules/mod_case_filter.so -#LoadModule case_filter_in_module modules/mod_case_filter_in.so -#LoadModule example_ipc_module modules/mod_example_ipc.so -#LoadModule buffer_module modules/mod_buffer.so -#LoadModule data_module modules/mod_data.so -#LoadModule ratelimit_module modules/mod_ratelimit.so -LoadModule reqtimeout_module modules/mod_reqtimeout.so -#LoadModule ext_filter_module modules/mod_ext_filter.so -#LoadModule request_module modules/mod_request.so -#LoadModule include_module modules/mod_include.so -LoadModule filter_module modules/mod_filter.so -#LoadModule reflector_module modules/mod_reflector.so -#LoadModule substitute_module modules/mod_substitute.so -#LoadModule sed_module modules/mod_sed.so -#LoadModule charset_lite_module modules/mod_charset_lite.so -#LoadModule deflate_module modules/mod_deflate.so -#LoadModule xml2enc_module modules/mod_xml2enc.so -#LoadModule proxy_html_module modules/mod_proxy_html.so -#LoadModule brotli_module modules/mod_brotli.so -LoadModule mime_module modules/mod_mime.so -#LoadModule ldap_module modules/mod_ldap.so -LoadModule log_config_module modules/mod_log_config.so -#LoadModule log_debug_module modules/mod_log_debug.so -#LoadModule log_forensic_module modules/mod_log_forensic.so -#LoadModule logio_module modules/mod_logio.so -#LoadModule lua_module modules/mod_lua.so -LoadModule env_module modules/mod_env.so -#LoadModule mime_magic_module modules/mod_mime_magic.so -#LoadModule cern_meta_module modules/mod_cern_meta.so -#LoadModule expires_module modules/mod_expires.so -LoadModule headers_module modules/mod_headers.so -#LoadModule ident_module modules/mod_ident.so -#LoadModule usertrack_module modules/mod_usertrack.so -#LoadModule unique_id_module modules/mod_unique_id.so -LoadModule setenvif_module modules/mod_setenvif.so -LoadModule version_module modules/mod_version.so -#LoadModule remoteip_module modules/mod_remoteip.so -#LoadModule proxy_module modules/mod_proxy.so -#LoadModule proxy_connect_module modules/mod_proxy_connect.so -#LoadModule proxy_ftp_module modules/mod_proxy_ftp.so -#LoadModule proxy_http_module modules/mod_proxy_http.so -#LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so -#LoadModule proxy_scgi_module modules/mod_proxy_scgi.so -#LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so -#LoadModule proxy_fdpass_module modules/mod_proxy_fdpass.so -#LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so -#LoadModule proxy_ajp_module modules/mod_proxy_ajp.so -#LoadModule proxy_balancer_module modules/mod_proxy_balancer.so -#LoadModule proxy_express_module modules/mod_proxy_express.so -#LoadModule proxy_hcheck_module modules/mod_proxy_hcheck.so -#LoadModule session_module modules/mod_session.so -#LoadModule session_cookie_module modules/mod_session_cookie.so -#LoadModule session_crypto_module modules/mod_session_crypto.so -#LoadModule session_dbd_module modules/mod_session_dbd.so -#LoadModule slotmem_shm_module modules/mod_slotmem_shm.so -#LoadModule slotmem_plain_module modules/mod_slotmem_plain.so -#LoadModule ssl_module modules/mod_ssl.so -#LoadModule optional_hook_export_module modules/mod_optional_hook_export.so -#LoadModule optional_hook_import_module modules/mod_optional_hook_import.so -#LoadModule optional_fn_import_module modules/mod_optional_fn_import.so -#LoadModule optional_fn_export_module modules/mod_optional_fn_export.so -#LoadModule dialup_module modules/mod_dialup.so -#LoadModule http2_module modules/mod_http2.so -#LoadModule proxy_http2_module modules/mod_proxy_http2.so -#LoadModule md_module modules/mod_md.so -#LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so -#LoadModule lbmethod_bytraffic_module modules/mod_lbmethod_bytraffic.so -#LoadModule lbmethod_bybusyness_module modules/mod_lbmethod_bybusyness.so -#LoadModule lbmethod_heartbeat_module modules/mod_lbmethod_heartbeat.so -LoadModule unixd_module modules/mod_unixd.so -#LoadModule heartbeat_module modules/mod_heartbeat.so -#LoadModule heartmonitor_module modules/mod_heartmonitor.so -#LoadModule dav_module modules/mod_dav.so -LoadModule status_module modules/mod_status.so -LoadModule autoindex_module modules/mod_autoindex.so -#LoadModule asis_module modules/mod_asis.so -#LoadModule info_module modules/mod_info.so -#LoadModule suexec_module modules/mod_suexec.so - - #LoadModule cgid_module modules/mod_cgid.so - - - #LoadModule cgi_module modules/mod_cgi.so - -#LoadModule dav_fs_module modules/mod_dav_fs.so -#LoadModule dav_lock_module modules/mod_dav_lock.so -#LoadModule vhost_alias_module modules/mod_vhost_alias.so -#LoadModule negotiation_module modules/mod_negotiation.so -LoadModule dir_module modules/mod_dir.so -#LoadModule imagemap_module modules/mod_imagemap.so -#LoadModule actions_module modules/mod_actions.so -#LoadModule speling_module modules/mod_speling.so -#LoadModule userdir_module modules/mod_userdir.so -LoadModule alias_module modules/mod_alias.so -#LoadModule rewrite_module modules/mod_rewrite.so - - -# -# If you wish httpd to run as a different user or group, you must run -# httpd as root initially and it will switch. -# -# User/Group: The name (or #number) of the user/group to run httpd as. -# It is usually good practice to create a dedicated user and group for -# running httpd, as with most system services. -# -User daemon -Group daemon - - - -# 'Main' server configuration -# -# The directives in this section set up the values used by the 'main' -# server, which responds to any requests that aren't handled by a -# definition. These values also provide defaults for -# any containers you may define later in the file. -# -# All of these directives may appear inside containers, -# in which case these default settings will be overridden for the -# virtual host being defined. -# - -# -# ServerAdmin: Your address, where problems with the server should be -# e-mailed. This address appears on some server-generated pages, such -# as error documents. e.g. admin@your-domain.com -# -ServerAdmin me@olblak.com - -# -# ServerName gives the name and port that the server uses to identify itself. -# This can often be determined automatically, but we recommend you specify -# it explicitly to prevent problems during startup. -# -# If your host doesn't have a registered DNS name, enter its IP address here. -# -#ServerName www.example.com:80 - -# -# Deny access to the entirety of your server's filesystem. You must -# explicitly permit access to web content directories in other -# blocks below. -# - - AllowOverride none - Require all denied - - -# -# Note that from this point forward you must specifically allow -# particular features to be enabled - so if something's not working as -# you might expect, make sure that you have specifically enabled it -# below. -# - -# -# DocumentRoot: The directory out of which you will serve your -# documents. By default, all requests are taken from this directory, but -# symbolic links and aliases may be used to point to other locations. -# -DocumentRoot "/usr/local/apache2/htdocs" - - Options Indexes FollowSymLinks - AllowOverride FileInfo - Require all granted - - IndexOptions FancyIndexing VersionSort TrackModified FoldersFirst SuppressHTMLPreamble NameWidth=* SuppressRules SuppressIcon SuppressDescription - - HeaderName HEADER.html - ReadmeName FOOTER.html - - IndexIgnore HEADER* FOOTER* index.html - - - -# -# DirectoryIndex: sets the file that Apache will serve if a directory -# is requested. -# - - DirectoryIndex index.html - - -# -# The following lines prevent .htaccess and .htpasswd files from being -# viewed by Web clients. -# - - Require all denied - - -# -# ErrorLog: The location of the error log file. -# If you do not specify an ErrorLog directive within a -# container, error messages relating to that virtual host will be -# logged here. If you *do* define an error logfile for a -# container, that host's errors will be logged there and not here. -# -ErrorLog /proc/self/fd/2 - -# -# LogLevel: Control the number of messages logged to the error_log. -# Possible values include: debug, info, notice, warn, error, crit, -# alert, emerg. -# -LogLevel warn - - - # - # The following directives define some format nicknames for use with - # a CustomLog directive (see below). - # - LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined - LogFormat "%h %l %u %t \"%r\" %>s %b" common - - - # You need to enable mod_logio.c to use %I and %O - LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio - - - # - # The location and format of the access logfile (Common Logfile Format). - # If you do not define any access logfiles within a - # container, they will be logged here. Contrariwise, if you *do* - # define per- access logfiles, transactions will be - # logged therein and *not* in this file. - # - CustomLog /proc/self/fd/1 common - - # - # If you prefer a logfile with access, agent, and referer information - # (Combined Logfile Format) you can use the following directive. - # - #CustomLog "logs/access_log" combined - - - - # - # Redirect: Allows you to tell clients about documents that used to - # exist in your server's namespace, but do not anymore. The client - # will make a new request for the document at its new location. - # Example: - # Redirect permanent /foo http://www.example.com/bar - - # - # Alias: Maps web paths into filesystem paths and is used to - # access content that does not live under the DocumentRoot. - # Example: - # Alias /webpath /full/filesystem/path - # - # If you include a trailing / on /webpath then the server will - # require it to be present in the URL. You will also likely - # need to provide a section to allow access to - # the filesystem path. - - # - # ScriptAlias: This controls which directories contain server scripts. - # ScriptAliases are essentially the same as Aliases, except that - # documents in the target directory are treated as applications and - # run by the server when requested rather than as documents sent to the - # client. The same rules about trailing "/" apply to ScriptAlias - # directives as to Alias. - # - ScriptAlias /cgi-bin/ "/usr/local/apache2/cgi-bin/" - - - - - # - # ScriptSock: On threaded servers, designate the path to the UNIX - # socket used to communicate with the CGI daemon of mod_cgid. - # - #Scriptsock cgisock - - -# -# "/usr/local/apache2/cgi-bin" should be changed to whatever your ScriptAliased -# CGI directory exists, if you have that configured. -# - - AllowOverride None - Options None - Require all granted - - - - # - # Avoid passing HTTP_PROXY environment to CGI's on this or any proxied - # backend servers which have lingering "httpoxy" defects. - # 'Proxy' request header is undefined by the IETF, not listed by IANA - # - RequestHeader unset Proxy early - - - - # - # TypesConfig points to the file containing the list of mappings from - # filename extension to MIME-type. - # - TypesConfig conf/mime.types - - # - # AddType allows you to add to or override the MIME configuration - # file specified in TypesConfig for specific file types. - # - #AddType application/x-gzip .tgz - # - # AddEncoding allows you to have certain browsers uncompress - # information on the fly. Note: Not all browsers support this. - # - #AddEncoding x-compress .Z - #AddEncoding x-gzip .gz .tgz - # - # If the AddEncoding directives above are commented-out, then you - # probably should define those extensions to indicate media types: - # - AddType application/x-compress .Z - AddType application/x-gzip .gz .tgz - - # - # AddHandler allows you to map certain file extensions to "handlers": - # actions unrelated to filetype. These can be either built into the server - # or added with the Action directive (see below) - # - # To use CGI scripts outside of ScriptAliased directories: - # (You will also need to add "ExecCGI" to the "Options" directive.) - # - #AddHandler cgi-script .cgi - - # For type maps (negotiated resources): - #AddHandler type-map var - - # - # Filters allow you to process content before it is sent to the client. - # - # To parse .shtml files for server-side includes (SSI): - # (You will also need to add "Includes" to the "Options" directive.) - # - #AddType text/html .shtml - #AddOutputFilter INCLUDES .shtml - - -# -# The mod_mime_magic module allows the server to use various hints from the -# contents of the file itself to determine its type. The MIMEMagicFile -# directive tells the module where the hint definitions are located. -# -#MIMEMagicFile conf/magic - -# -# Customizable error responses come in three flavors: -# 1) plain text 2) local redirects 3) external redirects -# -# Some examples: -#ErrorDocument 500 "The server made a boo boo." -#ErrorDocument 404 /missing.html -#ErrorDocument 404 "/cgi-bin/missing_handler.pl" -#ErrorDocument 402 http://www.example.com/subscription_info.html -# - -# -# MaxRanges: Maximum number of Ranges in a request before -# returning the entire resource, or one of the special -# values 'default', 'none' or 'unlimited'. -# Default setting is to accept 200 Ranges. -#MaxRanges unlimited - -# -# EnableMMAP and EnableSendfile: On systems that support it, -# memory-mapping or the sendfile syscall may be used to deliver -# files. This usually improves server performance, but must -# be turned off when serving from networked-mounted -# filesystems or if support for these functions is otherwise -# broken on your system. -# Defaults: EnableMMAP On, EnableSendfile Off -# -EnableMMAP off -#EnableSendfile on - -# Supplemental configuration -# -# The configuration files in the conf/extra/ directory can be -# included to add extra features or to modify the default configuration of -# the server, or you may simply copy their contents here and change as -# necessary. - -# Server-pool management (MPM specific) -#Include conf/extra/httpd-mpm.conf - -# Multi-language error messages -#Include conf/extra/httpd-multilang-errordoc.conf - -# Fancy directory listings -#Include conf/extra/httpd-autoindex.conf - -# Language settings -#Include conf/extra/httpd-languages.conf - -# User home directories -#Include conf/extra/httpd-userdir.conf - -# Real-time info on requests and configuration -#Include conf/extra/httpd-info.conf - -# Virtual hosts -#Include conf/extra/httpd-vhosts.conf - -# Local access to the Apache HTTP Server Manual -#Include conf/extra/httpd-manual.conf - -# Distributed authoring and versioning (WebDAV) -#Include conf/extra/httpd-dav.conf - -# Various default settings -#Include conf/extra/httpd-default.conf - -# Configure mod_proxy_html to understand HTML4/XHTML1 - -Include conf/extra/proxy-html.conf - - -# Secure (SSL/TLS) connections -#Include conf/extra/httpd-ssl.conf -# -# Note: The following must must be present to support -# starting without SSL on platforms with no /dev/random equivalent -# but a statically compiled-in mod_ssl. -# - -SSLRandomSeed startup builtin -SSLRandomSeed connect builtin - diff --git a/pkgserver/nginx-default-vhost.conf b/pkgserver/nginx-default-vhost.conf new file mode 100644 index 00000000..c690d780 --- /dev/null +++ b/pkgserver/nginx-default-vhost.conf @@ -0,0 +1,33 @@ +server { + listen 80; + listen [::]:80; + # The underscore is a "catchall" for Host headers (even invalid hosts) + server_name localhost test.pkg.jenkins.io _; + + ############### + # Only for testing: replace 'pkg.jenkins.io' by 'test.pkg.jenkins.io' + # Otherwise the HTML instructions won't be up to date + # Do NOT enable this in production (performance is not good) + sub_filter 'pkg.jenkins.io' 'test.pkg.jenkins.io'; + sub_filter_types '*'; + sub_filter_once off; + ############### + + location / { + root /usr/share/nginx/html; + index index.html index.htm; + + # Enable directory listing + autoindex on; + } + + add_header X-Content-Type-Options "nosniff"; + add_header X-Frame-Options "DENY"; + + # redirect server error pages to the static page /50x.html + # + error_page 500 502 503 504 /50x.html; + location = /50x.html { + root /usr/share/nginx/html; + } +} diff --git a/prep.sh b/prep.sh index ea7a93ae..64ed74b3 100755 --- a/prep.sh +++ b/prep.sh @@ -8,8 +8,29 @@ cd "$(dirname "$0")" # releases, ideally implemented in the Makefile. Then both this repository and # jenkins-infra/release should be refactored to consume the new functionality. -if [[ ! -f $WAR ]]; then - jv download +# Harcoding a safe Weekly version to 2.541 in this stable-2.541 branch for 2.541 LTS as 2.545 doesn't currently pass tests +safeWeeklyVersion="2.541" + +if [[ ! -f "$WAR" ]]; then + # # jv utilizes the JENKINS_VERSION environment variable which can be the line (latest/weekly/lts/stable) or an exact version + # jv download --version-identifier "${jenkinsVersion}" + + # Using hardcoded safe version for this stable branch + jv download --version-identifier "${safeWeeklyVersion}" +fi + +if [[ ! -f "${WAR}.asc" ]]; then + # # jv utilizes the JENKINS_VERSION environment variable which can be the line (latest/weekly/lts/stable) or an exact version + # jenkinsVersion="$(jv get)" + + # Using hardcoded safe version for this stable branch + jenkinsVersion="${safeWeeklyVersion}" + + # Download signature from Artifactory (signed by Maven during the release process) + # TODO: switch to get.jenkins.io once https://github.com/jenkins-infra/helpdesk/issues/4055 is finished + warSignatureUrl="https://repo.jenkins-ci.org/releases/org/jenkins-ci/main/jenkins-war/${jenkinsVersion}/jenkins-war-${jenkinsVersion}.war.asc" + curl --fail --silent --show-error --location --output "${WAR}.asc" \ + "${warSignatureUrl}" fi if ! gpg --fingerprint "${GPG_KEYNAME}"; then diff --git a/rpm/publish/publish.sh b/rpm/publish/publish.sh index 87eb19e4..01cda99b 100755 --- a/rpm/publish/publish.sh +++ b/rpm/publish/publish.sh @@ -8,6 +8,7 @@ set -euxo pipefail : "${RPM_URL:?Require rpm repository url}" : "${RELEASELINE?Require rpm release line}" : "${BASE:?Require base directory}" +: "${GPG_PUBLIC_KEY_FILENAME:="${ORGANIZATION}.key"}" # $$ Contains current pid D="$AGENT_WORKDIR/$$" @@ -17,10 +18,14 @@ function clean() { } function generateSite() { - local gpg_publickey="$D/repodata/repomd.xml.key" - mkdir -p "$(dirname "${gpg_publickey}")" - gpg --export -a --output "${gpg_publickey}" "${GPG_KEYNAME}" - gpg --import-options show-only --import "${gpg_publickey}" >"$D/${ORGANIZATION}.key.info" + local gpg_publickey_repomd="$D/repodata/repomd.xml.key" + local gpg_publickey_file="$D/${GPG_PUBLIC_KEY_FILENAME}" + local gpg_publickey_info_file="$D/${GPG_PUBLIC_KEY_FILENAME}.info" + + mkdir -p "$(dirname "${gpg_publickey_repomd}")" + gpg --export -a --output "${gpg_publickey_repomd}" "${GPG_KEYNAME}" + gpg --import-options show-only --import "${gpg_publickey_repomd}" > "${gpg_publickey_info_file}" + cp "${gpg_publickey_repomd}" "${gpg_publickey_file}" # Duplicate between repository files and user facing website cat >"$D/${ARTIFACTNAME}.repo" < - + @@ -71,64 +71,10 @@

{% block title %}{{product_name | capitalize }} {{ os_family | capitalize}} (as described above) or another Java vendor (e.g., Adoptium).

-

- Weekly Release Line -

- -

- Supported Java versions for the weekly release line are: -

- -
-
2.463 (June 2024) and newer
-
Java 17 or Java 21
- -
2.419 (August 2023) and newer
-
Java 11, Java 17, or Java 21
- -
2.357 (June 2022) and newer
-
Java 11 or Java 17
- -
2.164 (February 2019) and newer
-
Java 8 or Java 11
- -
2.54 (April 2017) and newer
-
Java 8
- -
1.612 (May 2015) and newer
-
Java 7
-
- -

- Long Term Support (LTS) Release Line -

- -

- Supported Java versions for the LTS release line are: -

- -
-
2.479.1 (October 2024) and newer
-
Java 17 or Java 21
- -
2.426.1 (November 2023) and newer
-
Java 11, Java 17 or Java 21
- -
2.361.1 (September 2022) and newer
-
Java 11 or Java 17
- -
2.346.1 (June 2022) and newer
-
Java 8, Java 11, or Java 17
- -
2.164.1 (March 2019) and newer
-
Java 8 or Java 11
- -
2.60.1 (June 2017) and newer
-
Java 8
+

+ To determine the Java version that is supported for your Jenkins environment, please refer to the Java support policy. +

-
1.625.1 (October 2015) and newer
-
Java 7
-
{% endblock %}

diff --git a/templates/header.debian.html b/templates/header.debian.html index cb96f629..89cfaf1d 100644 --- a/templates/header.debian.html +++ b/templates/header.debian.html @@ -10,7 +10,7 @@ 

     
   sudo wget -O /etc/apt/keyrings/jenkins-keyring.asc \
-    {{web_url}}/{{organization}}-2023.key
+    {{web_url}}/{{gpg_public_key_filename}}
Then add a Jenkins apt repository entry: diff --git a/war/publish/publish.sh b/war/publish/publish.sh index b9fc9340..d6c6ac71 100755 --- a/war/publish/publish.sh +++ b/war/publish/publish.sh @@ -5,6 +5,9 @@ set -euxo pipefail : "${AGENT_WORKDIR:=/tmp}" : "${WAR:?Require Jenkins War file}" : "${WARDIR:? Require where to put binary files}" +: "${JENKINS_ASC:=${WAR}.asc}" +: "${GPG_PUBLIC_KEY_FILENAME:="${ORGANIZATION}.key"}" +: "${GPG_KEYNAME:?Require valid gpg keyname}" # $$ Contains current pid D="$AGENT_WORKDIR/$$" @@ -28,22 +31,19 @@ function uploadPackage() { sha256sum "${WAR}" | sed "s, .*, ${ARTIFACTNAME}.war," >"${WAR_SHASUM}" cat "${WAR_SHASUM}" - # Update the symlink to point to most recent Windows build - # - # Remove anything in current directory named 'latest' - # This is a safety measure just in case something was left there previously - rm -rf latest - - # Create a local symlink pointing to the MSI file in the VERSION directory. - # Don't need VERSION directory or MSI locally, just the unresolved symlink. - # The jenkins.io page downloads http://mirrors.jenkins-ci.org/windows/latest - # and assumes it points to the most recent MSI file. - ln -s "${VERSION}/$(basename "$WAR")" latest + gpg --export -a --output "${GPG_PUBLIC_KEY_FILENAME}" "${GPG_KEYNAME}" rsync --archive \ --verbose \ --progress \ - "${WAR}" "${WAR_SHASUM}" latest "${WARDIR}/${VERSION}/" + "${WAR}" "${WAR_SHASUM}" "${JENKINS_ASC}" "${GPG_PUBLIC_KEY_FILENAME}" `# Sources` \ + "${WARDIR}/${VERSION}/" `# Destination` + + # Update the symlink to point to most recent WAR directory + pushd "${WARDIR}" + rm -rf latest # This is a safety measure just in case something was left there previously + ln -s "${VERSION}" latest + popd } # Site html need to be located in the binary directory