From d255f24b6e3810876a68613faf76e848f1263650 Mon Sep 17 00:00:00 2001 From: Kirk Date: Tue, 28 Jan 2025 03:21:45 +0000 Subject: [PATCH 1/5] Update docs with new packages and improvements --- ...ocal-client.md => setup-local-exporter.md} | 36 ++++++++++------ docs/source/installation/container-jmp.md | 17 ++++---- docs/source/installation/index.md | 13 +++--- docs/source/installation/service/index.md | 4 +- docs/source/installation/service/kind-helm.md | 38 ++++++++++++++++- .../installation/service/kubernetes-helm.md | 4 +- .../installation/service/minikube-helm.md | 38 +++++++++++++++++ docs/source/introduction/clients.md | 6 +-- docs/source/introduction/drivers.md | 41 ++++++++++--------- docs/source/introduction/exporters.md | 13 ++++-- docs/source/introduction/how-it-works.md | 4 +- docs/source/introduction/service.md | 6 +-- 12 files changed, 158 insertions(+), 62 deletions(-) rename docs/source/getting-started/{setup-local-client.md => setup-local-exporter.md} (67%) diff --git a/docs/source/getting-started/setup-local-client.md b/docs/source/getting-started/setup-local-exporter.md similarity index 67% rename from docs/source/getting-started/setup-local-client.md rename to docs/source/getting-started/setup-local-exporter.md index 11ef4c303..2f98b5593 100644 --- a/docs/source/getting-started/setup-local-client.md +++ b/docs/source/getting-started/setup-local-exporter.md @@ -1,9 +1,12 @@ # Setup a Local Exporter -This guide walks you through the process of using Jumpstarter with a local exporter (the client and the exporter running on the same host) +This guide walks you through the process of using Jumpstarter with a local +exporter (i.e. the client and the exporter running on the same host). + +## Create an Exporter Config + +Create a text file with the following content: -## Create Exporter Configuration -Create a text file with the following content ```yaml # /etc/jumpstarter/exporters/demo.yaml apiVersion: jumpstarter.dev/v1alpha1 @@ -16,23 +19,30 @@ endpoint: "" token: "" # mock drivers for demo purpose export: - storage: - type: jumpstarter.drivers.storage.driver.MockStorageMux - power: - type: jumpstarter.drivers.power.driver.MockPower + storage: + type: jumpstarter.drivers.storage.driver.MockStorageMux + power: + type: jumpstarter.drivers.power.driver.MockPower ``` -Once the exporter configuration is ready it should be placed at `/etc/jumpstarter/exporters/demo.yaml`. +Once the exporter configuration is ready it should be placed at +`/etc/jumpstarter/exporters/demo.yaml`. ## Enter Exporter Shell -Now we can run the following command to enter the "Exporter Shell", inside which we can interact with the local exporter with Jumpstarter client. + +Now we can run the following command to enter the "Exporter Shell", inside +which we can interact with the local exporter with Jumpstarter client. + ```shell -$ jmp-exporter shell demo +$ jmp exporter shell demo ``` -## Use j Command -The `j` command is available in the exporter shell for controlling the exporter with shell commands. +## Use the `j` Command + +The `j` command is available in the exporter shell for controlling the exporter +with shell commands. + ```shell -# running inside exporter shell +# Running inside exporter shell $ j Usage: j [OPTIONS] COMMAND [ARGS]... diff --git a/docs/source/installation/container-jmp.md b/docs/source/installation/container-jmp.md index 5a6656098..a296c012c 100644 --- a/docs/source/installation/container-jmp.md +++ b/docs/source/installation/container-jmp.md @@ -1,4 +1,4 @@ -# Container package +# Running in a Container For interacting with the Jumpstarter service without installing the Python packages locally, you can create an alias to run the `jmp` client in a container. @@ -17,18 +17,19 @@ $ alias jmp='podman run --rm -it -e JUMPSTARTER_GRPC_INSECURE=1 \ Then you can try: ```bash -$ jmp list +$ jmp client list-configs CURRENT NAME ENDPOINT PATH * default grpc.devel.jumpstarter.dev:443 /root/.config/jumpstarter/clients/default.yaml - kirkb grpc.devel.jumpstarter.dev:443 /root/.config/jumpstarter/clients/kirkb.yaml + test grpc.devel.jumpstarter.dev:443 /root/.config/jumpstarter/clients/test.yaml ``` -## Hardware Access for exporters +## Hardware Access for Exporters -If you need access to your hardware, i.e. because you are running the `exporter` -or you are following the `local workflow` (without a distributed service), you need -to mount access to devices into the container, provide host network access, -and run the container in privileged mode, this probably needs to be run as **root**. +If you need access to your hardware, i.e. because you are running the `jmp exporter` +or you are following the [local-only workflow](../introduction/how-it-works.md#local-only) +(i.e. without a distributed service), you need to mount access to devices into +the container, provide host network access, and run the container in privileged +mode, this probably needs to be run as **root**. ```{code-block} bash diff --git a/docs/source/installation/index.md b/docs/source/installation/index.md index d3d5caf6d..59f5317f9 100644 --- a/docs/source/installation/index.md +++ b/docs/source/installation/index.md @@ -6,11 +6,14 @@ documentation. The Jumpstarter components which can be installed are: -| Component | Description | -| --------- | ----------- | -| jumpstarter-controller | The Jumpstarter Controller service, runs in k8s, manages exporters, clients, leases, and provides routing | -| jmpctl | The Jumpstarter administrator CLI tool, simplifies the administrator experience when managing clients or exporters on the controller | -| jmp /jmp-exporter / j / python packages | The Jumpstarter Python package. This is necessary to lease and interact with the exporters, it's also the component ran on the exporter hosts as a service. In most cases installation is not necessary and can be consumed through a container. | +| Component | Description | +| -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [`jumpstarter-controller`](https://github.com/jumpstarter-dev/jumpstarter-controller) | The Jumpstarter Controller service, runs in k8s, manages exporters, clients, leases, and provides routing between the clients and exporters. | +| [`jumpstarter`](https://github.com/jumpstarter-dev/jumpstarter/tree/main/packages/jumpstarter) | The core Jumpstarter Python package. This is necessary to lease and interact with the exporters, it's also the component that runs on the exporter hosts as a service. In most cases installation is not necessary and can be consumed through another package such as `jumpstarter-cli`. | +| [`jumpstarter-cli`](https://github.com/jumpstarter-dev/jumpstarter/tree/main/packages/jumpstarter-cli) | A metapackage containing all of the Jumpstarter CLI components including the cluster admin CLI `jumpstarter-cli-admin`, the client CLI `jumpstarter-cli-client`, and exporter CLI `jumpstarter-cli-exporter`. | +| [`jumpstarter-cli-admin`](https://github.com/jumpstarter-dev/jumpstarter/tree/main/packages/jumpstarter-cli-admin) | The Jumpstarter admin CLI (`jmp-admin`). This CLI can be used to install the Jumpstarter controller, manage client/exporter registrations, and monitor/control leases. | +| [`jumpstarter-cli-client`](https://github.com/jumpstarter-dev/jumpstarter/tree/main/packages/jumpstarter-cli-client) | The Jumpstarter client CLI (`jmp-client`/`j`). This CLI can be used to manage local client configs, start a lease on an exporter, and enter an interactive client shell. | +| [`jumpstarter-driver-*`](https://github.com/jumpstarter-dev/jumpstarter/tree/main/packages) | All community and official driver packages that are distributed as part of Jumpstarter are prefixed with `jumpstarter-driver-*`. This includes drivers for PySerial, SD Wire, HTTP, CAN, and more. Driver packages only need to be installed on the exporter/client if they are used by your testing environment. All drivers are optional. | ```{toctree} diff --git a/docs/source/installation/service/index.md b/docs/source/installation/service/index.md index ff9ffd96f..eea5fd001 100644 --- a/docs/source/installation/service/index.md +++ b/docs/source/installation/service/index.md @@ -1,11 +1,11 @@ # Service To manage your Jumpstarter lab from Kubernetes, the Jumpstarter Service must -be installed in your Kubernetes cluster. +be installed in your cluster. ## Prerequisites -- A *Kubernetes* or *OpenShift* cluster. +- A *Kubernetes* cluster. - [Kubectl](https://www.downloadkubernetes.com/) - [Helm](https://helm.sh/docs/intro/install/) diff --git a/docs/source/installation/service/kind-helm.md b/docs/source/installation/service/kind-helm.md index 5fb59a6de..4ee5e73e5 100644 --- a/docs/source/installation/service/kind-helm.md +++ b/docs/source/installation/service/kind-helm.md @@ -62,12 +62,48 @@ Next, create a kind cluster using the config you created. kind create cluster --config kind_config.yaml ``` -### Install Jumpstarter with Helm +### Install Jumpstarter with the CLI + +To simplify the installation in your Kubernetes cluster, the Jumpstarter CLI +provides the `jmp admin install` command to automatically run Helm with the +correct arguments. ```{tip} If you do not have Helm installed, please [install the latest release](https://helm.sh/docs/intro/install/). ``` +``` +# Install Jumpstarter with default options +$ jmp admin install + +# Options provided by the Jumpstarter CLI +$ jmp admin install --help +Usage: jmp admin install [OPTIONS] + + Install the Jumpstarter service in a Kubernetes cluster + +Options: + --helm TEXT Path or name of a helm executable + --name TEXT The name of the chart installation + -c, --chart TEXT The URL of a Jumpstarter helm chart to install + -n, --namespace TEXT Namespace to install Jumpstarter components in + -i, --ip TEXT IP address of your host machine + -b, --basedomain TEXT Base domain of the Jumpstarter service + -g, --grpc-endpoint TEXT The gRPC endpoint to use for the Jumpstarter API + -r, --router-endpoint TEXT The gRPC endpoint to use for the router + --nodeport Use Nodeport routing (recommended) + --ingress Use a Kubernetes ingress + --route Use an OpenShift route + -v, --version TEXT The version of the service to install + --kubeconfig FILENAME path to the kubeconfig file + --context TEXT Kubernetes context to use + --help Show this message and exit. +``` + +### Install Jumpstarter with Helm + +If you prefer to manually install with Helm, the following command should work. + ```{code-block} bash :substitutions: # Get the IP address of your computer diff --git a/docs/source/installation/service/kubernetes-helm.md b/docs/source/installation/service/kubernetes-helm.md index 0f2f87454..513078bd9 100644 --- a/docs/source/installation/service/kubernetes-helm.md +++ b/docs/source/installation/service/kubernetes-helm.md @@ -2,9 +2,7 @@ The Jumpstarter service can be installed on a Kubernetes cluster using Helm. -```{warning} -This configuration hasn't been fully tested, please report any issues you may find. -``` +## Install with Helm ```{note} Please note that `global.baseDomain` is used to create the host names for the services, diff --git a/docs/source/installation/service/minikube-helm.md b/docs/source/installation/service/minikube-helm.md index e88eab64e..7f1797ca4 100644 --- a/docs/source/installation/service/minikube-helm.md +++ b/docs/source/installation/service/minikube-helm.md @@ -24,6 +24,44 @@ First, we must start a local minikube cluster with the correct features enabled minikube start --extra-config=apiserver.service-node-port-range=8000-9000 ``` +### Install Jumpstarter with the CLI + +To simplify the installation in your Kubernetes cluster, the Jumpstarter CLI +provides the `jmp admin install` command to automatically run Helm with the +correct arguments. + +```{tip} +If you do not have Helm installed, please [install the latest release](https://helm.sh/docs/intro/install/). +``` + +``` +# Install Jumpstarter with default options +$ jmp admin install --ip $(minikube ip) + +# Options provided by the Jumpstarter CLI +$ jmp admin install --help +Usage: jmp admin install [OPTIONS] + + Install the Jumpstarter service in a Kubernetes cluster + +Options: + --helm TEXT Path or name of a helm executable + --name TEXT The name of the chart installation + -c, --chart TEXT The URL of a Jumpstarter helm chart to install + -n, --namespace TEXT Namespace to install Jumpstarter components in + -i, --ip TEXT IP address of your host machine + -b, --basedomain TEXT Base domain of the Jumpstarter service + -g, --grpc-endpoint TEXT The gRPC endpoint to use for the Jumpstarter API + -r, --router-endpoint TEXT The gRPC endpoint to use for the router + --nodeport Use Nodeport routing (recommended) + --ingress Use a Kubernetes ingress + --route Use an OpenShift route + -v, --version TEXT The version of the service to install + --kubeconfig FILENAME path to the kubeconfig file + --context TEXT Kubernetes context to use + --help Show this message and exit. +``` + ### Install Jumpstarter with Helm ```{tip} diff --git a/docs/source/introduction/clients.md b/docs/source/introduction/clients.md index d128ca3fd..b0d1ba753 100644 --- a/docs/source/introduction/clients.md +++ b/docs/source/introduction/clients.md @@ -10,7 +10,7 @@ The Python client can be used either as a library through a testing tool such as Jumpstarter supports two types of client configurations: local and remote. -### Local +### Local Clients When using Jumpstarter in *local-only* mode, you can use the local client functionality to directly access your hardware. The local client will @@ -20,10 +20,10 @@ exporter instance running in the background. You can get started with a local client by following the [Setup a Local Client](../getting-started/setup-local-client.md) guide. -### Remote +### Remote Clients When using Jumpstarter in *distributed* mode, the client must be configured to -connect to an instance of the Jumpstarter Service that can authenticate clients +connect to an instance of the Jumpstarter Service that can authenticate and tunnel requests to the appropriate exporter instance. The following parameters are required to setup a remote client: diff --git a/docs/source/introduction/drivers.md b/docs/source/introduction/drivers.md index 1420c6a02..fbdd45580 100644 --- a/docs/source/introduction/drivers.md +++ b/docs/source/introduction/drivers.md @@ -4,12 +4,12 @@ Jumpstarter uses a modular driver model to build abstractions around the hardwar interfaces used to interact with a target device. An [Exporter](./exporters.md) uses Drivers to "export" the hardware interfaces -from the host machine to the clients via gRPC. Drivers can be thought of as a -simplified API for an interface or device type. +from a host machine to the clients via [gRPC](https://grpc.io/). +Drivers can be thought of as a simplified API for an interface or device type. Each driver consists of two components: -- A driver class that implements the "backend" functionality of the driver. -- A client class (optional) that provides a Python and CLI "frontend" for the driver. +- An exporter-side module that implements the "backend" functionality of the driver. +- A client that provides a Python interface and optionally a CLI "frontend" for the driver. While Jumpstarter comes with drivers for many basic interfaces, custom drivers can also be developed for specialized hardware/interfaces or to provide @@ -21,7 +21,7 @@ Drivers are configured using a YAML Exporter config file, which specifies the drivers to load and the parameters for each. Drivers are distributed as Python packages making it easy to develop and install your own drivers. -Here is an example exporter config that loads a driver: +Here is an example exporter config that loads a custom driver called `jumpstarter_driver_dutlink`: ```yaml apiVersion: jumpstarter.dev/v1alpha1 @@ -32,12 +32,14 @@ metadata: endpoint: grpc.jumpstarter.example.com:443 token: xxxxx export: - # a DUTLink interface to the DUT + # The name to register the driver instance as dutlink: + # A fully-qualified Python module type: jumpstarter_driver_dutlink.driver.Dutlink + # Configuration parameters passed to the driver implementation config: storage_device: "/dev/disk/by-id/usb-SanDisk_3.2_Gen_1_5B4C0AB025C0-0:0" - # using the DUTLink power interface individually + # Another driver instance for this device power: type: jumpstarter_driver_dutlink.driver.DutlinkPower config: @@ -64,17 +66,18 @@ TCP port forwarding and CAN bus emulation. ## Composite Drivers -In Jumpstarter, drivers are organized in a tree structure which allows for -the development of hierarchical relationships between interfaces and their -associated devices. +In Jumpstarter, drivers are organized in a tree structure which allows for the +representation of complex device trees that may be found in your environment. -Multiple drivers can be combined to create a Composite Driver with additional -device-specific functionality for your use case. +For example, your team may use a custom test harness that connects to the host +via USB, but provides multiple hardware interfaces through a built-in USB hub. +Jumpstarter allows you to create a custom Composite Driver that provides a unified +interface to access all the interfaces provided by your harness. Here is an example of a driver tree with two custom composite drivers: ``` -MyDevice # Custom composite driver for the entire target device +MyHarness # Custom composite driver for the entire target device harness ├─ TcpNetwork # TCP Network driver to tunnel port 8000 ├─ MyPower # Custom power driver to control device power ├─ SDWire # SD Wire storage emulator to enable re-flash on demand @@ -83,10 +86,10 @@ MyDevice # Custom composite driver for the entire target device    └─ PySerial # Serial debugger with PySerial ``` -Creating a composite driver can automate common tasks related to your specific -hardware configuration such as flashing a system image or entering a debug mode. - -For example, you may want to develop a composite driver that enables a test script -to perform a complex action that requires multiple other driver interfaces with -a single method invocation. +The `jumpstarter_driver_composite` package provides the base `Composite` and +`CompositeClient` classes that can be used to build custom composite drivers. +A composite driver can also be used to orchestrate multiple interfaces to preform +a specific task such as flashing a system image or entering a debug mode. While +the driver may internally preform a complex task, a simple, user-friendly interface +can be provided to the driver clients. diff --git a/docs/source/introduction/exporters.md b/docs/source/introduction/exporters.md index c9f0d3735..90bc12e05 100644 --- a/docs/source/introduction/exporters.md +++ b/docs/source/introduction/exporters.md @@ -22,6 +22,7 @@ and the configuration required. Here is an example Exporter config file: ```yaml +# /etc/jumpstarter/exporters/myexporter.yaml apiVersion: jumpstarter.dev/v1alpha1 kind: ExporterConfig metadata: @@ -40,8 +41,14 @@ export: ## Running an Exporter To run an Exporter on a host system, you must have Python {{requires_python}} installed -and the required driver packages installed locally. +and the driver packages specified in the config installed in your current Python +environment. -Exporters can be run in a privileged container or as a systemd daemon. It is +```bash +# Run the exporter myexporter in your local terminal +$ jmp exporter run myexporter +``` + +Exporters can also be run in a privileged container or as a systemd daemon. It is recommended to run the Exporter service in the background with auto-restart -in case something goes wrong. +in case something goes wrong and it needs to be restarted. diff --git a/docs/source/introduction/how-it-works.md b/docs/source/introduction/how-it-works.md index 71863fb64..42eaae58b 100644 --- a/docs/source/introduction/how-it-works.md +++ b/docs/source/introduction/how-it-works.md @@ -38,9 +38,9 @@ depending on your hardware development needs. ### Local-Only When using Jumpstarter locally, you can easily develop drivers, write automated -tests, and control your hardware from your development machine +tests, and control your hardware directly from your development machine. -The *local-only mode* is useful when working directly with hardware on your desk +The *local-only mode* is useful when working with hardware on your desk that you have unlimited access to. ![local mode](how-it-works-local.svg) diff --git a/docs/source/introduction/service.md b/docs/source/introduction/service.md index 90863e9de..da4f5b054 100644 --- a/docs/source/introduction/service.md +++ b/docs/source/introduction/service.md @@ -2,7 +2,7 @@ When building a lab with many devices under test, it quickly becomes difficult to keep track of devices, schedule access for automated tests, and perform routine -tasks such as batch updates. +maintenance such as batch updates. Jumpstarter provides a [Cloud Native](https://www.cncf.io/) service that can be installed in any [Kubernetes](https://kubernetes.io/) cluster to manage @@ -42,5 +42,5 @@ exported interfaces via the client. Once a lease is established, all traffic between the client and the exporter flows through a router instance. While there may only be one controller, -the router can be scaled to handle traffic from many clients/exporters -simultaneously. +the router can be scaled with multiple instance handle traffic between many +clients and exporters simultaneously. From 0e19dd9fb3b0a8d481440ad178072b37bc549feb Mon Sep 17 00:00:00 2001 From: Kirk Date: Tue, 28 Jan 2025 03:58:30 +0000 Subject: [PATCH 2/5] Update local exporter guide --- .../getting-started/setup-local-exporter.md | 135 +++++++++++++++--- 1 file changed, 119 insertions(+), 16 deletions(-) diff --git a/docs/source/getting-started/setup-local-exporter.md b/docs/source/getting-started/setup-local-exporter.md index 2f98b5593..47987608c 100644 --- a/docs/source/getting-started/setup-local-exporter.md +++ b/docs/source/getting-started/setup-local-exporter.md @@ -5,7 +5,21 @@ exporter (i.e. the client and the exporter running on the same host). ## Create an Exporter Config -Create a text file with the following content: +First, we must create an exporter config to define the "shape" of the exporter +that we are going to test locally. This config is identical to a regular exporter +config, however, the `endpoint` and `token` fields may be left empty as we do +not need to connect to the controller service. + +Make sure the following driver packages are installed in your Python environment: +- `jumpstarter-driver-opendal` +- `jumpstarter-driver-power` + +```{tip} +Both of these driver packages provide mock implementations, this makes it easier +to debug the connection between an exporter and client without hardware. +``` + +Create a text file in `/etc/jumpstarter/exporters` with the following content: ```yaml # /etc/jumpstarter/exporters/demo.yaml @@ -20,28 +34,35 @@ token: "" # mock drivers for demo purpose export: storage: - type: jumpstarter.drivers.storage.driver.MockStorageMux + type: jumpstarter_driver_opendal.driver.MockStorageMux power: - type: jumpstarter.drivers.power.driver.MockPower + type: jumpstarter_driver_power.driver.MockPower ``` -Once the exporter configuration is ready it should be placed at -`/etc/jumpstarter/exporters/demo.yaml`. -## Enter Exporter Shell +```{note} +The name of this file is used when referring to the exporter in later steps. +``` -Now we can run the following command to enter the "Exporter Shell", inside -which we can interact with the local exporter with Jumpstarter client. +## Span an Exporter Shell + +To interact locally with the exporter we created above, we can use the +"exporter shell" functionality within the `jmp` CLI. When a shell is spawned, +a local exporter instance is run interactively, allowing you to interact with +it through the client CLI. ```shell +# Spawn a new exporter shell for "demo" $ jmp exporter shell demo ``` -## Use the `j` Command +## Interact with the Exporter Shell -The `j` command is available in the exporter shell for controlling the exporter -with shell commands. +If the drivers specified in the exporter config provide a CLI interface, it will +be available though the magic `j` command within the exporter shell. ```shell +# Enter the shell +$ jmp exporter shell demo # Running inside exporter shell $ j Usage: j [OPTIONS] COMMAND [ARGS]... @@ -54,22 +75,104 @@ Options: Commands: power Generic power storage Generic storage mux +# Simulate turning on the power $ j power on ok +# Exit the shell +$ exit ``` -## Use Python API -Or we can use the Jumpstarter Python API to programmatically control the exporter. +## Use the Python API in a Shell + +As the shell exposes the local exporter via environment variables, we can run +any command or Python script that interact with the exporter. + +This allows us to use the Python API directly without having to interact with +the CLI. This is often useful for more complex scripts or if a specific driver +doesn't provide a CLI. + +### Running a Python Script + +The easiest way to interact with the exporter is to run a quick Python script +directly from the command line. This comes in handy when no CLI is available. ```shell -# running inside exporter shell +# Enter the shell +$ jmp exporter shell demo +# Running python inside exporter shell $ python - < Date: Wed, 29 Jan 2025 03:52:06 +0000 Subject: [PATCH 3/5] Expand getting started guides --- docs/source/getting-started/index.md | 5 +- .../getting-started/setup-controller.md | 1 - .../getting-started/setup-exporter-client.md | 172 ++++++++++++++++++ docs/source/getting-started/setup-exporter.md | 1 - .../getting-started/setup-local-exporter.md | 57 +++--- .../getting-started/setup-remote-client.md | 1 - docs/source/installation/index.md | 1 + 7 files changed, 207 insertions(+), 31 deletions(-) delete mode 100644 docs/source/getting-started/setup-controller.md create mode 100644 docs/source/getting-started/setup-exporter-client.md delete mode 100644 docs/source/getting-started/setup-exporter.md delete mode 100644 docs/source/getting-started/setup-remote-client.md diff --git a/docs/source/getting-started/index.md b/docs/source/getting-started/index.md index ca828c6a3..4866df629 100644 --- a/docs/source/getting-started/index.md +++ b/docs/source/getting-started/index.md @@ -4,7 +4,6 @@ This section provides getting started guide to help you set up Jumpstarter to test your software on real hardware. ```{toctree} -setup-local-client.md -setup-exporter.md -setup-controller.md +setup-local-exporter.md +setup-exporter-client.md ``` diff --git a/docs/source/getting-started/setup-controller.md b/docs/source/getting-started/setup-controller.md deleted file mode 100644 index 63a93cc5b..000000000 --- a/docs/source/getting-started/setup-controller.md +++ /dev/null @@ -1 +0,0 @@ -# Setup the Controller Service \ No newline at end of file diff --git a/docs/source/getting-started/setup-exporter-client.md b/docs/source/getting-started/setup-exporter-client.md new file mode 100644 index 000000000..09434d86c --- /dev/null +++ b/docs/source/getting-started/setup-exporter-client.md @@ -0,0 +1,172 @@ +# Setup a Remote Exporter/Client + +This guide walks you through the process of creating an exporter using the +controller service, configuring drivers, and running the exporter. + +## Prerequisites + +Make sure the following packages are installed in your Python environment: +- `jumpstarter-cli` - The core Jumpstarter CLI. +- `jumpstarter-driver-opendal` - The OpenDAL storage driver. +- `jumpstarter-driver-power` - The base power driver. + +You should also have the [Jumpstarter Service](../introduction/service.md) +running in a Kubernetes cluster you have admin access to. +For instructions on how to install Jumpstarter in a cluster, +refer to the [installation guide](../installation/service/index.md). + +```{tip} +Make sure you have the correct cluster in your `kubeconfig` file and the right +context selected. +``` + +## Create an Exporter + +First, we must create an exporter using the controller service API. +The `jmp admin` CLI provides methods to interact with the controller directly. + +To create an exporter and save the config locally, run the following command: + +```bash +# Creates an exporter called "testing" and saves the config +$ jmp admin create exporter testing --save + +# Usage for jmp admin create exporter +$ jmp admin create exporter + --help +Usage: jmp admin create exporter [OPTIONS] [NAME] + + Create an exporter object in the Kubernetes cluster + +Options: + -s, --save Save the config file for the created exporter. + -o, --out FILE Specify an output file for the exporter config. + -n, --namespace TEXT Kubernetes namespace to use + --kubeconfig FILENAME path to the kubeconfig file + --context TEXT Kubernetes context to use + --help Show this message and exit. +``` + +### Edit the Exporter Config + +Once the exporter has been created, a new config file will be saved to +`/etc/jumpstarter/exporters/testing.yaml`. + +To edit the config file with your default text editor, run the following command: + +```bash +# Opens the config for "testing" in your default editor +$ jmp exporter edit-config testing +``` + +Add the `storage` and `power` drivers under the `export` field in the config file. +The finished config should look like this: + +```yaml +# /etc/jumpstarter/exporters/testing.yaml +apiVersion: jumpstarter.dev/v1alpha1 +kind: ExporterConfig +# These values are automatically filled by the controller +endpoint: "..." +token: "..." +# Mock drivers for demo purpose +export: + storage: + type: jumpstarter_driver_opendal.driver.MockStorageMux + power: + type: jumpstarter_driver_power.driver.MockPower +``` + +## Run an Exporter + +To run the exporter locally, we can use the `jmp exporter` CLI tool. + +Run the following command to start the exporter locally using the config file: + +```bash +# Runs the exporter "testing" locally +$ jmp exporter run testing +``` + +The exporter will stay running until the process is exited via `^C` or the shell +is closed. + +## Create a Client + +To connect to the new exporter, a client must be created. We can also use the +`jmp admin` CLI tool to create a client using the controller. + +```bash +# This will create a client called "hello", allow unsafe drivers, and save the config +$ jmp admin create client hello --save --unsafe + +# Usage for jmp admin create client +jmp admin create client --help +Usage: jmp admin create client [OPTIONS] [NAME] + + Create a client object in the Kubernetes cluster + +Options: + -s, --save Save the config file for the created client. + -a, --allow TEXT A comma-separated list of driver client packages to + load. + --unsafe Should all driver client packages be allowed to load + (UNSAFE!). + -o, --out FILE Specify an output file for the client config. + -n, --namespace TEXT Kubernetes namespace to use + --kubeconfig FILENAME path to the kubeconfig file + --context TEXT Kubernetes context to use + --help Show this message and exit. +``` + +## Connect to the Exporter + +To interact with the exporter we created above, we can use the +"client shell" functionality within the `jmp` CLI. When a shell is spawned, +the client attempts to acquire a lease on an exporter. Once the lease is acquired, +the client can be interacted with through the magic `j` command or via the +Python API. + +```bash +# Spawn a shell using the "hello" client +$ jmp client shell hello + +# Usage for jmp client shell +$ jmp client shell --help +Usage: jmp client shell [OPTIONS] [NAME] + + Spawns a shell connecting to a leased remote exporter + +Options: + -l, --label ... + -n, --lease TEXT + --help Show this message and exit. +``` + +Once a lease is acquired, we can interact with the drivers hosted by the exporter +within the shell instance. + +```bash +# Spawn a shell using the "hello" client +$ jmp client shell hello + +# Running inside client shell +$ j +Usage: j [OPTIONS] COMMAND [ARGS]... + + Generic composite device + +Options: + --help Show this message and exit. + +Commands: + power Generic power + storage Generic storage mux + +# Simulate turning on the power +$ j power on +ok + +# Exit the shell +$ exit +``` diff --git a/docs/source/getting-started/setup-exporter.md b/docs/source/getting-started/setup-exporter.md deleted file mode 100644 index 96a89b61d..000000000 --- a/docs/source/getting-started/setup-exporter.md +++ /dev/null @@ -1 +0,0 @@ -# Setup an Exporter \ No newline at end of file diff --git a/docs/source/getting-started/setup-local-exporter.md b/docs/source/getting-started/setup-local-exporter.md index 47987608c..2dd41b37b 100644 --- a/docs/source/getting-started/setup-local-exporter.md +++ b/docs/source/getting-started/setup-local-exporter.md @@ -3,6 +3,18 @@ This guide walks you through the process of using Jumpstarter with a local exporter (i.e. the client and the exporter running on the same host). +## Prerequisites + +Make sure the following packages are installed in your Python environment: +- `jumpstarter-cli` - The core Jumpstarter CLI. +- `jumpstarter-driver-opendal` - The OpenDAL storage driver. +- `jumpstarter-driver-power` - The base power driver. + +```{tip} +Both of these driver packages provide mock implementations, this makes it easier +to debug the connection between an exporter and client without hardware. +``` + ## Create an Exporter Config First, we must create an exporter config to define the "shape" of the exporter @@ -10,17 +22,12 @@ that we are going to test locally. This config is identical to a regular exporte config, however, the `endpoint` and `token` fields may be left empty as we do not need to connect to the controller service. -Make sure the following driver packages are installed in your Python environment: -- `jumpstarter-driver-opendal` -- `jumpstarter-driver-power` +Create a text file in `/etc/jumpstarter/exporters` with the following content: -```{tip} -Both of these driver packages provide mock implementations, this makes it easier -to debug the connection between an exporter and client without hardware. +```{note} +The name of this file is used when referring to the exporter in later steps. ``` -Create a text file in `/etc/jumpstarter/exporters` with the following content: - ```yaml # /etc/jumpstarter/exporters/demo.yaml apiVersion: jumpstarter.dev/v1alpha1 @@ -39,23 +46,19 @@ export: type: jumpstarter_driver_power.driver.MockPower ``` -```{note} -The name of this file is used when referring to the exporter in later steps. -``` - -## Span an Exporter Shell +## Spawn an Exporter Shell To interact locally with the exporter we created above, we can use the "exporter shell" functionality within the `jmp` CLI. When a shell is spawned, -a local exporter instance is run interactively, allowing you to interact with -it through the client CLI. +a local exporter instance is run in the background while the shell session is +running. ```shell # Spawn a new exporter shell for "demo" $ jmp exporter shell demo ``` -## Interact with the Exporter Shell +### Interact with the Exporter Shell If the drivers specified in the exporter config provide a CLI interface, it will be available though the magic `j` command within the exporter shell. @@ -63,6 +66,7 @@ be available though the magic `j` command within the exporter shell. ```shell # Enter the shell $ jmp exporter shell demo + # Running inside exporter shell $ j Usage: j [OPTIONS] COMMAND [ARGS]... @@ -75,23 +79,25 @@ Options: Commands: power Generic power storage Generic storage mux + # Simulate turning on the power $ j power on ok + # Exit the shell $ exit ``` -## Use the Python API in a Shell +### Use the Python API in a Shell As the shell exposes the local exporter via environment variables, we can run -any command or Python script that interact with the exporter. +any command or Python script that interacts with a client/exporter. This allows us to use the Python API directly without having to interact with the CLI. This is often useful for more complex scripts or if a specific driver -doesn't provide a CLI. +doesn't provide a client CLI. -### Running a Python Script +#### Running a Python Script The easiest way to interact with the exporter is to run a quick Python script directly from the command line. This comes in handy when no CLI is available. @@ -110,7 +116,7 @@ ok $ exit ``` -### Running a Python File +#### Running a Python File For more complex use cases, a Python file can also be run directly from within the shell. @@ -146,10 +152,11 @@ $ python ./demo.py $ exit ``` -### Running `pytest` in the Shell +#### Running `pytest` in the Shell -If you are running multiple test cases, it may make sense to run a `pytest` test -instead. +If you are running multiple test cases, it may make sense to run a `pytest` +suite instead. Jumpstarter provides a built-in testing library called +`jumpstarter_testing` which provides the `JumpstarterTest` fixture. ```python # mytest.py @@ -172,7 +179,7 @@ $ pytest ./mytest.py $ exit ``` -## Exiting the Exporter Shell +### Exiting the Exporter Shell Once you are done with the exporter, simply exit the exporter shell and the local exporter will be terminated. diff --git a/docs/source/getting-started/setup-remote-client.md b/docs/source/getting-started/setup-remote-client.md deleted file mode 100644 index bf2624711..000000000 --- a/docs/source/getting-started/setup-remote-client.md +++ /dev/null @@ -1 +0,0 @@ -# Setup a Remote Client diff --git a/docs/source/installation/index.md b/docs/source/installation/index.md index 59f5317f9..f80a20609 100644 --- a/docs/source/installation/index.md +++ b/docs/source/installation/index.md @@ -14,6 +14,7 @@ The Jumpstarter components which can be installed are: | [`jumpstarter-cli-admin`](https://github.com/jumpstarter-dev/jumpstarter/tree/main/packages/jumpstarter-cli-admin) | The Jumpstarter admin CLI (`jmp-admin`). This CLI can be used to install the Jumpstarter controller, manage client/exporter registrations, and monitor/control leases. | | [`jumpstarter-cli-client`](https://github.com/jumpstarter-dev/jumpstarter/tree/main/packages/jumpstarter-cli-client) | The Jumpstarter client CLI (`jmp-client`/`j`). This CLI can be used to manage local client configs, start a lease on an exporter, and enter an interactive client shell. | | [`jumpstarter-driver-*`](https://github.com/jumpstarter-dev/jumpstarter/tree/main/packages) | All community and official driver packages that are distributed as part of Jumpstarter are prefixed with `jumpstarter-driver-*`. This includes drivers for PySerial, SD Wire, HTTP, CAN, and more. Driver packages only need to be installed on the exporter/client if they are used by your testing environment. All drivers are optional. | +| [`jumpstarter-testing`](https://github.com/jumpstarter-dev/jumpstarter/tree/main/packages/jumpstarter-testing) | Testing tools for writing Jumpstarter-powered tests with `pytest`. | ```{toctree} From c111f3d4f12c50df8d03e7394f88151d6c500b4a Mon Sep 17 00:00:00 2001 From: Kirk Brauer Date: Thu, 6 Feb 2025 13:59:57 -0500 Subject: [PATCH 4/5] Update CLI getting started --- docs/source/cli/getting-started.md | 31 +++++++++++++----------------- 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/docs/source/cli/getting-started.md b/docs/source/cli/getting-started.md index e869cc6a4..77783d726 100644 --- a/docs/source/cli/getting-started.md +++ b/docs/source/cli/getting-started.md @@ -1,26 +1,21 @@ # Getting Started -Jumpstarter provides the following command line tools: +To simplify the management and operation of Jumpstarter, we provide several command-line tools for different use cases. These CLI tools can also be installed together through the `jmpstarter-cli` package and accessed using the `jmp` command for simplicity. -An **admin CLI** tool called `jmpctl`, which is useful for the administration of exporters and -clients in the distributed service. Installation instructions can be found [here](../installation/service-cli.md). +Each tool can also be installed separately for users who want to reduce the dependencies on their developer machine or an embedded exporter with limited available resources. -A **client CLI** called `jmp`, you can use it to interact with your connected -hardware, request leases, write tests, and develop custom drivers for your hardware. +## Admin CLI `jmp-admin` -A **exporter CLI** called `jmp-exporter`, you can use it to manage local exporter -configurations, and run transient or persistent exporter instances. +The `jmp-admin` or `jmp admin` CLI allows administration of exporters and clients +in a Kubernetes cluster. To use this CLI, you must have a valid `kubeconfig` and +access to the cluter/namespace where the Jumpstarter controller resides. -The `jmp` and `jmp-exporter` CLI tools are available as part of the `jumpstarter` Python package. -You can learn how to install this package [here](../installation/python-package.md), -but it is also available as a container image -```{code-block} -:substitutions: -quay.io/jumpstarter-dev/jumpstarter:{{version}} -``` +## Client CLI `jmp-client` -To check if you have the client CLI installed, run: +The `jmp-client` or `jmp client` CLI allows interaction with Jumpstarter clients +and the management of client configs. -```bash -$ jmp version -``` +## Exporter CLI `jmp-exporter` + +The `jmp-exporter` or `jmp exporter` CLI allows you to run Jumpstarter exporters +and management of exporter configs. From eb1e117b933a54c5f662f4aa3f247b32f1884815 Mon Sep 17 00:00:00 2001 From: Nick Cao Date: Thu, 6 Feb 2025 14:37:06 -0500 Subject: [PATCH 5/5] Update docs/source/introduction/drivers.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --- docs/source/introduction/drivers.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/introduction/drivers.md b/docs/source/introduction/drivers.md index fbdd45580..e8860b52b 100644 --- a/docs/source/introduction/drivers.md +++ b/docs/source/introduction/drivers.md @@ -89,7 +89,7 @@ MyHarness # Custom composite driver for the entire target device harness The `jumpstarter_driver_composite` package provides the base `Composite` and `CompositeClient` classes that can be used to build custom composite drivers. -A composite driver can also be used to orchestrate multiple interfaces to preform +A composite driver can also be used to orchestrate multiple interfaces to perform a specific task such as flashing a system image or entering a debug mode. While -the driver may internally preform a complex task, a simple, user-friendly interface +the driver may internally perform a complex task, a simple, user-friendly interface can be provided to the driver clients.