Key concepts

Note to readers: This documentation is still in its early stages and may be (in fact, it is) incomplete.

Key Concepts

Bluish provides several key concepts to define and manage automation workflows, offering a flexible framework for organizing development, testing, and deployment tasks. Below are the fundamental components for effectively using Bluish.

Workflows, Jobs, and Steps

These are the essential building blocks in Bluish for defining what actions should be performed:

Workflows

• A Workflow is the highest-level entity in Bluish. It is a collection of one or more Jobs that define a complete automated process, such as a CI/CD pipeline. Workflows encompass everything required for automation, from code building to deployment.
• Workflows run locally or within Docker environments, allowing developers to control the development and deployment process without relying on cloud services.
• A Workflow consists of a series of Jobs that can run independently or in a specific sequence, depending on the defined relationships between Jobs (e.g., using depends_on).

Jobs

• A Job is a collection of Steps designed to achieve a related set of tasks, such as building a project, running tests, or deploying an application.
• Jobs are independent units that can include conditional logic, define dependencies, and control specific execution environments.

Steps

• A Step represents a single action or command, such as copying a file, building a Docker image, restarting a service, or running a script.
• Steps are the most granular unit of work and are defined within a Job. They execute sequentially within a Job, one after another.
• Steps can execute commands directly, use predefined actions, or manipulate values such as environment variables.

The relationship between these elements can be understood through the following YAML example:

# yaml root = workflow scope

jobs:
  job_1:  # job scope
    steps:
      - name: Step 1  # step scope
        run: echo "Step 1 execution"

      - name: Step 2
        run: echo "Step 2 execution"

  job_2:
    depends_on: [job_1]
    steps:
      - name: Step 3
        run: echo "Step 3 execution after job_1"

In the example above:

• Workflow is the root level that includes multiple Jobs.
• Each Job consists of several Steps that run in sequence.

Values and Variables

Bluish provides a powerful mechanism to use and manage values that can be referenced, expanded, or modified throughout workflows. Values are similar to variables in traditional programming, allowing dynamic and reusable workflows.

Types of Values

There are three main types of values you can use in your workflows:

1. Variables (var): These are custom values that you can define and modify at the workflow, job, or step level. These variables are ideal for storing configuration details or dynamic values that may change during the workflow's lifecycle.

2. Environment Variables (env): These are values visible to shell commands during execution. Environment variables can be defined globally (at the workflow level) or locally (at the job or step level). They are accessible during step execution and are useful when values are needed directly within scripts.

3. Job and Step Properties: Each job and step has properties that can be accessed or manipulated. These properties include outputs, results, and other metadata about the execution context. For example, you can capture the output of a step and use it as input in subsequent steps.

Scopes and Value Propagation

Bluish variables can be defined at different levels, and each level has a specific scope:

• Workflow Scope: Values defined here are accessible throughout the entire workflow, including all jobs and steps.
• Job Scope: Variables defined at the job level are available to all steps within that job but are not accessible to other jobs.
• Step Scope: Variables defined within a step are only accessible to that specific step.

Fully Qualified Access

Bluish allows you to access values using fully qualified paths. This enables you to reference variables, outputs, or properties from different scopes in a flexible and clear way:

• .<attribute>: Accesses <attribute> for the current scope.
• job.<attribute>: Accesses <attribute> from the current job when inside a step.
• [job].steps.<step_id>.<attribute>: Accesses <attribute> of a specific step within the current job.
• [workflow].jobs.<job_id>.steps.<step_id>.<attribute>: Accesses <attribute> of a specific step in another job.
• workflow.<attribute>: Accesses <attribute> at the workflow level from any scope.

var and env values

• var values: These are flexible storage values that can be accessed and modified throughout the workflow. They are ideal for storing general information, such as configuration parameters or results from previous steps.
• env values: These values are passed to the shell and are accessible during command execution. They are particularly useful when working with scripts that expect specific environment settings.

General Tip: Use var for general-purpose variables and env for values that need to be injected into command execution environments.

Examples of values usage

Example 1: Using var values

In this example, var values are defined and used within a command. Bluish expands these values before executing the command.

var:
  USER: Luis
  OS: Linux

jobs:
  my_job:
    steps:
      - run: |
          echo "${{ var.USER }} runs ${{ var.OS }}"

The output will be: "Luis runs Linux", as Bluish handles the expansion of var values before executing the command.

Example 2: Using env values

Environment values are passed directly to the shell and are expanded during command execution by the shell itself.

env:
  USER: Luis
  OS: Linux

jobs:
  my_job:
    steps:
      - run: |
          echo "$USER runs $OS"

In this case, the output will also be: "Luis runs Linux", but the expansion occurs in the shell's context rather than directly by Bluish.

Values shadowing and scope overriding

Values can be overridden within narrower scopes, known as shadowing:

env:
  USER: Luis

jobs:
  my_job:
    steps:
      - run: |
          echo "$USER runs Linux"
        env:
          USER: Nobody  # Overrides the global value for this step

      - run: |
          echo "$USER runs Linux"

• The first step will output: "Nobody runs Linux" because USER is overridden locally within the step.
• The second step will output: "Luis runs Linux" as the override does not persist beyond the specific step.

Outputs and Capturing Values

Bluish allows capturing the outputs of commands, which can then be used in subsequent steps or jobs. Outputs provide a way to dynamically pass information throughout the workflow.

• Step Results: Each step has set of attributes that represent the outcome of the last executed step:

  • returncode: The exit code of the command (typically 0 for success, non-zero for errors).
  • stdout: The standard output produced by the command.
  • stderr: The standard error produced by the command.

  For jobs, the result attributes reflect the outcome of the final step within the job. For workflows, it represents the overall outcome of the final job executed.

• Step Outputs (outputs): These are values captured during the execution of a step. This is useful for extracting results from commands, such as file paths, API responses, or status messages, and using them later in the workflow.

Capturing and using outputs

Consider a scenario where you need to pass information between steps:

jobs:
  capture_output:
    steps:
      - id: step_1
        run: echo "Deployment successful!"

      - id: step_2
        run: echo "Result from Step 1: ${{ jobs.capture_output.steps.step_1.result }}"

In this example:

• Step 1 echoes a message.
• Step 2 captures the output of Step 1 and reuses it, displaying: "Result from Step 1: Deployment successful!".

Conditional Logic Using Outputs

You can also use outputs to make decisions within your workflow:

jobs:
  decision_based_job:
    steps:
      - id: check_status
        run: echo "success"
        set: workflow.var.STATUS: ${{ .result }}
        
      - run: |
          if [ "${{ workflow.var.STATUS }}" = "success" ]; then
            echo "Proceeding with deployment"
          else
            echo "Deployment halted"
          fi

This approach allows you to add conditional logic to your workflows based on the outputs of previous steps.