Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 38 additions & 30 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,82 @@
# Contributing to ScriptBee

Welcome! We are excited that you want to contribute to ScriptBee. This document outlines the guidelines and best practices for contributing to the project.
Welcome! We are excited that you want to contribute to ScriptBee. This document outlines the guidelines and best
practices for contributing to the project.

## How to Contribute

### Reporting Bugs
If you find a bug, please check if an issue already exists. If not, open a new issue using the **Bug Report** template for the relevant component (WebApp, Client, Plugins).

If you find a bug, please check if an issue already exists. If not, open a new issue using the **Bug Report** template
for the relevant component (WebApp, Client, Plugins).

### Suggesting Enhancements
We welcome ideas for new features or improvements. Please open an issue using the **Feature Request** template to discuss your ideas with the maintainers.

We welcome ideas for new features or improvements. Please open an issue using the **Feature Request** template to
discuss your ideas with the maintainers.

### Pull Requests

1. **Fork the repository** and create your branch from `main`.
2. **Implement changes** following the coding standards described below.
3. **Ensure tests pass** and add new tests if necessary.
4. **Write conventional commit messages** (see below).
5. **Submit a Pull Request** with a clear description of the changes and link any related issues.

## Project Structure
## Repository Structure

- **ScriptBeeWebApp**: The backend server built with C# and .NET.
- **ScriptBeeClient**: The frontend application built with Angular and TypeScript.
- **Plugins**: Community and core plugins for ScriptBee.
- **DxWorks.ScriptBee.Plugin.Api**: The API used for building plugins.
- **docs**: Documentation built with MkDocs.
To help you navigate the project, here is a breakdown of the key directories:

- **`DxWorks.ScriptBee.Plugin.Api/`**: The core library containing the interfaces required for developing ScriptBee
plugins (e.g., `IModelLoader`, `IModelLinker`, `IScriptRunner`).
- **`Plugins/`**: Contains the default plugin implementations and helper functions.
- `HelperFunctions/`: Default helper functions for script execution.
- `ScriptRunner/`: Default runners for C#, Javascript, and Python scripts.
- **`ScriptBeeClient/`**: The Angular-based frontend application (UI).
- **`ScriptBeeWebApp/`**: The main backend service implemented in .NET. It includes:
- `src/Gateway`: Orchestration, project management, and instance allocation.
- `src/Analysis`: Core logic for model loading and script execution.
- `src/Workspace`: Model persistence and management.
- **`docs/`**: Documentation source files, built with **VitePress**.
- **`scripts/`**: Utility scripts for building, packing, and maintaining the project.

## Coding Standards

### Backend (C#)

- Follow the rules defined in [.editorconfig](../.editorconfig).
- We use **CSharpier** for automatic code formatting.
- Ensure all public APIs are documented.

### Frontend (TypeScript / Angular)

- We use **Prettier** for formatting.
- We use **ESLint** for linting.
- Components should follow the Angular Style Guide.

### Documentation
- Documentation is located in the [docs](../docs) directory and configured via `mkdocs.yml`.
- Write clear, concise, and professional English.

- Documentation is located in the [docs](../docs) directory and built with **VitePress**.
- Run the documentation locally:
```bash
cd docs
npm install
npm run docs:dev
```

## Commit Messages

We use **[Conventional Commits](https://www.conventionalcommits.org/)** for all commit messages. This keeps the Git history clean and enables automated tooling.
We use **[Conventional Commits](https://www.conventionalcommits.org/)** for all commit messages.

**Format:** `<type>(<scope>): <description>`

Common types:

| Type | When to use |
|------|-------------|
| `feat` | A new feature |
| `fix` | A bug fix |
| `docs` | Documentation changes only |
| `refactor` | Code change that is not a fix or feature |
| `test` | Adding or updating tests |
| `chore` | Build process, tooling, dependencies |

**Examples:**
```
feat: add dark mode toggle
fix: resolve project loading race condition
docs: update plugin API reference
```
Common types: `feat`, `fix`, `docs`, `refactor`, `test`, `chore`.

## Development Setup

- **Backend**: Requires .NET 10 or higher. Open `ScriptBee.slnx` or the individual project folders.
- **Frontend**: Requires Node.js and npm. Navigate to `ScriptBeeClient` and run `npm install`.
- **Documentation**: Requires Python and MkDocs. Run `pip install -r docs/requirements.txt` (if applicable) or `mkdocs serve`.
- **Docker**: Many components rely on Docker for container orchestration during development (e.g., running MongoDB or
Analysis instances).

Thank you for your contributions!
80 changes: 58 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,77 @@
# ScriptBee
# ScriptBee

<p align="center">
<img src="docs/public/assets/logo.png" alt="ScriptBee Logo" width="200"/>
</p>

## What is ScriptBee?

ScriptBee is a tool that helps the analysis of different models. Different tools create data and with the help of
loaders, ScriptBee can load the data and create a model. The model can then be analyzed by running different scripts
written in C#, Javascript and Python on it.

## Documentation
## Quick Start

The fastest way to get started with ScriptBee is using Docker Compose.

The full documentation can be found [here](https://dxworks.org/scriptbee/).
### Run with Docker

### Run locally MkDocs
To get ScriptBee running quickly, create a `docker-compose.yaml` file with the following content and run
`docker compose up -d`:

```bash
docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material
```yaml
version: "3.8"
services:
mongo:
image: mongo:4.4
container_name: mongo
restart: unless-stopped
volumes:
- mongodb_data:/data/db

scriptbee:
image: dxworks/scriptbee
ports:
- "4201:80"
volumes:
- scriptbee_data:/root/.scriptbee
environment:
- ConnectionStrings__mongodb=mongodb://mongo:27017/ScriptBee?authSource=admin
- UserFolder__UserFolderPath=/root/.scriptbee
depends_on:
- mongo

volumes:
mongodb_data:
scriptbee_data:
```

## How to run
### Accessing the UI

### Run
Once the containers are running, open your browser and navigate to **[http://localhost:4201](http://localhost:4201)**.

To run ScriptBee simply run the following command:
---

```bash
docker-compose up
```
## Configuration

For local development, run the following command:
ScriptBee can be configured using environment variables. Below are the most important settings:

```sh
docker-compose -f docker-compose-dev.yaml up
```
| Variable | Description | Default |
|------------------------------|------------------------------------------------------------------|----------------------------|
| `ConnectionStrings__mongodb` | Connection string for the MongoDB instance used for persistence. | `mongodb://mongo:27017...` |
| `UserFolder__UserFolderPath` | Host path for storing project data and shared files. | |
| `SCRIPTBEE_ANALYSIS__DRIVER` | How analysis instances are managed: `docker` or `kubernetes`. | `docker` |

For more detailed information on advanced configuration, check the *
*[Deployment Features](docs/architecture/features.md)** documentation.

---

## Documentation

Full documentation is available at [https://dxworks.org/scriptbee/](https://dxworks.org/scriptbee/).

## Project structure
## Contributing

- DxWorks.ScriptBee.Plugin.Api - the project with the plugin interface
- Plugins - folder with the default bundle plugin
- Domain - main business logic
- Driving - Web Application
- Driven - Database adapter
For information on the repository structure, development setup, and how to contribute, please refer to our *
*[Contributing Guide](.github/CONTRIBUTING.md)**.
2 changes: 0 additions & 2 deletions docs/.vitepress/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ export default defineConfig({
items: [
{ text: 'Introduction', link: '/' },
{ text: 'Installation', link: '/home/installation' },
{ text: 'Run', link: '/home/run' },
],
},
{
Expand Down Expand Up @@ -81,7 +80,6 @@ export default defineConfig({
items: [
{ text: 'Diagram', link: '/architecture/diagram' },
{ text: 'Overview', link: '/architecture/overview' },
{ text: 'Features', link: '/architecture/features' },
{ text: 'RBAC', link: '/architecture/rbac' },
],
},
Expand Down
2 changes: 0 additions & 2 deletions docs/architecture/diagram.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# Architecture Diagram

> Work in Progress

![architecture_diagram](../assets/ScriptBee_Architecture.png 'Architecture Diagram')
33 changes: 0 additions & 33 deletions docs/architecture/features.md

This file was deleted.

61 changes: 32 additions & 29 deletions docs/architecture/overview.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,57 @@
# Overview
# Overview

> Work in Progress
This section provides a high-level overview of ScriptBee's architecture and the roles of its core components.

## UI

The UI uses plugins defined by the gateway.
The User Interface provides a web-based experience for managing projects, loading data from various sources, and running analysis scripts.

The plugins can be of several types
## Gateway

### Output Type plugin
The Gateway is the central orchestration component. Its responsibilities include:

To offer different visualization of the data
- Handling UI interactions and requests.
- Managing projects and user workspaces.
- Orchestrating the lifecycle of analysis instances.

## Gateway
> [!NOTE]
> **Authentication & Authorization** are handled by a separate, external service. The Gateway integrates with this service to manage project access.

## Analysis

Is responsible for the interaction with UI, user authentication and authorization, project management, communicating
with the calculation instances
Analysis is responsible for the core logic of processing models and executing scripts. It performs the following functions:

### Persistence
- **Data Loading**: Ingesting data from various sources using loader plugins.
- **Model Linking**: Connecting loaded models using linker plugins to create a unified data structure.
- **Script Execution**: Running C#, Javascript, or Python scripts against the linked models.

Data is stored in MongoDB
Analysis can be performed in two modes:

## Calculation
- **Permanent Instances**: Manually managed by the user for long-running exploration and interactive analysis.
- **Temporary Instances**: Automatically created for one-time processing, ideal for automated pipelines or quick checks.

Is responsible for loading data, linking the models and executing scripts to obtain different results
## Persistence

Calculation can be done in 2 types of instances:
Both the **Gateway** and **Analysis** services rely on a MongoDB database for data persistence:

- Permanent
- Temporary
- **Gateway**: Stores project configurations, workspace metadata, and instance statuses.
- **Analysis**: Uses the database for temporary data storage during processing and for persisting analysis results.

A permanent instance is allocated and deallocated by the user on command. The purpose is to have a long-running instance
with the linked model to run different types of analysis
## Deployment & Orchestration

A temporary instance is used for one time processing of the models. The model is loaded and linked, a set of predefined
scripts is run to perform the analysis. After the calculations are done, the instance is deallocated. It is used for
CI/CD analysis
The orchestration of analysis components varies based on the environment:

### Persistence
### Docker

Data is stored in MongoDB
In standard containerized environments, a **Docker Executor** manages analysis instances dynamically. It handles:

## Deployment
- **Networking**: Ensuring the Gateway can communicate with the analysis instance.
- **Data Access**: Mapping local data folders to the analysis container so loaders and scripts can access source files.

See [Features](features.md) for more information about the deployment
See [Docker Compose](../home/installation.md#docker-compose) for more information about deployment to Docker

### Kubernetes

The deployment in Kubernetes is done deploying the calculation engine via a CRD

### Docker
In Kubernetes deployments, analysis is managed through specialized controllers that handle the lifecycle of each instance as a native resource.

The deployment is done using Docker API
See [Kubernetes](../home/installation.md#) for more information about deployment to Kubernetes
Loading