AI-Driven Open-Source Video IoT and Intelligent Stream Management Platform
简体中文 | English
Quick Start • Key Features • System Screenshots • Application Scenarios • Architecture • Technology Stack • Deployment • Help
Important
Online environment: https://vlstream.oortcloudsmart.com:2443/bus/vls-ui/login
Default account: admin / Codex@123456
This is the current online environment. Change the default password immediately after the first sign-in.
VLStream Cloud is an open-source Video IoT platform for device and stream management, intelligent video analysis, algorithm lifecycle management, monitoring, and alerting. It combines a Vue-based management console with a Spring Boot multi-module backend and provides workflow, permission, scheduling, object storage, and operational support for enterprise video applications.
Important
Connect only devices and video streams that you are authorized to access. Make sure your deployment and use of intelligent analysis comply with applicable privacy, security, and data-protection requirements.
| Feature | Description |
|---|---|
| Video Device Management | Device registration, grouping, tagging, health monitoring, connection tests, PTZ control, and stream discovery |
| Multi-Protocol Playback | Web video playback and low-latency streaming capabilities for common Video IoT scenarios |
| Intelligent Analysis | Analysis requests, real-time task monitoring, result management, and event governance |
| Algorithm Lifecycle | Algorithm warehouse, training tasks, annotations, model management, Hi3519DV500 OM conversion, and device deployment |
| Workflow Automation | Flowable-based process definition, deployment, tasks, and approval workflows |
| Enterprise Permissions | Sa-Token authentication, RBAC, data permissions, user management, and role management |
| Platform Services | Scheduled jobs, object storage, SMS integration, monitoring, and XXL-Job support |
| Visual Operations | Vue 3 management console with dashboards, GIS views, reusable CRUD components, and video layouts |
Algorithm training supports an exclusive single-GPU queue on one physical GPU server. A Docker container is created when a training job starts. Jobs wait automatically while the GPU is busy, and the container is removed when training finishes while job records, logs, and model artifacts are retained. See Single-Node GPU Training Scheduler.
VLStream delivers trained models to devices through MQTT. Hardware connection, model delivery, event reporting, media upload, status receipts, and integration acceptance follow the VLS Platform and Camera Unified Communication Protocol.
Configure these environment variables:
VLSTREAM_MQTT_HOST=127.0.0.1
VLSTREAM_MQTT_PORT=1883
VLSTREAM_MQTT_USERNAME=vlstream
VLSTREAM_MQTT_PASSWORD=replace-me
VLSTREAM_MODEL_PUBLIC_BASE_URL=https://vlstream.example.com
VLSTREAM_MODEL_DOWNLOAD_SIGNING_SECRET=replace-with-a-long-random-secretVLSTREAM_MODEL_PUBLIC_BASE_URL must be the backend address reachable by the
devices, not the browser-facing frontend address. Model download URLs use
short-lived HMAC signatures. Generate and inject a unique random signing secret
for each environment; never commit the real secret to Git.
 Active Safety Event Management |
 Event Feedback & Workflow |
 Work Order Management |
 Visual Workflow Designer |
 Algorithm Training Management |
 Algorithm Training Console |
Click any screenshot to view it at full resolution.
 Chemical Production Safety |
 Smart Water Conservancy |
 Wastewater Treatment |
 Smart Construction Site |
 Smart Community |
 Gas Station Safety |
 Smart Kitchen |
 Smart Campus |
 Smart City Management |
| Category | Technology |
|---|---|
| Runtime | Java 8 |
| Framework | Spring Boot 2.7.11, RuoYi-Flowable-Plus 0.8.3 |
| Persistence | MyBatis-Plus 3.5.3.1 |
| Authentication | Sa-Token 1.34.0 |
| Workflow | Flowable 6.8.0 |
| Cache and Locking | Redis, Redisson 3.20.1, Lock4j |
| API Documentation | Springdoc OpenAPI, Knife4j |
| Build | Maven 3.6+ |
| Category | Technology |
|---|---|
| Framework | Vue 3.3, Vue Router 4 |
| Build Tool | Vite 4.4 |
| UI | Element Plus 2.3, Avue 3.7 |
| State Management | Pinia 2.1 |
| Video | hls.js, xgplayer |
| GIS | Leaflet 1.9 |
| HTTP | Axios 1.4 |
VLStream Cloud's core business architecture is organized into three categories:
- Hardware: IPC, BOX, and NVR devices. The lifecycle covers production provisioning, installation and protocol access, platform operations, and device transfer.
- Platform servers: VLS owns AI events, model delivery, and platform business; WVP is the sole video-device center for VLStream and other protocols, device state, and video control; ZLMediaKit provides the media server behind WVP; MQTT, MySQL, Redis, and MinIO provide messaging, persistence, cache, and object storage.
- Client: VLStream-ui provides platform operations, while the WVP UI provides video preview, playback, PTZ, and channel management.
The complete lifecycle sequence diagram and dependency inventory are maintained in Core Business and Technical Architecture.
sequenceDiagram
autonumber
participant P as Production Provisioning
participant H as Hardware<br/>IPC / BOX / NVR
participant C as Client<br/>VLStream-ui / WVP UI
participant V as VLS Server
participant M as MQTT Broker<br/>EMQX
participant W as WVP Server
participant Z as ZLMediaKit
participant D as MySQL / Redis
participant O as MinIO / S3
rect rgb(255, 248, 235)
Note over P,H: 1. Production provisioning
P->>H: Write device ID, secret, MQTT address and base configuration
H->>M: Connect with pre-provisioned identity
M-->>V: Forward device identity and online message
V->>D: Persist identity and status
end
rect rgb(239, 246, 255)
Note over C,H: 2. Initialization, installation and video access
C->>V: Initialize or register device
V->>M: Publish initialization and control configuration
M->>H: MQTT configuration/control message
alt GB28181 / SIP
H->>W: SIP registration, heartbeat and catalog
C->>W: Preview or playback request
W->>H: SIP INVITE / playback control
H->>Z: RTP media
else RTSP / ONVIF
C->>W: Discovery, pull or device control
W->>H: ONVIF / RTSP request
H->>Z: RTSP / RTP media
end
W->>Z: REST API, Hook and stream coordination
Z-->>C: WebRTC / HTTP-FLV / HLS / RTSP playback
end
rect rgb(240, 253, 244)
Note over C,H: 3. Platform operations and hardware interaction
C->>V: Device management, user binding and status query
H->>M: Heartbeat, event, status and model receipt
M-->>V: Forward hardware message
V->>D: Persist business state and event result
C->>V: Send control or model task
V->>M: Publish command or model task
M->>H: MQTT command
H-->>M: Execution receipt
M-->>V: Forward result
V->>O: Store or read event media and model artifacts
end
rect rgb(254, 242, 242)
Note over C,H: 4. Device transfer
C->>V: Unbind or transfer device
V->>M: Clear binding and reset device
M->>H: Reset to pending-binding state
H-->>M: Reset receipt
M-->>V: Forward receipt
V->>D: Clean up user-device relationship
end
The following versions are taken from the current release Compose or project configuration. A version marked not pinned must be fixed in the formal deployment manifest before production release.
| Name | Purpose | Version | License |
|---|---|---|---|
| VLStream Server (VLS) | Device registration, user binding, events, model tasks, and platform APIs | Maven 0.8.3; Spring Boot 2.7.11; release image 1.1.2 |
MIT |
| WVP Server | Required unified video-device center for VLStream, GB28181/SIP, ONVIF, RTSP, preview, playback, PTZ, and video control | 3.8.9; Spring Boot 2.7.18 |
MIT |
| ZLMediaKit | RTP ingest, media management, REST/Hook, and playback output | Not pinned in WVP/VLStream repositories | MIT |
| MQTT Broker / EMQX | Device messaging, heartbeat, events, commands, and model receipts | 5.4; external service in release Compose |
Apache-2.0 |
| MySQL | Business database | 8.4.10-oraclelinux9 |
GPLv2 or commercial license |
| Redis | Cache, sessions, online state, and runtime state | 7.4.9-alpine |
RSALv2 or SSPLv1 |
| MinIO / S3 | Event media, model files, and object storage | RELEASE.2025-09-07T16-13-09Z |
AGPLv3 or commercial license |
Nginx or an equivalent gateway is normally required for frontend static files
and reverse proxying. WebRTC Streamer v0.8.16 is optional for the VLS direct
RTSP-to-WebRTC path; FFmpeg is an optional WVP/ZLMediaKit pull and conversion
helper, not another standalone media platform.
The backend paths in the following table are relative to
VLStream-Cloud-Backend-Server/vls-stream/.
| Layer | Main paths | Responsibility |
|---|---|---|
| Operator client | VLStream-Web/VLStream-ui/ |
Dashboards, device and stream management, AI operations, workflow, and system administration |
| Device client | sdk/ |
Native camera-side RTSP/WebRTC streaming, AI inference, event reporting, and model updates |
| Application services | ruoyi-admin/, ruoyi-vlstream/ |
API entry point and VLStream domain services |
| Platform services | ruoyi-common/, ruoyi-framework/, ruoyi-system/, ruoyi-flowable/, ruoyi-job/, ruoyi-oss/, ruoyi-sms/, ruoyi-extend/ |
Shared infrastructure, authentication, permissions, workflows, jobs, storage, messaging, and monitoring |
| Operations and documentation | deploy/, docs/, backend deploy/ and script/ |
Container deployment, database initialization, migration support, protocols, and operational documentation |
VLStream-Cloud/
├── VLStream-Cloud-Backend-Server/
│ └── vls-stream/ # Java 8 / Spring Boot Maven reactor
│ ├── ruoyi-admin/ # Executable application and REST APIs
│ ├── ruoyi-vlstream/ # Devices, streams, AI, events, and models
│ ├── ruoyi-system/ # Users, roles, permissions, and system services
│ ├── ruoyi-framework/ # Web, security, and framework configuration
│ ├── ruoyi-flowable/ # Workflow and approval services
│ ├── ruoyi-common/ # Shared models, utilities, and base components
│ ├── ruoyi-generator/ # Code generation
│ ├── ruoyi-job/ # Scheduled jobs
│ ├── ruoyi-oss/ # Object storage integration
│ ├── ruoyi-sms/ # SMS integration
│ ├── ruoyi-extend/ # Monitoring and XXL-Job services
│ ├── ruoyi-demo/ # Examples and integration tests
│ ├── deploy/ # Backend deployment resources
│ └── script/ # Database and Docker scripts
├── VLStream-Web/
│ └── VLStream-ui/ # Vue 3 management console
├── sdk/ # Hi3519DV500 native camera business SDK
├── deploy/ # Repository-level deployment assets
├── docs/ # Repository-level documentation
├── assets/ # Screenshots and application imagery
├── tools/ # Development and validation tools
├── LICENSE
├── README.md # English documentation (default)
└── README.zh-CN.md # Simplified Chinese documentation
The sdk/ directory is the camera-side native component, not a Maven or npm
module. It exports the business source used to build the rtsp_streamer
executable for the Hi3519DV500 board and depends on the original HiSilicon
MPP/ACL SDK, the cross toolchain, and an external WebRTC Streamer SDK.
| Area | Contents |
|---|---|
| Media pipeline | src/rtsp_streamer.c, rtsp_lib/ — RTSP input, frame handling, and stream orchestration |
| WebRTC bridge | src/webrtc_bridge.c, include/webrtc_bridge.h — WebRTC lifecycle, sessions, codec headers, and keyframe gating |
| AI runtime | src/ai_bridge.cpp, src/ai_acl_adapter.cpp, src/ai_runtime_config.cpp — ACL inference, OM model validation/hot switching, and runtime configuration |
| Platform integration | src/http_reporter.cpp, src/model_receiver.cpp — asynchronous event/JPEG reporting and HTTP model reception |
| Configuration and examples | config/, examples/ — board settings, class labels, and an MQTT model-dispatch example |
| Dependencies and notes | third_party/, docs/, Makefile — external declarations, porting notes, debugging records, and board build rules |
The SDK is intentionally kept separate from the server build: the root Maven and frontend commands do not compile it. For prerequisites, original project paths, excluded vendor binaries, and board-side build instructions, see the SDK guide.
| Component | Requirement |
|---|---|
| Java | JDK 8 |
| Maven | 3.6+ |
| Database | MySQL 5.7+ |
| Cache | Redis |
| Object Storage | MinIO or another S3-compatible service; required for complete annotation support |
| Messaging | MQTT broker; required for device control and model delivery |
| Training Node | Linux GPU server with SSH/SFTP; required for algorithm training |
| AI Service | apaas-ai routed through an APaaS gateway; required for AI text/image features |
| Frontend | Node.js and npm |
Browsers cannot play RTSP directly. Camera live preview uses WebRTC Streamer to
convert RTSP to WebRTC. The pinned, validated Docker image for this project is
mpromonet/webrtc-streamer:v0.8.16. Keep this exact tag instead of using an
untested latest image or an older Windows binary.
To start it independently on a local machine:
docker run -d --name vlstream-webrtc --restart unless-stopped -p 8000:8000 `
mpromonet/webrtc-streamer:v0.8.16 -H 0.0.0.0:8000 -vvvVerify the runtime with curl.exe http://127.0.0.1:8000/api/version; it should
report v0.8.16/Linux-x86_64. The backend declaration is in
ruoyi-admin/src/main/resources/application.yml:
VLSTREAM_WEBRTC_ENABLED=true
VLSTREAM_WEBRTC_RUNTIME_IMAGE=mpromonet/webrtc-streamer:v0.8.16
VLSTREAM_WEBRTC_INTERNAL_URL=http://127.0.0.1:8000
VLSTREAM_WEBRTC_PUBLIC_URL=/bus/webrtc-streamer-serverThe release Compose deployment uses the same version through
WEBRTC_STREAMER_IMAGE=mpromonet/webrtc-streamer:v0.8.16 in
deploy/release/.env. runtime-image is a backend declaration and status
value only; the backend does not pull or start Docker containers.
git clone https://github.com/OortCloudGroup/VLStream-Cloud.git
cd VLStream-CloudCREATE DATABASE vlstream CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;cd VLStream-Cloud-Backend-Server/vls-stream
mysql -u root -p vlstream --execute="source script/sql/mysql/mysql_ry_v0.8.X.sql"SQL initialization scripts for Oracle, PostgreSQL, and SQL Server are also
available under script/sql/. Application schema upgrades are managed by
Flyway when the backend starts. Add every new database change as a new,
immutable migration under
ruoyi-admin/src/main/resources/db/migration/; do not edit a migration that
has already run. See
DATABASE_MIGRATIONS.md.
Review the main configuration and the active profile configuration:
ruoyi-admin/src/main/resources/application.ymlruoyi-admin/src/main/resources/application-dev.ymlruoyi-admin/src/main/resources/application-prod.yml
The Maven profiles are dev, local, and prod; dev is active by default.
Do not use repository test addresses or example passwords for a complete deployment. Configure at least the following services before startup:
| Configuration | Purpose | Location |
|---|---|---|
| MySQL | Business data, training jobs, and delivery jobs | application-dev.yml / application-prod.yml |
| Redis | Sessions, cache, and distributed state | application-dev.yml / application-prod.yml |
| WVP Server | Required unified video-device center and VLStream device validation | VLSTREAM_WVP_INTERNAL_BASE_URL |
| MinIO | Annotation images, datasets, and file uploads | Database table sys_oss_config |
| GPU training server | Training, conversion, and model artifacts | VLSTREAM_SSH_*, VLSTREAM_TRAINING_* |
| MQTT broker | Device control, model delivery, and receipts | VLSTREAM_MQTT_* |
| Model download entry | Device-side HTTP model download | VLSTREAM_MODEL_* |
| GPT/AI service | AI text and image generation | Frontend APaaS gateway and a separate apaas-ai service |
Inject secrets through the deployment environment and never commit real passwords or keys:
MYSQL_HOST=mysql.example.internal
MYSQL_PORT=3306
MYSQL_DB_NAME=vlstream
MYSQL_USERNAME=vlstream
MYSQL_PASSWORD=replace-me
REDIS_HOST=redis.example.internal
REDIS_PORT=6379
REDIS_PASSWORD=replace-me
# WVP is required; this address must be reachable from the VLS backend
VLSTREAM_WVP_INTERNAL_BASE_URL=http://wvp-server:9080
VLSTREAM_NATIVE_DEVICE_LEGACY_ENABLED=false
VLSTREAM_SSH_HOST=gpu.example.internal
VLSTREAM_SSH_PORT=22
VLSTREAM_SSH_USERNAME=vlstream
VLSTREAM_SSH_PASSWORD=replace-me
VLSTREAM_TRAINING_HOST_DATA_DIR=/data/work
VLSTREAM_TRAINING_WORK_DIR=/data/work/ultralytics_yolov8-main/datasets
VLSTREAM_MQTT_HOST=127.0.0.1
VLSTREAM_MQTT_PORT=1883
VLSTREAM_MQTT_USERNAME=vlstream
VLSTREAM_MQTT_PASSWORD=replace-me
VLSTREAM_MQTT_QOS=1
VLSTREAM_MODEL_PUBLIC_BASE_URL=https://vlstream.example.com
VLSTREAM_MODEL_DOWNLOAD_SIGNING_SECRET=replace-with-a-long-random-secret
VLSTREAM_MODEL_DOWNLOAD_URL_TTL_SECONDS=1800
VLSTREAM_MODEL_DISPATCH_MQTT_CLIENT_ID=vls-model-dispatch-backend-01
VLSTREAM_DEVICE_MEDIA_OSS_CONFIG_KEY=vlstream-events
VLSTREAM_DEVICE_MEDIA_UPLOAD_TTL_SECONDS=600
VLSTREAM_DEVICE_MEDIA_MAX_IMAGE_BYTES=10485760
VLSTREAM_DEVICE_MEDIA_ALLOW_UNAUTHENTICATED=falseWVP owns VLStream device registration, heartbeat, video streams, and firmware
jobs. VLS keeps the existing hardware-facing HTTP and MQTT contracts and calls
WVP internally when issuing media upload URLs or consuming device events. Start
WVP before VLS. Keep VLSTREAM_NATIVE_DEVICE_LEGACY_ENABLED=false; the switch
exists only to roll back to the legacy VLS device-management implementation.
Each backend instance must use a unique
VLSTREAM_MODEL_DISPATCH_MQTT_CLIENT_ID. MQTT topics, ACL rules, and hardware
behavior are defined by
VLS-Protocol.md.
Annotation uploads use the enabled config_key=minio record in
sys_oss_config, not fixed credentials in application.yml. Configure the
access key, secret key, bucket, API endpoint, external domain, HTTPS flag,
access policy, and enabled status. Persist MinIO data and verify that the
backend, browser, and GPU server can all reach the generated object URLs.
For device event images, reuse the MinIO service but configure a separate
private OSS entry and bucket (for example config_key=vlstream-events).
Devices receive only short-lived, single-object presigned PUT URLs and must
never receive MinIO credentials. The unauthenticated upload-grant endpoint is
for LAN development only and must remain disabled in production. Apply
db/2026-07-29-vls-device-event-media.sql before enabling MQTT event ingestion.
The frontend calls apaas-ai through the configured APaaS gateway:
{APaaS gateway prefix}/apaas-ai/api/v1/text_completion
{APaaS gateway prefix}/apaas-ai/api/v1/text_img
Configure the provider base URL, API key, model names, timeout, retries, and
network access in the separate apaas-ai service. That service is not included
in this repository.
mvn -ntp -Pdev clean package
mvn -ntp -Pdev -pl ruoyi-admin spring-boot:runAfter startup:
- Knife4j:
http://localhost:8080/doc.html - Swagger UI:
http://localhost:8080/swagger-ui.html
Note
The backend parent POM references internal Maven repositories. Dependency
resolution may require access to the project network or a compatible mirror in
your Maven settings.xml.
Open a new terminal from the repository root:
cd VLStream-Web/VLStream-ui
npm install
npm run devFor local development, configure:
VITE_DEV_PROXY_TARGET=http://127.0.0.1:8080
VITE_APAAS_PROXY_TARGET=http://apaas-gateway.example.internal:21410Use npm run build to create a production frontend bundle.
- Verify
/actuator/healthand MySQL/Redis connectivity. - Upload an image and open the returned MinIO URL.
- Create an annotation job and save annotation results.
- Verify that an AI text request reaches
apaas-ai. - Complete MQTT and model-delivery checks defined in
VLS-Protocol.md. - Run one training job and verify scheduling, logs, and model artifacts.
| Method | Path | Description |
|---|---|---|
GET |
/vlsDeviceInfo/page |
Query devices with pagination |
GET |
/vlsDeviceInfo/{id} |
Query a device by ID |
POST |
/vlsDeviceInfo |
Add a device |
PUT |
/vlsDeviceInfo/{id} |
Update a device |
DELETE |
/vlsDeviceInfo/{id} |
Delete a device |
GET |
/vlsDeviceInfo/statistics |
Retrieve device statistics |
Standard API responses use the shared R<T> structure:
{
"code": 200,
"msg": "Operation successful",
"data": {}
}Use the generated OpenAPI documentation for the complete and current API list.
Download the deployment package from GitHub Releases, extract it, copy the environment template, and start the bundled services:
Copy-Item .env.example .env
docker compose up -dStop the services with:
docker compose downTip
The package includes MySQL, Redis, MinIO, WebRTC-streamer, the backend, and the frontend. Existing external infrastructure is also supported. See the deployment guide for configuration and upgrade instructions.
| Resource | Link |
|---|---|
| Frontend Guide | VLStream-Web/README.md |
| Frontend Guide (Chinese) | VLStream-Web/README-cn.md |
| Device SDK Guide | sdk/README.md |
| Core Business and Technical Architecture | architecture/vlstream-core-business-technical-architecture.md |
| Backend Environment Variables | ENVIRONMENT_VARIABLES.md |
| Deployment Guide | deploy/release/README.md |
| Database Migrations | DATABASE_MIGRATIONS.md |
| VLS Device Protocol | VLS-Protocol.md |
| VLS Protocol Specification (English) | VLS-Protocol-EN.docx |
| VLS Protocol Specification (Chinese) | VLS-Protocol.docx |
| API Documentation | Start the backend and open Knife4j or Swagger UI |
- Project Homepage: vls.oortcloudsmart.com
- Issue Tracker: GitHub Issues
- Technical Support: zhangxuelian@oortcloudsmart.com
Contributions are welcome. You can report bugs, propose features, improve the documentation, or submit pull requests.
VLStream Cloud is released under the MIT License.
If this project helps you, consider giving it a ⭐ on GitHub.
Project Homepage • Issue Tracker • GitHub Repository
Built with ❤️ by OortCloud