diff --git a/src/.vuepress/public/img/AINode-0-en.png b/src/.vuepress/public/img/AINode-0-en.png new file mode 100644 index 000000000..1b107d75a Binary files /dev/null and b/src/.vuepress/public/img/AINode-0-en.png differ diff --git a/src/.vuepress/public/img/AINode-0.png b/src/.vuepress/public/img/AINode-0.png new file mode 100644 index 000000000..6c72d0615 Binary files /dev/null and b/src/.vuepress/public/img/AINode-0.png differ diff --git a/src/.vuepress/public/img/AINode-State-apache-en.png b/src/.vuepress/public/img/AINode-State-apache-en.png new file mode 100644 index 000000000..3146a6f0d Binary files /dev/null and b/src/.vuepress/public/img/AINode-State-apache-en.png differ diff --git a/src/.vuepress/public/img/AINode-State-apache.png b/src/.vuepress/public/img/AINode-State-apache.png new file mode 100644 index 000000000..2d757b1e4 Binary files /dev/null and b/src/.vuepress/public/img/AINode-State-apache.png differ diff --git a/src/.vuepress/public/img/AINode-State-en.png b/src/.vuepress/public/img/AINode-State-en.png new file mode 100644 index 000000000..f4d8ee700 Binary files /dev/null and b/src/.vuepress/public/img/AINode-State-en.png differ diff --git a/src/.vuepress/public/img/AINode-State.png b/src/.vuepress/public/img/AINode-State.png new file mode 100644 index 000000000..a4a4f7f80 Binary files /dev/null and b/src/.vuepress/public/img/AINode-State.png differ diff --git a/src/.vuepress/public/img/AINode-en.png b/src/.vuepress/public/img/AINode-en.png new file mode 100644 index 000000000..c3252231a Binary files /dev/null and b/src/.vuepress/public/img/AINode-en.png differ diff --git a/src/.vuepress/public/img/AINode-new.png b/src/.vuepress/public/img/AINode-new.png new file mode 100644 index 000000000..3d17e9e8a Binary files /dev/null and b/src/.vuepress/public/img/AINode-new.png differ diff --git a/src/UserGuide/Master/Table/AI-capability/AINode_apache.md b/src/UserGuide/Master/Table/AI-capability/AINode_apache.md index 421bbcde3..b566b72cb 100644 --- a/src/UserGuide/Master/Table/AI-capability/AINode_apache.md +++ b/src/UserGuide/Master/Table/AI-capability/AINode_apache.md @@ -23,15 +23,25 @@ AINode is a native IoTDB node that supports the registration, management, and invocation of time-series-related models. It comes with built-in industry-leading self-developed time-series large models, such as the Timer series developed by Tsinghua University. These models can be invoked through standard SQL statements, enabling real-time inference of time series data at the millisecond level, and supporting application scenarios such as trend forecasting, missing value imputation, and anomaly detection for time series data. +> Available since V2.0.5 + The system architecture is shown below: ::: center - + ::: + The responsibilities of the three nodes are as follows: -- **ConfigNode**: responsible for storing and managing the meta-information of the model; responsible for distributed node management. -- **DataNode**: responsible for receiving and parsing SQL requests from users; responsible for storing time-series data; responsible for preprocessing computation of data. -- **AINode**: responsible for model file import creation and model inference. +- **ConfigNode:** + - Manages distributed nodes and handles load balancing across the system. +- **DataNode:** + - Receives and parses user SQL queries. + - Stores time-series data. + - Performs preprocessing computations on raw data. +- **AINode:** + - Manages and utilizes time-series models (including training/inference). + - Supports deep learning and machine learning workflows. + ## 1. Advantageous features @@ -55,11 +65,11 @@ Compared with building a machine learning service alone, it has the following ad - **Create**: Load externally designed or trained model files/algorithms into AINode for unified management and usage by IoTDB. - **Inference**: Use the created model to complete time series analysis tasks applicable to the model on specified time series data. - **Built-in Capabilities**: AINode comes with machine learning algorithms or self-developed models for common time series analysis scenarios (e.g., forecasting and anomaly detection). -![](/img/h3.png) +![](/img/AINode-en.png) ## 3. Installation and Deployment -The deployment of AINode can be found in the document [Deployment Guidelines](../Deployment-and-Maintenance/AINode_Deployment_apache.md#ainode-deployment) . +The deployment of AINode can be found in the document [AINode Deployment](../Deployment-and-Maintenance/AINode_Deployment_apache.md). ## 4. Usage Guide @@ -80,13 +90,13 @@ Models that meet the following criteria can be registered with AINode: The SQL syntax for model registration is defined as follows: ```SQL -create model using uri +create model using uri ``` Detailed meanings of SQL parameters: -- **model_name**: The global unique identifier for the model, non-repeating. Model names have the following constraints: - - Allowed characters: [0-9 a-z A-Z _] (letters, numbers, underscores) +- **model_id**: The global unique identifier for the model, non-repeating. Model names have the following constraints: + - Allowed characters: [0-9 a-z A-Z _] (letters, digits (not at the beginning), underscores (not at the beginning)) - Length: 2-64 characters - Case-sensitive - **uri**: The resource path of the model registration files, which should include the **model structure and weight file `model.pt` and the model configuration file `config.yaml`** @@ -117,7 +127,7 @@ In addition to registering local model files, remote resource paths can be speci #### Example -The current example folder contains model.pt (trained model) and config.yaml with the following content: +The [example folder](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example) contains model.pt (trained model) and config.yaml with the following content: ```YAML configs: @@ -157,41 +167,47 @@ Registered models can be queried using the `show models` command. The SQL defini ```SQL show models -show models +show models ``` In addition to displaying all models, specifying a `model_id` shows details of a specific model. The display includes: -| **ModelId** | **State** | **Configs** | **Attributes** | -| ----------- | --------------------------------------------------------- | ------------------------------------------------ | -------------- | -| Unique ID | Registration status (INACTIVE, LOADING, ACTIVE, DROPPING) | InputShape, outputShape, inputTypes, outputTypes | User notes | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|---------------|----------------|-------------| +| Model ID | Model Type | Model Category | Model State | -**State descriptions:** +- Model State Transition Diagram -- **INACTIVE**: Model is unavailable. -- **LOADING**: Model is being loaded. -- **ACTIVE**: Model is available. -- **DROPPING**: Model is being deleted. +![](/img/AINode-State-apache-en.png) -#### Example +**Instructions:** + +1. Initialization: + - When AINode starts, show models only displays BUILT-IN models. +2. Custom Model Import: + - Users can import custom models (marked as USER-DEFINED). + - The system attempts to parse the ModelTypefrom the config file. + - If parsing fails, the field remains empty. +3. Foundation Model Weights: + - Time-series foundation model weights are not bundled with AINode. + - AINode automatically downloads them during startup. + - Download state: LOADING. +4. Download Outcomes: + - Success → State changes to ACTIVE. + - Failure → State changes to INACTIVE. + +**Example** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 Deleting Models @@ -243,10 +259,10 @@ Parameter descriptions: 1. The `forecast` function predicts all columns in the input table by default (excluding the time column and columns specified in `partition by`). 2. The `forecast` function does not require the input data to be in any specific order. It sorts the input data in ascending order by the timestamp (specified by the `TIMECOL` parameter) before invoking the model for prediction. -3. Different models have different requirements for the number of input data rows: - - If the input data has fewer rows than the minimum requirement, an error will be thrown. - - If the input data exceeds the maximum row limit, the last rows that meet the requirement will be automatically truncated for processing. - - Among the currently built-in models in AINode, only `sundial` has a row limit. It supports a maximum of 2880 input rows. If exceeded, the last 2880 rows will be automatically used. +3. Different models have varying requirements for the number of input data rows. If the input data has fewer rows than the minimum requirement, an error will be reported. + - Among the current built-in models in AINode: + - Timer-XL requires at least 96 rows of input data. + - Timer-Sundial requires at least 16 rows of input data. 4. The result columns of the `forecast` function include all input columns from the input table, with their original data types preserved. If `preserve_input = true`, an additional `is_input` column will be included to indicate whether a row is from the input data. - Currently, only columns of type INT32, INT64, FLOAT, or DOUBLE are supported for prediction. Otherwise, an error will occur: "The type of the column [%s] is [%s], only INT32, INT64, FLOAT, DOUBLE is allowed." 5. `output_start_time` and `output_interval` only affect the generation of the timestamp column in the output results. Both are optional parameters: @@ -254,7 +270,13 @@ Parameter descriptions: - `output_interval` defaults to the sampling interval of the input data, calculated as: (last timestamp - first timestamp) / (number of rows - 1). - The timestamp of the Nth output row is calculated as: `output_start_time + (N - 1) * output_interval`. -#### Example +**Example: Database and table must be pre-created** + +```sql +create database etth +create table eg (hufl FLOAT FIELD, hull FLOAT FIELD, mufl FLOAT FIELD, mull FLOAT FIELD, lufl FLOAT FIELD, lull FLOAT FIELD, ot FLOAT FIELD) +``` + Using the ETTh1-tab dataset:[ETTh1-tab](/img/ETTh1-tab.csv)。 @@ -314,7 +336,7 @@ It costs 1.615s ### 4.5 Time Series Large Model Import Steps -AINode supports multiple time series large models. For deployment, refer to [Time Series Large Model](https://timecho.com/docs/zh/UserGuide/latest/AI-capability/TimeSeries-Large-Model.html) +AINode supports multiple time series large models. For deployment, refer to [Time Series Large Model](../AI-capability/TimeSeries-Large-Model.md) ## 5 Permission Management diff --git a/src/UserGuide/Master/Table/AI-capability/AINode_timecho.md b/src/UserGuide/Master/Table/AI-capability/AINode_timecho.md index a22f96bf8..4c4f3ca9a 100644 --- a/src/UserGuide/Master/Table/AI-capability/AINode_timecho.md +++ b/src/UserGuide/Master/Table/AI-capability/AINode_timecho.md @@ -23,15 +23,24 @@ AINode is a native IoTDB node that supports the registration, management, and invocation of time-series-related models. It comes with built-in industry-leading self-developed time-series large models, such as the Timer series developed by Tsinghua University. These models can be invoked through standard SQL statements, enabling real-time inference of time series data at the millisecond level, and supporting application scenarios such as trend forecasting, missing value imputation, and anomaly detection for time series data. +> Available since V2.0.5.1 + The system architecture is shown below: ::: center - + ::: + The responsibilities of the three nodes are as follows: -- **ConfigNode**: responsible for storing and managing the meta-information of the model; responsible for distributed node management. -- **DataNode**: responsible for receiving and parsing SQL requests from users; responsible for storing time-series data; responsible for preprocessing computation of data. -- **AINode**: responsible for model file import creation and model inference. +- **ConfigNode:** + - Manages distributed nodes and handles load balancing across the system. +- **DataNode:** + - Receives and parses user SQL queries. + - Stores time-series data. + - Performs preprocessing computations on raw data. +- **AINode:** + - Manages and utilizes time-series models (including training/inference). + - Supports deep learning and machine learning workflows. ## 1. Advantageous features @@ -55,11 +64,11 @@ Compared with building a machine learning service alone, it has the following ad - **Create**: Load externally designed or trained model files/algorithms into AINode for unified management and usage by IoTDB. - **Inference**: Use the created model to complete time series analysis tasks applicable to the model on specified time series data. - **Built-in Capabilities**: AINode comes with machine learning algorithms or self-developed models for common time series analysis scenarios (e.g., forecasting and anomaly detection). -![](/img/h3.png) +![](/img/AINode-en.png) ## 3. Installation and Deployment -The deployment of AINode can be found in the document [Deployment Guidelines](../Deployment-and-Maintenance/AINode_Deployment_timecho.md#ainode-deployment) . +The deployment of AINode can be found in the document [AINode Deployment](../Deployment-and-Maintenance/AINode_Deployment_timecho.md) . ## 4. Usage Guide @@ -80,13 +89,13 @@ Models that meet the following criteria can be registered with AINode: The SQL syntax for model registration is defined as follows: ```SQL -create model using uri +create model using uri ``` Detailed meanings of SQL parameters: -- **model_name**: The global unique identifier for the model, non-repeating. Model names have the following constraints: - - Allowed characters: [0-9 a-z A-Z _] (letters, numbers, underscores) +- **model_id**: The global unique identifier for the model, non-repeating. Model names have the following constraints: + - Allowed characters: [0-9 a-z A-Z _] (letters, digits (not at the beginning), underscores (not at the beginning)) - Length: 2-64 characters - Case-sensitive - **uri**: The resource path of the model registration files, which should include the **model structure and weight file `model.pt` and the model configuration file `config.yaml`** @@ -117,7 +126,7 @@ In addition to registering local model files, remote resource paths can be speci #### Example -The current example folder contains model.pt (trained model) and config.yaml with the following content: +The [example folder](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example) contains model.pt (trained model) and config.yaml with the following content: ```YAML configs: @@ -157,45 +166,59 @@ Registered models can be queried using the `show models` command. The SQL defini ```SQL show models -show models +show models ``` In addition to displaying all models, specifying a `model_id` shows details of a specific model. The display includes: -| **ModelId** | **State** | **Configs** | **Attributes** | -| ----------- | --------------------------------------------------------- | ------------------------------------------------ | -------------- | -| Unique ID | Registration status (INACTIVE, LOADING, ACTIVE,TRAINING,FAILED, DROPPING) | InputShape, outputShape, inputTypes, outputTypes | User notes | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|---------------|----------------|-------------| +| Model ID | Model Type | Model Category | Model State | -**State descriptions:** +- Model State Transition Diagram -- **INACTIVE**: The model is in an unavailable state. -- **LOADING**: The model is being loaded. -- **ACTIVE**: The model is in an available state. -- **TRAINING**: The model is in the fine-tuning state. -- **FAILED**: The model fine-tuning failed. -- **DROPPING**: The model is being deleted. +![](/img/AINode-State-en.png) -#### Example +**Instructions:** + +1. Initialization: + - When AINode starts, show models only displays BUILT-IN models. +2. Custom Model Import: + - Users can import custom models (marked as USER-DEFINED). + - The system attempts to parse the ModelTypefrom the config file. + - If parsing fails, the field remains empty. +3. Foundation Model Weights: + - Time-series foundation model weights are not bundled with AINode. + - AINode automatically downloads them during startup. + - Download state: LOADING. +4. Download Outcomes: + - Success → State changes to ACTIVE. + - Failure → State changes to INACTIVE. +5. Fine-Tuning Process: + - When fine-tuning starts: State becomes TRAINING. + - Successful training → State transitions to ACTIVE. + - Training failure → State changes to FAILED. + +**Example** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| +| sundialx_1| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_2| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_4| timer-sundial| FINE-TUNED| TRAINING| +| sundialx_5| timer-sundial| FINE-TUNED| FAILED| ++---------------------+--------------------+--------------+---------+ ``` + ### 4.3 Deleting Models Registered models can be deleted via SQL, which removes all related files under AINode: @@ -246,10 +269,10 @@ Parameter descriptions: 1. The `forecast` function predicts all columns in the input table by default (excluding the time column and columns specified in `partition by`). 2. The `forecast` function does not require the input data to be in any specific order. It sorts the input data in ascending order by the timestamp (specified by the `TIMECOL` parameter) before invoking the model for prediction. -3. Different models have different requirements for the number of input data rows: - - If the input data has fewer rows than the minimum requirement, an error will be thrown. - - If the input data exceeds the maximum row limit, the last rows that meet the requirement will be automatically truncated for processing. - - Among the currently built-in models in AINode, only `sundial` has a row limit. It supports a maximum of 2880 input rows. If exceeded, the last 2880 rows will be automatically used. +3. Different models have varying requirements for the number of input data rows. If the input data has fewer rows than the minimum requirement, an error will be reported. + - Among the current built-in models in AINode: + - Timer-XL requires at least 96 rows of input data. + - Timer-Sundial requires at least 16 rows of input data. 4. The result columns of the `forecast` function include all input columns from the input table, with their original data types preserved. If `preserve_input = true`, an additional `is_input` column will be included to indicate whether a row is from the input data. - Currently, only columns of type INT32, INT64, FLOAT, or DOUBLE are supported for prediction. Otherwise, an error will occur: "The type of the column [%s] is [%s], only INT32, INT64, FLOAT, DOUBLE is allowed." 5. `output_start_time` and `output_interval` only affect the generation of the timestamp column in the output results. Both are optional parameters: @@ -257,7 +280,12 @@ Parameter descriptions: - `output_interval` defaults to the sampling interval of the input data, calculated as: (last timestamp - first timestamp) / (number of rows - 1). - The timestamp of the Nth output row is calculated as: `output_start_time + (N - 1) * output_interval`. -#### Example +**Example: Database and table must be pre-created** + +```sql +create database etth +create table eg (hufl FLOAT FIELD, hull FLOAT FIELD, mufl FLOAT FIELD, mull FLOAT FIELD, lufl FLOAT FIELD, lull FLOAT FIELD, ot FLOAT FIELD) +``` Using the ETTh1-tab dataset:[ETTh1-tab](/img/ETTh1-tab.csv)。 @@ -380,7 +408,7 @@ IoTDB> show models ### 4.6 Time Series Large Model Import Steps -AINode supports multiple time series large models. For deployment, refer to [Time Series Large Model](https://timecho.com/docs/zh/UserGuide/latest/AI-capability/TimeSeries-Large-Model.html) +AINode supports multiple time series large models. For deployment, refer to [Time Series Large Model](../AI-capability/TimeSeries-Large-Model.md) ## 5 Permission Management diff --git a/src/UserGuide/Master/Tree/AI-capability/AINode_apache.md b/src/UserGuide/Master/Tree/AI-capability/AINode_apache.md index 31b3b9e2d..d8626a953 100644 --- a/src/UserGuide/Master/Tree/AI-capability/AINode_apache.md +++ b/src/UserGuide/Master/Tree/AI-capability/AINode_apache.md @@ -23,15 +23,24 @@ AINode is a native IoTDB node that supports the registration, management, and invocation of time-series-related models. It comes with built-in industry-leading self-developed time-series large models, such as the Timer series developed by Tsinghua University. These models can be invoked through standard SQL statements, enabling real-time inference of time series data at the millisecond level, and supporting application scenarios such as trend forecasting, missing value imputation, and anomaly detection for time series data. +> Available since V2.0.5 + The system architecture is shown below: ::: center - -::: + +::: + The responsibilities of the three nodes are as follows: -- **ConfigNode**: responsible for storing and managing the meta-information of the model; responsible for distributed node management. -- **DataNode**: responsible for receiving and parsing SQL requests from users; responsible for storing time-series data; responsible for preprocessing computation of data. -- **AINode**: responsible for model file import creation and model inference. +- **ConfigNode:** + - Manages distributed nodes and handles load balancing across the system. +- **DataNode:** + - Receives and parses user SQL queries. + - Stores time-series data. + - Performs preprocessing computations on raw data. +- **AINode:** + - Manages and utilizes time-series models (including training/inference). + - Supports deep learning and machine learning workflows. ## 1. Advantageous features @@ -58,12 +67,12 @@ Compared with building a machine learning service alone, it has the following ad - **Built-in capabilities**: AINode comes with machine learning algorithms or home-grown models for common timing analysis scenarios (e.g., prediction and anomaly detection). ::: center - + :::: ## 3. Installation and Deployment -The deployment of AINode can be found in the document [Deployment Guidelines](../Deployment-and-Maintenance/AINode_Deployment_apache.md#ainode-deployment) . +The deployment of AINode can be found in the document [AINode Deployment](../Deployment-and-Maintenance/AINode_Deployment_apache.md). ## 4. Usage Guidelines @@ -84,13 +93,13 @@ Models that meet the following criteria can be registered with AINode: The SQL syntax for model registration is defined as follows: ```SQL -create model using uri +create model using uri ``` Detailed meanings of SQL parameters: -- **model_name**: The global unique identifier for the model, non-repeating. Model names have the following constraints: - - Allowed characters: [0-9 a-z A-Z _] (letters, numbers, underscores) +- **model_id**: The global unique identifier for the model, non-repeating. Model names have the following constraints: + - Allowed characters: [0-9 a-z A-Z _] (letters, digits (not at the beginning), underscores (not at the beginning)) - Length: 2-64 characters - Case-sensitive - **uri**: The resource path of the model registration files, which should include the **model structure and weight file `model.pt` and the model configuration file `config.yaml`** @@ -121,7 +130,7 @@ In addition to registering local model files, remote resource paths can be speci #### Example -The current example folder contains model.pt (trained model) and config.yaml with the following content: +The [example folder](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example) contains model.pt (trained model) and config.yaml with the following content: ```YAML configs: @@ -144,12 +153,6 @@ Register the model by specifying this folder as the loading path: IoTDB> create model dlinear_example using uri "file://./example" ``` -Models can also be downloaded from HuggingFace for registration: - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - After SQL execution, registration proceeds asynchronously. The registration status can be checked via model display (see Model Display section). The registration success time mainly depends on the model file size. Once registered, the model can be invoked for inference through normal query syntax. @@ -161,42 +164,49 @@ Successfully registered models can be queried for model-specific information thr ```SQL show models -show models +show models ``` In addition to displaying all models, specifying a `model_id` shows details of a specific model. The display includes: -| **ModelId** | **State** | **Configs** | **Attributes** | -| ----------- | --------------------------------------------------------- | ------------------------------------------------ | -------------- | -| Unique ID | Registration status (INACTIVE, LOADING, ACTIVE, DROPPING) | InputShape, outputShape, inputTypes, outputTypes | User notes | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|---------------|----------------|-------------| +| Model ID | Model Type | Model Category | Model State | -**State descriptions:** +- Model State Transition Diagram -- **INACTIVE**: Model is unavailable. -- **LOADING**: Model is being loaded. -- **ACTIVE**: Model is available. -- **DROPPING**: Model is being deleted. +![](/img/AINode-State-apache-en.png) -#### Example +**Instructions:** + +1. Initialization: + - When AINode starts, show models only displays BUILT-IN models. +2. Custom Model Import: + - Users can import custom models (marked as USER-DEFINED). + - The system attempts to parse the ModelTypefrom the config file. + - If parsing fails, the field remains empty. +3. Foundation Model Weights: + - Time-series foundation model weights are not bundled with AINode. + - AINode automatically downloads them during startup. + - Download state: LOADING. +4. Download Outcomes: + - Success → State changes to ACTIVE. + - Failure → State changes to INACTIVE. + +**Example** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| ++---------------------+--------------------+--------------+---------+ ``` + ### 4.3 Deleting Models Registered models can be deleted via SQL, which removes all related files under AINode: @@ -248,7 +258,7 @@ The following machine learning models are currently built-in, please refer to th After completing the registration of the model, the inference function can be used by calling the inference function through the call keyword, and its corresponding parameters are described as follows: -- **model_name**: corresponds to a registered model +- **model_id**: corresponds to a registered model - **sql**: sql query statement, the result of the query is used as input to the model for model inference. The dimensions of the rows and columns in the result of the query need to match the size specified in the specific model config. (It is not recommended to use the `SELECT *` clause for the sql here because in IoTDB, `*` does not sort the columns, so the order of the columns is undefined, you can use `SELECT s0,s1` to ensure that the columns order matches the expectations of the model input) - **window_function**: Window functions that can be used in the inference process, there are currently three types of window functions provided to assist in model inference: - **head(window_size)**: Get the top window_size points in the data for model inference, this window can be used for data cropping. @@ -392,6 +402,11 @@ Total line number = 4 In the result set, each row's label corresponds to the output of the anomaly detection model after inputting each group of 24 rows of data. + +### 4.5 Time Series Large Model Import Steps + +AINode supports multiple time series large models. For deployment, refer to [Time Series Large Model](../AI-capability/TimeSeries-Large-Model.md) + ## 5. Privilege Management When using AINode related functions, the authentication of IoTDB itself can be used to do a permission management, users can only use the model management related functions when they have the USE_MODEL permission. When using the inference function, the user needs to have the permission to access the source sequence corresponding to the SQL of the input model. @@ -416,10 +431,10 @@ On this dataset, the model inference function of IoTDB-ML can predict the oil te #### Step 1: Data Import -Users can import the ETT dataset into IoTDB using `import-csv.sh` in the tools folder +Users can import the ETT dataset into IoTDB using `import-data.sh` in the tools folder ``Bash -bash . /import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /ETTh1.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/ETTh1.csv `` #### Step 2: Model Import @@ -486,10 +501,10 @@ On this dataset, the model inference function of IoTDB-ML can predict the C-phas #### Step 1: Data Import -Users can import the dataset using `import-csv.sh` in the tools folder +Users can import the dataset using `import-data.sh` in the tools folder ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### Step 2: Model Import @@ -549,10 +564,10 @@ On this dataset, the model inference function of IoTDB-ML can empower the transp #### Step 1: Data Import -Users can import the dataset using `import-csv.sh` in the tools folder +Users can import the dataset using `import-data.sh` in the tools folder ``Bash -bash . /import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv `` #### Step 2: Model Inference diff --git a/src/UserGuide/Master/Tree/AI-capability/AINode_timecho.md b/src/UserGuide/Master/Tree/AI-capability/AINode_timecho.md index d3662dcf6..fd21be5c8 100644 --- a/src/UserGuide/Master/Tree/AI-capability/AINode_timecho.md +++ b/src/UserGuide/Master/Tree/AI-capability/AINode_timecho.md @@ -23,16 +23,24 @@ AINode is a native IoTDB node that supports the registration, management, and invocation of time-series-related models. It comes with built-in industry-leading self-developed time-series large models, such as the Timer series developed by Tsinghua University. These models can be invoked through standard SQL statements, enabling real-time inference of time series data at the millisecond level, and supporting application scenarios such as trend forecasting, missing value imputation, and anomaly detection for time series data. +> Available since V2.0.5.1 The system architecture is shown below: ::: center - -::: + +::: + The responsibilities of the three nodes are as follows: -- **ConfigNode**: responsible for storing and managing the meta-information of the model; responsible for distributed node management. -- **DataNode**: responsible for receiving and parsing SQL requests from users; responsible for storing time-series data; responsible for preprocessing computation of data. -- **AINode**: responsible for model file import creation and model inference. +- **ConfigNode:** + - Manages distributed nodes and handles load balancing across the system. +- **DataNode:** + - Receives and parses user SQL queries. + - Stores time-series data. + - Performs preprocessing computations on raw data. +- **AINode:** + - Manages and utilizes time-series models (including training/inference). + - Supports deep learning and machine learning workflows. ## 1. Advantageous features @@ -59,12 +67,12 @@ Compared with building a machine learning service alone, it has the following ad - **Built-in Capabilities**: AINode comes with machine learning algorithms or self-developed models for common time series analysis scenarios (e.g., forecasting and anomaly detection). ::: center - + :::: ## 3. Installation and Deployment -The deployment of AINode can be found in the document [Deployment Guidelines](../Deployment-and-Maintenance/AINode_Deployment_timecho.md#ainode-deployment) . +The deployment of AINode can be found in the document [AINode Deployment](../Deployment-and-Maintenance/AINode_Deployment_apache.md). ## 4. Usage Guidelines @@ -86,13 +94,13 @@ Models that meet the following criteria can be registered with AINode: The SQL syntax for model registration is defined as follows: ```SQL -create model using uri +create model using uri ``` Detailed meanings of SQL parameters: -- **model_name**: The global unique identifier for the model, non-repeating. Model names have the following constraints: - - Allowed characters: [0-9 a-z A-Z _] (letters, numbers, underscores) +- **model_id**: The global unique identifier for the model, non-repeating. Model names have the following constraints: + - Allowed characters: [0-9 a-z A-Z _] (letters, digits (not at the beginning), underscores (not at the beginning)) - Length: 2-64 characters - Case-sensitive - **uri**: The resource path of the model registration files, which should include the **model structure and weight file `model.pt` and the model configuration file `config.yaml`** @@ -124,7 +132,7 @@ In addition to registering local model files, remote resource paths can be speci #### Example -The current example folder contains model.pt (trained model) and config.yaml with the following content: +The [example folder](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example) contains model.pt (trained model) and config.yaml with the following content: ```YAML configs: @@ -147,12 +155,6 @@ Register the model by specifying this folder as the loading path: IoTDB> create model dlinear_example using uri "file://./example" ``` -Models can also be downloaded from HuggingFace for registration: - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - After SQL execution, registration proceeds asynchronously. The registration status can be checked via model display (see Model Display section). The registration success time mainly depends on the model file size. Once registered, the model can be invoked for inference through normal query syntax. @@ -164,43 +166,56 @@ Registered models can be queried using the `show models` command. The SQL defini ```SQL show models -show models +show models ``` In addition to displaying all models, specifying a `model_id` shows details of a specific model. The display includes: -| **ModelId** | **State** | **Configs** | **Attributes** | -| ----------- | --------------------------------------------------------- | ------------------------------------------------ | -------------- | -| Unique ID | Registration status (INACTIVE, LOADING, ACTIVE,TRAINING,FAILED, DROPPING) | InputShape, outputShape, inputTypes, outputTypes | User notes | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|---------------|----------------|-------------| +| Model ID | Model Type | Model Category | Model State | -**State descriptions:** +- Model State Transition Diagram -- **INACTIVE**: The model is in an unavailable state. -- **LOADING**: The model is being loaded. -- **ACTIVE**: The model is in an available state. -- **TRAINING**: The model is in the fine-tuning state. -- **FAILED**: The model fine-tuning failed. -- **DROPPING**: The model is being deleted. +![](/img/AINode-State-en.png) -#### Example +**Instructions:** + +1. Initialization: + - When AINode starts, show models only displays BUILT-IN models. +2. Custom Model Import: + - Users can import custom models (marked as USER-DEFINED). + - The system attempts to parse the ModelTypefrom the config file. + - If parsing fails, the field remains empty. +3. Foundation Model Weights: + - Time-series foundation model weights are not bundled with AINode. + - AINode automatically downloads them during startup. + - Download state: LOADING. +4. Download Outcomes: + - Success → State changes to ACTIVE. + - Failure → State changes to INACTIVE. +5. Fine-Tuning Process: + - When fine-tuning starts: State becomes TRAINING. + - Successful training → State transitions to ACTIVE. + - Training failure → State changes to FAILED. + +**Example** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| +| sundialx_1| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_2| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_4| timer-sundial| FINE-TUNED| TRAINING| +| sundialx_5| timer-sundial| FINE-TUNED| FAILED| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 Deleting Models @@ -252,7 +267,7 @@ The following machine learning models are currently built-in, please refer to th After completing the registration of the model, the inference function can be used by calling the inference function through the call keyword, and its corresponding parameters are described as follows: -- **model_name**: corresponds to a registered model +- **model_id**: corresponds to a registered model - **sql**: sql query statement, the result of the query is used as input to the model for model inference. The dimensions of the rows and columns in the result of the query need to match the size specified in the specific model config. (It is not recommended to use the `SELECT *` clause for the sql here because in IoTDB, `*` does not sort the columns, so the order of the columns is undefined, you can use `SELECT s0,s1` to ensure that the columns order matches the expectations of the model input) - **window_function**: Window functions that can be used in the inference process, there are currently three types of window functions provided to assist in model inference: - **head(window_size)**: Get the top window_size points in the data for model inference, this window can be used for data cropping. @@ -458,7 +473,7 @@ IoTDB> show models ### 4.6 TimeSeries Large Models Import Steps -AINode currently supports a variety of time series large models. For deployment and usage, please refer to [TimeSeries Large Models](../AI-capability/TimeSeries-Large-Model) +The deployment of AINode can be found in the document [AINode Deployment](../Deployment-and-Maintenance/AINode_Deployment_timecho.md) . ## 5. Privilege Management @@ -485,10 +500,10 @@ On this dataset, the model inference function of IoTDB-ML can predict the oil te #### Step 1: Data Import -Users can import the ETT dataset into IoTDB using `import-csv.sh` in the tools folder +Users can import the ETT dataset into IoTDB using `import-data.sh` in the tools folder ``Bash -bash . /import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /ETTh1.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/ETTh1.csv `` #### Step 2: Model Import @@ -555,10 +570,10 @@ On this dataset, the model inference function of IoTDB-ML can predict the C-phas #### Step 1: Data Import -Users can import the dataset using `import-csv.sh` in the tools folder +Users can import the dataset using `import-data.sh` in the tools folder ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### Step 2: Model Import @@ -618,10 +633,10 @@ On this dataset, the model inference function of IoTDB-ML can empower the transp #### Step 1: Data Import -Users can import the dataset using `import-csv.sh` in the tools folder +Users can import the dataset using `import-data.sh` in the tools folder ``Bash -bash . /import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv `` #### Step 2: Model Inference diff --git a/src/UserGuide/latest-Table/AI-capability/AINode_apache.md b/src/UserGuide/latest-Table/AI-capability/AINode_apache.md index 421bbcde3..b566b72cb 100644 --- a/src/UserGuide/latest-Table/AI-capability/AINode_apache.md +++ b/src/UserGuide/latest-Table/AI-capability/AINode_apache.md @@ -23,15 +23,25 @@ AINode is a native IoTDB node that supports the registration, management, and invocation of time-series-related models. It comes with built-in industry-leading self-developed time-series large models, such as the Timer series developed by Tsinghua University. These models can be invoked through standard SQL statements, enabling real-time inference of time series data at the millisecond level, and supporting application scenarios such as trend forecasting, missing value imputation, and anomaly detection for time series data. +> Available since V2.0.5 + The system architecture is shown below: ::: center - + ::: + The responsibilities of the three nodes are as follows: -- **ConfigNode**: responsible for storing and managing the meta-information of the model; responsible for distributed node management. -- **DataNode**: responsible for receiving and parsing SQL requests from users; responsible for storing time-series data; responsible for preprocessing computation of data. -- **AINode**: responsible for model file import creation and model inference. +- **ConfigNode:** + - Manages distributed nodes and handles load balancing across the system. +- **DataNode:** + - Receives and parses user SQL queries. + - Stores time-series data. + - Performs preprocessing computations on raw data. +- **AINode:** + - Manages and utilizes time-series models (including training/inference). + - Supports deep learning and machine learning workflows. + ## 1. Advantageous features @@ -55,11 +65,11 @@ Compared with building a machine learning service alone, it has the following ad - **Create**: Load externally designed or trained model files/algorithms into AINode for unified management and usage by IoTDB. - **Inference**: Use the created model to complete time series analysis tasks applicable to the model on specified time series data. - **Built-in Capabilities**: AINode comes with machine learning algorithms or self-developed models for common time series analysis scenarios (e.g., forecasting and anomaly detection). -![](/img/h3.png) +![](/img/AINode-en.png) ## 3. Installation and Deployment -The deployment of AINode can be found in the document [Deployment Guidelines](../Deployment-and-Maintenance/AINode_Deployment_apache.md#ainode-deployment) . +The deployment of AINode can be found in the document [AINode Deployment](../Deployment-and-Maintenance/AINode_Deployment_apache.md). ## 4. Usage Guide @@ -80,13 +90,13 @@ Models that meet the following criteria can be registered with AINode: The SQL syntax for model registration is defined as follows: ```SQL -create model using uri +create model using uri ``` Detailed meanings of SQL parameters: -- **model_name**: The global unique identifier for the model, non-repeating. Model names have the following constraints: - - Allowed characters: [0-9 a-z A-Z _] (letters, numbers, underscores) +- **model_id**: The global unique identifier for the model, non-repeating. Model names have the following constraints: + - Allowed characters: [0-9 a-z A-Z _] (letters, digits (not at the beginning), underscores (not at the beginning)) - Length: 2-64 characters - Case-sensitive - **uri**: The resource path of the model registration files, which should include the **model structure and weight file `model.pt` and the model configuration file `config.yaml`** @@ -117,7 +127,7 @@ In addition to registering local model files, remote resource paths can be speci #### Example -The current example folder contains model.pt (trained model) and config.yaml with the following content: +The [example folder](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example) contains model.pt (trained model) and config.yaml with the following content: ```YAML configs: @@ -157,41 +167,47 @@ Registered models can be queried using the `show models` command. The SQL defini ```SQL show models -show models +show models ``` In addition to displaying all models, specifying a `model_id` shows details of a specific model. The display includes: -| **ModelId** | **State** | **Configs** | **Attributes** | -| ----------- | --------------------------------------------------------- | ------------------------------------------------ | -------------- | -| Unique ID | Registration status (INACTIVE, LOADING, ACTIVE, DROPPING) | InputShape, outputShape, inputTypes, outputTypes | User notes | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|---------------|----------------|-------------| +| Model ID | Model Type | Model Category | Model State | -**State descriptions:** +- Model State Transition Diagram -- **INACTIVE**: Model is unavailable. -- **LOADING**: Model is being loaded. -- **ACTIVE**: Model is available. -- **DROPPING**: Model is being deleted. +![](/img/AINode-State-apache-en.png) -#### Example +**Instructions:** + +1. Initialization: + - When AINode starts, show models only displays BUILT-IN models. +2. Custom Model Import: + - Users can import custom models (marked as USER-DEFINED). + - The system attempts to parse the ModelTypefrom the config file. + - If parsing fails, the field remains empty. +3. Foundation Model Weights: + - Time-series foundation model weights are not bundled with AINode. + - AINode automatically downloads them during startup. + - Download state: LOADING. +4. Download Outcomes: + - Success → State changes to ACTIVE. + - Failure → State changes to INACTIVE. + +**Example** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 Deleting Models @@ -243,10 +259,10 @@ Parameter descriptions: 1. The `forecast` function predicts all columns in the input table by default (excluding the time column and columns specified in `partition by`). 2. The `forecast` function does not require the input data to be in any specific order. It sorts the input data in ascending order by the timestamp (specified by the `TIMECOL` parameter) before invoking the model for prediction. -3. Different models have different requirements for the number of input data rows: - - If the input data has fewer rows than the minimum requirement, an error will be thrown. - - If the input data exceeds the maximum row limit, the last rows that meet the requirement will be automatically truncated for processing. - - Among the currently built-in models in AINode, only `sundial` has a row limit. It supports a maximum of 2880 input rows. If exceeded, the last 2880 rows will be automatically used. +3. Different models have varying requirements for the number of input data rows. If the input data has fewer rows than the minimum requirement, an error will be reported. + - Among the current built-in models in AINode: + - Timer-XL requires at least 96 rows of input data. + - Timer-Sundial requires at least 16 rows of input data. 4. The result columns of the `forecast` function include all input columns from the input table, with their original data types preserved. If `preserve_input = true`, an additional `is_input` column will be included to indicate whether a row is from the input data. - Currently, only columns of type INT32, INT64, FLOAT, or DOUBLE are supported for prediction. Otherwise, an error will occur: "The type of the column [%s] is [%s], only INT32, INT64, FLOAT, DOUBLE is allowed." 5. `output_start_time` and `output_interval` only affect the generation of the timestamp column in the output results. Both are optional parameters: @@ -254,7 +270,13 @@ Parameter descriptions: - `output_interval` defaults to the sampling interval of the input data, calculated as: (last timestamp - first timestamp) / (number of rows - 1). - The timestamp of the Nth output row is calculated as: `output_start_time + (N - 1) * output_interval`. -#### Example +**Example: Database and table must be pre-created** + +```sql +create database etth +create table eg (hufl FLOAT FIELD, hull FLOAT FIELD, mufl FLOAT FIELD, mull FLOAT FIELD, lufl FLOAT FIELD, lull FLOAT FIELD, ot FLOAT FIELD) +``` + Using the ETTh1-tab dataset:[ETTh1-tab](/img/ETTh1-tab.csv)。 @@ -314,7 +336,7 @@ It costs 1.615s ### 4.5 Time Series Large Model Import Steps -AINode supports multiple time series large models. For deployment, refer to [Time Series Large Model](https://timecho.com/docs/zh/UserGuide/latest/AI-capability/TimeSeries-Large-Model.html) +AINode supports multiple time series large models. For deployment, refer to [Time Series Large Model](../AI-capability/TimeSeries-Large-Model.md) ## 5 Permission Management diff --git a/src/UserGuide/latest-Table/AI-capability/AINode_timecho.md b/src/UserGuide/latest-Table/AI-capability/AINode_timecho.md index a22f96bf8..4c4f3ca9a 100644 --- a/src/UserGuide/latest-Table/AI-capability/AINode_timecho.md +++ b/src/UserGuide/latest-Table/AI-capability/AINode_timecho.md @@ -23,15 +23,24 @@ AINode is a native IoTDB node that supports the registration, management, and invocation of time-series-related models. It comes with built-in industry-leading self-developed time-series large models, such as the Timer series developed by Tsinghua University. These models can be invoked through standard SQL statements, enabling real-time inference of time series data at the millisecond level, and supporting application scenarios such as trend forecasting, missing value imputation, and anomaly detection for time series data. +> Available since V2.0.5.1 + The system architecture is shown below: ::: center - + ::: + The responsibilities of the three nodes are as follows: -- **ConfigNode**: responsible for storing and managing the meta-information of the model; responsible for distributed node management. -- **DataNode**: responsible for receiving and parsing SQL requests from users; responsible for storing time-series data; responsible for preprocessing computation of data. -- **AINode**: responsible for model file import creation and model inference. +- **ConfigNode:** + - Manages distributed nodes and handles load balancing across the system. +- **DataNode:** + - Receives and parses user SQL queries. + - Stores time-series data. + - Performs preprocessing computations on raw data. +- **AINode:** + - Manages and utilizes time-series models (including training/inference). + - Supports deep learning and machine learning workflows. ## 1. Advantageous features @@ -55,11 +64,11 @@ Compared with building a machine learning service alone, it has the following ad - **Create**: Load externally designed or trained model files/algorithms into AINode for unified management and usage by IoTDB. - **Inference**: Use the created model to complete time series analysis tasks applicable to the model on specified time series data. - **Built-in Capabilities**: AINode comes with machine learning algorithms or self-developed models for common time series analysis scenarios (e.g., forecasting and anomaly detection). -![](/img/h3.png) +![](/img/AINode-en.png) ## 3. Installation and Deployment -The deployment of AINode can be found in the document [Deployment Guidelines](../Deployment-and-Maintenance/AINode_Deployment_timecho.md#ainode-deployment) . +The deployment of AINode can be found in the document [AINode Deployment](../Deployment-and-Maintenance/AINode_Deployment_timecho.md) . ## 4. Usage Guide @@ -80,13 +89,13 @@ Models that meet the following criteria can be registered with AINode: The SQL syntax for model registration is defined as follows: ```SQL -create model using uri +create model using uri ``` Detailed meanings of SQL parameters: -- **model_name**: The global unique identifier for the model, non-repeating. Model names have the following constraints: - - Allowed characters: [0-9 a-z A-Z _] (letters, numbers, underscores) +- **model_id**: The global unique identifier for the model, non-repeating. Model names have the following constraints: + - Allowed characters: [0-9 a-z A-Z _] (letters, digits (not at the beginning), underscores (not at the beginning)) - Length: 2-64 characters - Case-sensitive - **uri**: The resource path of the model registration files, which should include the **model structure and weight file `model.pt` and the model configuration file `config.yaml`** @@ -117,7 +126,7 @@ In addition to registering local model files, remote resource paths can be speci #### Example -The current example folder contains model.pt (trained model) and config.yaml with the following content: +The [example folder](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example) contains model.pt (trained model) and config.yaml with the following content: ```YAML configs: @@ -157,45 +166,59 @@ Registered models can be queried using the `show models` command. The SQL defini ```SQL show models -show models +show models ``` In addition to displaying all models, specifying a `model_id` shows details of a specific model. The display includes: -| **ModelId** | **State** | **Configs** | **Attributes** | -| ----------- | --------------------------------------------------------- | ------------------------------------------------ | -------------- | -| Unique ID | Registration status (INACTIVE, LOADING, ACTIVE,TRAINING,FAILED, DROPPING) | InputShape, outputShape, inputTypes, outputTypes | User notes | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|---------------|----------------|-------------| +| Model ID | Model Type | Model Category | Model State | -**State descriptions:** +- Model State Transition Diagram -- **INACTIVE**: The model is in an unavailable state. -- **LOADING**: The model is being loaded. -- **ACTIVE**: The model is in an available state. -- **TRAINING**: The model is in the fine-tuning state. -- **FAILED**: The model fine-tuning failed. -- **DROPPING**: The model is being deleted. +![](/img/AINode-State-en.png) -#### Example +**Instructions:** + +1. Initialization: + - When AINode starts, show models only displays BUILT-IN models. +2. Custom Model Import: + - Users can import custom models (marked as USER-DEFINED). + - The system attempts to parse the ModelTypefrom the config file. + - If parsing fails, the field remains empty. +3. Foundation Model Weights: + - Time-series foundation model weights are not bundled with AINode. + - AINode automatically downloads them during startup. + - Download state: LOADING. +4. Download Outcomes: + - Success → State changes to ACTIVE. + - Failure → State changes to INACTIVE. +5. Fine-Tuning Process: + - When fine-tuning starts: State becomes TRAINING. + - Successful training → State transitions to ACTIVE. + - Training failure → State changes to FAILED. + +**Example** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| +| sundialx_1| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_2| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_4| timer-sundial| FINE-TUNED| TRAINING| +| sundialx_5| timer-sundial| FINE-TUNED| FAILED| ++---------------------+--------------------+--------------+---------+ ``` + ### 4.3 Deleting Models Registered models can be deleted via SQL, which removes all related files under AINode: @@ -246,10 +269,10 @@ Parameter descriptions: 1. The `forecast` function predicts all columns in the input table by default (excluding the time column and columns specified in `partition by`). 2. The `forecast` function does not require the input data to be in any specific order. It sorts the input data in ascending order by the timestamp (specified by the `TIMECOL` parameter) before invoking the model for prediction. -3. Different models have different requirements for the number of input data rows: - - If the input data has fewer rows than the minimum requirement, an error will be thrown. - - If the input data exceeds the maximum row limit, the last rows that meet the requirement will be automatically truncated for processing. - - Among the currently built-in models in AINode, only `sundial` has a row limit. It supports a maximum of 2880 input rows. If exceeded, the last 2880 rows will be automatically used. +3. Different models have varying requirements for the number of input data rows. If the input data has fewer rows than the minimum requirement, an error will be reported. + - Among the current built-in models in AINode: + - Timer-XL requires at least 96 rows of input data. + - Timer-Sundial requires at least 16 rows of input data. 4. The result columns of the `forecast` function include all input columns from the input table, with their original data types preserved. If `preserve_input = true`, an additional `is_input` column will be included to indicate whether a row is from the input data. - Currently, only columns of type INT32, INT64, FLOAT, or DOUBLE are supported for prediction. Otherwise, an error will occur: "The type of the column [%s] is [%s], only INT32, INT64, FLOAT, DOUBLE is allowed." 5. `output_start_time` and `output_interval` only affect the generation of the timestamp column in the output results. Both are optional parameters: @@ -257,7 +280,12 @@ Parameter descriptions: - `output_interval` defaults to the sampling interval of the input data, calculated as: (last timestamp - first timestamp) / (number of rows - 1). - The timestamp of the Nth output row is calculated as: `output_start_time + (N - 1) * output_interval`. -#### Example +**Example: Database and table must be pre-created** + +```sql +create database etth +create table eg (hufl FLOAT FIELD, hull FLOAT FIELD, mufl FLOAT FIELD, mull FLOAT FIELD, lufl FLOAT FIELD, lull FLOAT FIELD, ot FLOAT FIELD) +``` Using the ETTh1-tab dataset:[ETTh1-tab](/img/ETTh1-tab.csv)。 @@ -380,7 +408,7 @@ IoTDB> show models ### 4.6 Time Series Large Model Import Steps -AINode supports multiple time series large models. For deployment, refer to [Time Series Large Model](https://timecho.com/docs/zh/UserGuide/latest/AI-capability/TimeSeries-Large-Model.html) +AINode supports multiple time series large models. For deployment, refer to [Time Series Large Model](../AI-capability/TimeSeries-Large-Model.md) ## 5 Permission Management diff --git a/src/UserGuide/latest/AI-capability/AINode_apache.md b/src/UserGuide/latest/AI-capability/AINode_apache.md index 31b3b9e2d..5e6acd33a 100644 --- a/src/UserGuide/latest/AI-capability/AINode_apache.md +++ b/src/UserGuide/latest/AI-capability/AINode_apache.md @@ -23,15 +23,24 @@ AINode is a native IoTDB node that supports the registration, management, and invocation of time-series-related models. It comes with built-in industry-leading self-developed time-series large models, such as the Timer series developed by Tsinghua University. These models can be invoked through standard SQL statements, enabling real-time inference of time series data at the millisecond level, and supporting application scenarios such as trend forecasting, missing value imputation, and anomaly detection for time series data. +> Available since V2.0.5 + The system architecture is shown below: ::: center - -::: + +::: + The responsibilities of the three nodes are as follows: -- **ConfigNode**: responsible for storing and managing the meta-information of the model; responsible for distributed node management. -- **DataNode**: responsible for receiving and parsing SQL requests from users; responsible for storing time-series data; responsible for preprocessing computation of data. -- **AINode**: responsible for model file import creation and model inference. +- **ConfigNode:** + - Manages distributed nodes and handles load balancing across the system. +- **DataNode:** + - Receives and parses user SQL queries. + - Stores time-series data. + - Performs preprocessing computations on raw data. +- **AINode:** + - Manages and utilizes time-series models (including training/inference). + - Supports deep learning and machine learning workflows. ## 1. Advantageous features @@ -58,12 +67,12 @@ Compared with building a machine learning service alone, it has the following ad - **Built-in capabilities**: AINode comes with machine learning algorithms or home-grown models for common timing analysis scenarios (e.g., prediction and anomaly detection). ::: center - + :::: ## 3. Installation and Deployment -The deployment of AINode can be found in the document [Deployment Guidelines](../Deployment-and-Maintenance/AINode_Deployment_apache.md#ainode-deployment) . +The deployment of AINode can be found in the document [AINode Deployment](../Deployment-and-Maintenance/AINode_Deployment_apache.md). ## 4. Usage Guidelines @@ -84,13 +93,13 @@ Models that meet the following criteria can be registered with AINode: The SQL syntax for model registration is defined as follows: ```SQL -create model using uri +create model using uri ``` Detailed meanings of SQL parameters: -- **model_name**: The global unique identifier for the model, non-repeating. Model names have the following constraints: - - Allowed characters: [0-9 a-z A-Z _] (letters, numbers, underscores) +- **model_id**: The global unique identifier for the model, non-repeating. Model names have the following constraints: + - Allowed characters: [0-9 a-z A-Z _] (letters, digits (not at the beginning), underscores (not at the beginning)) - Length: 2-64 characters - Case-sensitive - **uri**: The resource path of the model registration files, which should include the **model structure and weight file `model.pt` and the model configuration file `config.yaml`** @@ -121,7 +130,7 @@ In addition to registering local model files, remote resource paths can be speci #### Example -The current example folder contains model.pt (trained model) and config.yaml with the following content: +The [example folder](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example) contains model.pt (trained model) and config.yaml with the following content: ```YAML configs: @@ -144,12 +153,6 @@ Register the model by specifying this folder as the loading path: IoTDB> create model dlinear_example using uri "file://./example" ``` -Models can also be downloaded from HuggingFace for registration: - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - After SQL execution, registration proceeds asynchronously. The registration status can be checked via model display (see Model Display section). The registration success time mainly depends on the model file size. Once registered, the model can be invoked for inference through normal query syntax. @@ -161,42 +164,49 @@ Successfully registered models can be queried for model-specific information thr ```SQL show models -show models +show models ``` In addition to displaying all models, specifying a `model_id` shows details of a specific model. The display includes: -| **ModelId** | **State** | **Configs** | **Attributes** | -| ----------- | --------------------------------------------------------- | ------------------------------------------------ | -------------- | -| Unique ID | Registration status (INACTIVE, LOADING, ACTIVE, DROPPING) | InputShape, outputShape, inputTypes, outputTypes | User notes | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|---------------|----------------|-------------| +| Model ID | Model Type | Model Category | Model State | -**State descriptions:** +- Model State Transition Diagram -- **INACTIVE**: Model is unavailable. -- **LOADING**: Model is being loaded. -- **ACTIVE**: Model is available. -- **DROPPING**: Model is being deleted. +![](/img/AINode-State-apache-en.png) -#### Example +**Instructions:** + +1. Initialization: + - When AINode starts, show models only displays BUILT-IN models. +2. Custom Model Import: + - Users can import custom models (marked as USER-DEFINED). + - The system attempts to parse the ModelTypefrom the config file. + - If parsing fails, the field remains empty. +3. Foundation Model Weights: + - Time-series foundation model weights are not bundled with AINode. + - AINode automatically downloads them during startup. + - Download state: LOADING. +4. Download Outcomes: + - Success → State changes to ACTIVE. + - Failure → State changes to INACTIVE. + +**Example** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| ++---------------------+--------------------+--------------+---------+ ``` + ### 4.3 Deleting Models Registered models can be deleted via SQL, which removes all related files under AINode: @@ -248,7 +258,7 @@ The following machine learning models are currently built-in, please refer to th After completing the registration of the model, the inference function can be used by calling the inference function through the call keyword, and its corresponding parameters are described as follows: -- **model_name**: corresponds to a registered model +- **model_id**: corresponds to a registered model - **sql**: sql query statement, the result of the query is used as input to the model for model inference. The dimensions of the rows and columns in the result of the query need to match the size specified in the specific model config. (It is not recommended to use the `SELECT *` clause for the sql here because in IoTDB, `*` does not sort the columns, so the order of the columns is undefined, you can use `SELECT s0,s1` to ensure that the columns order matches the expectations of the model input) - **window_function**: Window functions that can be used in the inference process, there are currently three types of window functions provided to assist in model inference: - **head(window_size)**: Get the top window_size points in the data for model inference, this window can be used for data cropping. @@ -392,6 +402,11 @@ Total line number = 4 In the result set, each row's label corresponds to the output of the anomaly detection model after inputting each group of 24 rows of data. + +### 4.5 Time Series Large Model Import Steps + +AINode supports multiple time series large models. For deployment, refer to [Time Series Large Model](../AI-capability/TimeSeries-Large-Model.md) + ## 5. Privilege Management When using AINode related functions, the authentication of IoTDB itself can be used to do a permission management, users can only use the model management related functions when they have the USE_MODEL permission. When using the inference function, the user needs to have the permission to access the source sequence corresponding to the SQL of the input model. @@ -416,10 +431,10 @@ On this dataset, the model inference function of IoTDB-ML can predict the oil te #### Step 1: Data Import -Users can import the ETT dataset into IoTDB using `import-csv.sh` in the tools folder +Users can import the ETT dataset into IoTDB using `import-data.sh` in the tools folder -``Bash -bash . /import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /ETTh1.csv +``Bash +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/ETTh1.csv `` #### Step 2: Model Import @@ -486,10 +501,10 @@ On this dataset, the model inference function of IoTDB-ML can predict the C-phas #### Step 1: Data Import -Users can import the dataset using `import-csv.sh` in the tools folder +Users can import the dataset using `import-data.sh` in the tools folder ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### Step 2: Model Import @@ -549,10 +564,10 @@ On this dataset, the model inference function of IoTDB-ML can empower the transp #### Step 1: Data Import -Users can import the dataset using `import-csv.sh` in the tools folder +Users can import the dataset using `import-data.sh` in the tools folder ``Bash -bash . /import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv `` #### Step 2: Model Inference diff --git a/src/UserGuide/latest/AI-capability/AINode_timecho.md b/src/UserGuide/latest/AI-capability/AINode_timecho.md index d3662dcf6..11d85ef48 100644 --- a/src/UserGuide/latest/AI-capability/AINode_timecho.md +++ b/src/UserGuide/latest/AI-capability/AINode_timecho.md @@ -23,16 +23,24 @@ AINode is a native IoTDB node that supports the registration, management, and invocation of time-series-related models. It comes with built-in industry-leading self-developed time-series large models, such as the Timer series developed by Tsinghua University. These models can be invoked through standard SQL statements, enabling real-time inference of time series data at the millisecond level, and supporting application scenarios such as trend forecasting, missing value imputation, and anomaly detection for time series data. +> Available since V2.0.5.1 The system architecture is shown below: ::: center - -::: + +::: + The responsibilities of the three nodes are as follows: -- **ConfigNode**: responsible for storing and managing the meta-information of the model; responsible for distributed node management. -- **DataNode**: responsible for receiving and parsing SQL requests from users; responsible for storing time-series data; responsible for preprocessing computation of data. -- **AINode**: responsible for model file import creation and model inference. +- **ConfigNode:** + - Manages distributed nodes and handles load balancing across the system. +- **DataNode:** + - Receives and parses user SQL queries. + - Stores time-series data. + - Performs preprocessing computations on raw data. +- **AINode:** + - Manages and utilizes time-series models (including training/inference). + - Supports deep learning and machine learning workflows. ## 1. Advantageous features @@ -59,12 +67,12 @@ Compared with building a machine learning service alone, it has the following ad - **Built-in Capabilities**: AINode comes with machine learning algorithms or self-developed models for common time series analysis scenarios (e.g., forecasting and anomaly detection). ::: center - + :::: ## 3. Installation and Deployment -The deployment of AINode can be found in the document [Deployment Guidelines](../Deployment-and-Maintenance/AINode_Deployment_timecho.md#ainode-deployment) . +The deployment of AINode can be found in the document [AINode Deployment](../Deployment-and-Maintenance/AINode_Deployment_apache.md). ## 4. Usage Guidelines @@ -86,13 +94,13 @@ Models that meet the following criteria can be registered with AINode: The SQL syntax for model registration is defined as follows: ```SQL -create model using uri +create model using uri ``` Detailed meanings of SQL parameters: -- **model_name**: The global unique identifier for the model, non-repeating. Model names have the following constraints: - - Allowed characters: [0-9 a-z A-Z _] (letters, numbers, underscores) +- **model_id**: The global unique identifier for the model, non-repeating. Model names have the following constraints: + - Allowed characters: [0-9 a-z A-Z _] (letters, digits (not at the beginning), underscores (not at the beginning)) - Length: 2-64 characters - Case-sensitive - **uri**: The resource path of the model registration files, which should include the **model structure and weight file `model.pt` and the model configuration file `config.yaml`** @@ -124,7 +132,7 @@ In addition to registering local model files, remote resource paths can be speci #### Example -The current example folder contains model.pt (trained model) and config.yaml with the following content: +The [example folder](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example) contains model.pt (trained model) and config.yaml with the following content: ```YAML configs: @@ -147,12 +155,6 @@ Register the model by specifying this folder as the loading path: IoTDB> create model dlinear_example using uri "file://./example" ``` -Models can also be downloaded from HuggingFace for registration: - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - After SQL execution, registration proceeds asynchronously. The registration status can be checked via model display (see Model Display section). The registration success time mainly depends on the model file size. Once registered, the model can be invoked for inference through normal query syntax. @@ -164,43 +166,56 @@ Registered models can be queried using the `show models` command. The SQL defini ```SQL show models -show models +show models ``` In addition to displaying all models, specifying a `model_id` shows details of a specific model. The display includes: -| **ModelId** | **State** | **Configs** | **Attributes** | -| ----------- | --------------------------------------------------------- | ------------------------------------------------ | -------------- | -| Unique ID | Registration status (INACTIVE, LOADING, ACTIVE,TRAINING,FAILED, DROPPING) | InputShape, outputShape, inputTypes, outputTypes | User notes | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|---------------|----------------|-------------| +| Model ID | Model Type | Model Category | Model State | -**State descriptions:** +- Model State Transition Diagram -- **INACTIVE**: The model is in an unavailable state. -- **LOADING**: The model is being loaded. -- **ACTIVE**: The model is in an available state. -- **TRAINING**: The model is in the fine-tuning state. -- **FAILED**: The model fine-tuning failed. -- **DROPPING**: The model is being deleted. +![](/img/AINode-State-en.png) -#### Example +**Instructions:** + +1. Initialization: + - When AINode starts, show models only displays BUILT-IN models. +2. Custom Model Import: + - Users can import custom models (marked as USER-DEFINED). + - The system attempts to parse the ModelTypefrom the config file. + - If parsing fails, the field remains empty. +3. Foundation Model Weights: + - Time-series foundation model weights are not bundled with AINode. + - AINode automatically downloads them during startup. + - Download state: LOADING. +4. Download Outcomes: + - Success → State changes to ACTIVE. + - Failure → State changes to INACTIVE. +5. Fine-Tuning Process: + - When fine-tuning starts: State becomes TRAINING. + - Successful training → State transitions to ACTIVE. + - Training failure → State changes to FAILED. + +**Example** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| +| sundialx_1| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_2| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_4| timer-sundial| FINE-TUNED| TRAINING| +| sundialx_5| timer-sundial| FINE-TUNED| FAILED| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 Deleting Models @@ -252,7 +267,7 @@ The following machine learning models are currently built-in, please refer to th After completing the registration of the model, the inference function can be used by calling the inference function through the call keyword, and its corresponding parameters are described as follows: -- **model_name**: corresponds to a registered model +- **model_id**: corresponds to a registered model - **sql**: sql query statement, the result of the query is used as input to the model for model inference. The dimensions of the rows and columns in the result of the query need to match the size specified in the specific model config. (It is not recommended to use the `SELECT *` clause for the sql here because in IoTDB, `*` does not sort the columns, so the order of the columns is undefined, you can use `SELECT s0,s1` to ensure that the columns order matches the expectations of the model input) - **window_function**: Window functions that can be used in the inference process, there are currently three types of window functions provided to assist in model inference: - **head(window_size)**: Get the top window_size points in the data for model inference, this window can be used for data cropping. @@ -458,7 +473,7 @@ IoTDB> show models ### 4.6 TimeSeries Large Models Import Steps -AINode currently supports a variety of time series large models. For deployment and usage, please refer to [TimeSeries Large Models](../AI-capability/TimeSeries-Large-Model) +The deployment of AINode can be found in the document [AINode Deployment](../Deployment-and-Maintenance/AINode_Deployment_timecho.md) . ## 5. Privilege Management @@ -485,10 +500,10 @@ On this dataset, the model inference function of IoTDB-ML can predict the oil te #### Step 1: Data Import -Users can import the ETT dataset into IoTDB using `import-csv.sh` in the tools folder +Users can import the ETT dataset into IoTDB using `import-data.sh` in the tools folder ``Bash -bash . /import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /ETTh1.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/ETTh1.csv `` #### Step 2: Model Import @@ -555,10 +570,10 @@ On this dataset, the model inference function of IoTDB-ML can predict the C-phas #### Step 1: Data Import -Users can import the dataset using `import-csv.sh` in the tools folder +Users can import the dataset using `import-data.sh` in the tools folder -```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /data.csv +```Bash +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### Step 2: Model Import @@ -618,10 +633,10 @@ On this dataset, the model inference function of IoTDB-ML can empower the transp #### Step 1: Data Import -Users can import the dataset using `import-csv.sh` in the tools folder +Users can import the dataset using `import-data.sh` in the tools folder -``Bash -bash . /import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ... /... /data.csv +``Bash +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv `` #### Step 2: Model Inference diff --git a/src/zh/UserGuide/Master/Table/AI-capability/AINode_apache.md b/src/zh/UserGuide/Master/Table/AI-capability/AINode_apache.md index 9be944359..d3f1d14cb 100644 --- a/src/zh/UserGuide/Master/Table/AI-capability/AINode_apache.md +++ b/src/zh/UserGuide/Master/Table/AI-capability/AINode_apache.md @@ -23,15 +23,17 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点，内置业界领先的自研时序大模型，如清华自研时序模型 Timer 系列，可通过标准 SQL 语句进行调用，实现时序数据的毫秒级实时推理，可支持时序趋势预测、缺失值填补、异常值检测等应用场景。 +> V2.0.5及以后版本支持 + 系统架构如下图所示： -![](/img/h4.png) +![](/img/AINode-0.png) 三种节点的职责如下： -- **ConfigNode**：负责保存和管理模型的元信息；负责分布式节点管理。 +- **ConfigNode**：负责分布式节点管理和负载均衡。 - **DataNode**：负责接收并解析用户的 SQL请求；负责存储时间序列数据；负责数据的预处理计算。 -- **AINode**：负责模型文件的导入创建以及模型推理。 +- **AINode**：负责时序模型的管理和使用。 ## 1. 优势特点 @@ -56,11 +58,11 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点 - **推理（Inference）**：使用创建的模型在指定时序数据上完成该模型适用的时序分析任务。 - **内置能力（Built-in）**：AINode 自带常见时序分析场景（例如预测与异常检测）的机器学习算法或自研模型。 -![](/img/h3.png) +![](/img/AINode-new.png) ## 3. 安装部署 -AINode 的部署可参考文档 [部署指导](../Deployment-and-Maintenance/AINode_Deployment_apache.md#ainode-部署) 章节。 +AINode 的部署可参考文档 [AINode 部署](../Deployment-and-Maintenance/AINode_Deployment_apache.md)。 ## 4. 使用指导 @@ -79,14 +81,14 @@ AINode 对时序模型提供了模型创建及删除功能，内置模型无需 下方为模型注册的SQL语法定义。 ```SQL -create model using uri +create model using uri ``` SQL中参数的具体含义如下： -- model_name：模型的全局唯一标识，不可重复。模型名称具备以下约束： +- model_id：模型的全局唯一标识，不可重复。模型名称具备以下约束： - - 允许出现标识符 [ 0-9 a-z A-Z _ ] （字母，数字，下划线） + - 允许出现标识符 [ 0-9 a-z A-Z _ ]（字母，数字（非开头），下划线（非开头）） - 长度限制为2-64字符 - 大小写敏感 @@ -96,19 +98,19 @@ SQL中参数的具体含义如下： - 模型配置文件：模型注册时需要提供的与模型结构有关的参数，其中必须包含模型的输入输出维度用于模型推理： - - | **参数名** | **参数描述** | **示例** | - | ------------ | ---------------------------- | -------- | - | input_shape | 模型输入的行列，用于模型推理 | [96,2] | - | output_shape | 模型输出的行列，用于模型推理 | [48,2] | + - | **参数名** | **参数描述** | **示例** | + | ------------ | ---------------------------- | -------- | + | input_shape | 模型输入的行列，用于模型推理 | [96,2] | + | output_shape | 模型输出的行列，用于模型推理 | [48,2] | - - ​ 除了模型推理外，还可以指定模型输入输出的数据类型： + 除了模型推理外，还可以指定模型输入输出的数据类型： - | **参数名** | **参数描述** | **示例** | | ----------- | ------------------ | --------------------- | | input_type | 模型输入的数据类型 | ['float32','float32'] | | output_type | 模型输出的数据类型 | ['float32','float32'] | - - ​ 除此之外，可以额外指定备注信息用于在模型管理时进行展示 + 除此之外，可以额外指定备注信息用于在模型管理时进行展示 - | **参数名** | **参数描述** | **示例** | | ---------- | ---------------------------------------------- | ------------------------------------------- | @@ -119,7 +121,7 @@ SQL中参数的具体含义如下： #### 示例 -在当前的example文件夹下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： +在 [example 文件夹](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example)下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： ```YAML configs: @@ -142,12 +144,6 @@ attributes: # 可选项 为用户自定义的备注信息 IoTDB> create model dlinear_example using uri "file://./example" ``` -也可以从huggingFace上下载对应的模型文件进行注册 - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - SQL执行后会异步进行注册的流程，可以通过模型展示查看模型的注册状态（见模型展示章节），注册成功的耗时主要受到模型文件大小的影响。 模型注册完成后，就可以通过使用正常查询的方式调用具体函数，进行模型推理。 @@ -159,41 +155,38 @@ SQL执行后会异步进行注册的流程，可以通过模型展示查看模 ```SQL show models -show models +show models ``` 除了直接展示所有模型的信息外，可以指定model id来查看某一具体模型的信息。模型展示的结果中包含如下信息： -| **ModelId** | **State** | **Configs** | **Attributes** | -| ------------ | ------------------------------------- | ---------------------------------------------- | -------------- | -| 模型唯一标识 | 模型注册状态(INACTIVE,LOADING,ACTIVE,DROPPING) | InputShape, outputShapeInputTypes, outputTypes | 模型备注信息 | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|-----------|--------------|----------------| +| 模型ID | 模型类型 | 模型种类 | 模型状态 | -其中，State用于展示当前模型注册的状态，包含以下三个阶段 +- 模型状态机流转示意图如下 -- **INACTIVE**：模型处于不可用状态 -- **LOADING**：模型加载中状态 -- **ACTIVE**：模型处于可用状态 -- **DROPPING**:模型删除中状态 +![](/img/AINode-State-apache.png) -#### 示例 +**说明：** + +1. 启动 AINode，show models 只能看到 BUILT-IN 模型 +2. 用户可导入自己的模型，来源为 USER-DEFINED，可尝试从配置文件解析 ModelType，解析不到则为空 +3. 时序大模型权重不随 AINode 打包，AINode 启动时自动下载，下载过程中为 LOADING +4. 下载成功转变为 ACTIVE，失败则变成 INACTIVE + +**示例** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 删除模型 @@ -245,8 +238,8 @@ SELECT * FROM forecast( 1. forecast 函数默认对输入表中所有列进行预测（不包含time列和partition by 的列）。 2. forecast 函数对于输入数据无顺序性要求，默认对输入数据按照时间戳（由 TIMECOL 参数指定时间戳的列名）做升序排序后，再调用模型进行预测。 -3. 不同模型对于输入数据的行数要求不同，输入数据少于最低行数要求时，则报错；输入数据高于最大行数要求时，则自动截取最后的满足要求的行数。 - - 在当前的 AINdoe 内置模型中，仅_sundial对输入行数有限制。_sundial 目前支持的最大输入行数为 2880，超过 2880 时，会自动截取输入数据的最后 2880 行。 +3. 不同模型对于输入数据的行数要求不同，输入数据少于最低行数要求时会报错。 + - 在当前的 AINdoe 内置模型中，Timer-XL 模型至少需要输入 96 行数据，Timer-Sundial 模型至少需要输入 16 行数据。 4. forecast 函数的返回结果列包含 input 表的所有输入列，列的数据类型与原表列的数据类型一致。若 preserve_input= true，则还包含 is_input 列（表征当前行是否为输入行） - 目前只支持对 INT32、INT64、FLOAT、DOUBLE 进行预测，否则报错：The type of the column [%s] is [%s], only INT32, INT64, FLOAT, DOUBLE is allowed 5. output_start_time 和 output_interval 只会影响输出结果集的时间戳列生成，均为可选参数。 @@ -254,7 +247,12 @@ SELECT * FROM forecast( - output_interval = (输入数据的最后一个时间戳 - 输入数据的第一个时间戳) / n - 1， 默认为输入数据的采样间隔 - 第 N 个输出行的时间为 output_start_time + (N - 1) * output_interval -#### 示例 +**示例：需要提前创建数据库及表** + +```sql +create database etth +create table eg (hufl FLOAT FIELD, hull FLOAT FIELD, mufl FLOAT FIELD, mull FLOAT FIELD, lufl FLOAT FIELD, lull FLOAT FIELD, ot FLOAT FIELD) +``` 我们所使用的的测试集的数据为[ETTh1-tab](/img/ETTh1-tab.csv)。 @@ -314,7 +312,7 @@ It costs 1.615s ### 4.5 时序大模型导入步骤 -AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](https://timecho.com/docs/zh/UserGuide/latest/AI-capability/TimeSeries-Large-Model.html) +AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](../AI-capability/TimeSeries-Large-Model.md) ## 5 权限管理 diff --git a/src/zh/UserGuide/Master/Table/AI-capability/AINode_timecho.md b/src/zh/UserGuide/Master/Table/AI-capability/AINode_timecho.md index 562cd2f20..0e98d5d2f 100644 --- a/src/zh/UserGuide/Master/Table/AI-capability/AINode_timecho.md +++ b/src/zh/UserGuide/Master/Table/AI-capability/AINode_timecho.md @@ -23,15 +23,17 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点，内置业界领先的自研时序大模型，如清华自研时序模型 Timer 系列，可通过标准 SQL 语句进行调用，实现时序数据的毫秒级实时推理，可支持时序趋势预测、缺失值填补、异常值检测等应用场景。 +> V2.0.5.1及以后版本支持 + 系统架构如下图所示： -![](/img/h4.png) +![](/img/AINode-0.png) 三种节点的职责如下： -- **ConfigNode**：负责保存和管理模型的元信息；负责分布式节点管理。 +- **ConfigNode**：负责分布式节点管理和负载均衡。 - **DataNode**：负责接收并解析用户的 SQL请求；负责存储时间序列数据；负责数据的预处理计算。 -- **AINode**：负责模型文件的导入创建以及模型推理。 +- **AINode**：负责时序模型的管理和使用。 ## 1. 优势特点 @@ -56,11 +58,11 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点 - **推理（Inference）**：使用创建的模型在指定时序数据上完成该模型适用的时序分析任务。 - **内置能力（Built-in）**：AINode 自带常见时序分析场景（例如预测与异常检测）的机器学习算法或自研模型。 -![](/img/h3.png) +![](/img/AINode-new.png) ## 3. 安装部署 -AINode 的部署可参考文档 [部署指导](../Deployment-and-Maintenance/AINode_Deployment_apache.md#ainode-部署) 章节。 +AINode 的部署可参考文档 [AINode 部署](../Deployment-and-Maintenance/AINode_Deployment_timecho.md) 章节。 ## 4. 使用指导 @@ -79,14 +81,14 @@ AINode 对时序模型提供了模型创建及删除功能，内置模型无需 下方为模型注册的SQL语法定义。 ```SQL -create model using uri +create model using uri ``` SQL中参数的具体含义如下： -- model_name：模型的全局唯一标识，不可重复。模型名称具备以下约束： +- model_id：模型的全局唯一标识，不可重复。模型名称具备以下约束： - - 允许出现标识符 [ 0-9 a-z A-Z _ ] （字母，数字，下划线） + - 允许出现标识符 [ 0-9 a-z A-Z _ ]（字母，数字（非开头），下划线（非开头）） - 长度限制为2-64字符 - 大小写敏感 @@ -119,7 +121,7 @@ SQL中参数的具体含义如下： #### 示例 -在当前的example文件夹下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： +在 [example 文件夹](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example)下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： ```YAML configs: @@ -142,12 +144,6 @@ attributes: # 可选项 为用户自定义的备注信息 IoTDB> create model dlinear_example using uri "file://./example" ``` -也可以从huggingFace上下载对应的模型文件进行注册 - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - SQL执行后会异步进行注册的流程，可以通过模型展示查看模型的注册状态（见模型展示章节），注册成功的耗时主要受到模型文件大小的影响。 模型注册完成后，就可以通过使用正常查询的方式调用具体函数，进行模型推理。 @@ -159,43 +155,44 @@ SQL执行后会异步进行注册的流程，可以通过模型展示查看模 ```SQL show models -show models +show models ``` 除了直接展示所有模型的信息外，可以指定model id来查看某一具体模型的信息。模型展示的结果中包含如下信息： -| **ModelId** | **State** | **Configs** | **Attributes** | -| ------------ | ------------------------------------- | ---------------------------------------------- | -------------- | -| 模型唯一标识 | 模型注册状态(INACTIVE,LOADING,ACTIVE,TRAINING,FAILED,DROPPING) | InputShape, outputShapeInputTypes, outputTypes | 模型备注信息 | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|-----------|--------------|----------------| +| 模型ID | 模型类型 | 模型种类 | 模型状态 | -其中，State用于展示当前模型注册的状态，包含以下三个阶段 +- 模型状态机流转示意图如下 -- **INACTIVE**：模型处于不可用状态 -- **LOADING**：模型加载中 -- **ACTIVE**：模型处于可用状态 -- **TRAINING**：模型处于微调中状态 -- **FAILED**：模型微调失败状态 -- **DROPPING**:模型删除中状态 +![](/img/AINode-State.png) -#### 示例 +**说明：** + +1. 启动 AINode，show models 只能看到 BUILT-IN 模型 +2. 用户可导入自己的模型，来源为 USER-DEFINED，可尝试从配置文件解析 ModelType，解析不到则为空 +3. 时序大模型权重不随 AINode 打包，AINode 启动时自动下载，下载过程中为 LOADING +4. 下载成功转变为 ACTIVE，失败则变成 INACTIVE +5. 用户启动微调，正在训练的模型状态为 TRAINING，训练成功变为 ACTIVE，失败则是 FAILED + +**示例** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| +| sundialx_1| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_2| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_4| timer-sundial| FINE-TUNED| TRAINING| +| sundialx_5| timer-sundial| FINE-TUNED| FAILED| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 删除模型 @@ -247,8 +244,8 @@ SELECT * FROM forecast( 1. forecast 函数默认对输入表中所有列进行预测（不包含time列和partition by 的列）。 2. forecast 函数对于输入数据无顺序性要求，默认对输入数据按照时间戳（由 TIMECOL 参数指定时间戳的列名）做升序排序后，再调用模型进行预测。 -3. 不同模型对于输入数据的行数要求不同，输入数据少于最低行数要求时，则报错；输入数据高于最大行数要求时，则自动截取最后的满足要求的行数。 - - 在当前的 AINdoe 内置模型中，仅_sundial对输入行数有限制。_sundial 目前支持的最大输入行数为 2880，超过 2880 时，会自动截取输入数据的最后 2880 行。 +3. 不同模型对于输入数据的行数要求不同，输入数据少于最低行数要求时会报错。 + - 在当前的 AINdoe 内置模型中，Timer-XL 模型至少需要输入 96 行数据，Timer-Sundial 模型至少需要输入 16 行数据。 4. forecast 函数的返回结果列包含 input 表的所有输入列，列的数据类型与原表列的数据类型一致。若 preserve_input= true，则还包含 is_input 列（表征当前行是否为输入行） - 目前只支持对 INT32、INT64、FLOAT、DOUBLE 进行预测，否则报错：The type of the column [%s] is [%s], only INT32, INT64, FLOAT, DOUBLE is allowed 5. output_start_time 和 output_interval 只会影响输出结果集的时间戳列生成，均为可选参数。 @@ -256,7 +253,12 @@ SELECT * FROM forecast( - output_interval = (输入数据的最后一个时间戳 - 输入数据的第一个时间戳) / n - 1， 默认为输入数据的采样间隔 - 第 N 个输出行的时间为 output_start_time + (N - 1) * output_interval -#### 示例 +**示例：需要提前创建数据库及表** + +```sql +create database etth +create table eg (hufl FLOAT FIELD, hull FLOAT FIELD, mufl FLOAT FIELD, mull FLOAT FIELD, lufl FLOAT FIELD, lull FLOAT FIELD, ot FLOAT FIELD) +``` 我们所使用的的测试集的数据为[ETTh1-tab](/img/ETTh1-tab.csv)。 @@ -379,7 +381,7 @@ IoTDB> show models ### 4.6 时序大模型导入步骤 -AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](https://timecho.com/docs/zh/UserGuide/latest/AI-capability/TimeSeries-Large-Model.html) +AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](../AI-capability/TimeSeries-Large-Model.md) ## 5 权限管理 diff --git a/src/zh/UserGuide/Master/Tree/AI-capability/AINode_apache.md b/src/zh/UserGuide/Master/Tree/AI-capability/AINode_apache.md index 9d92fb365..30a4b1d32 100644 --- a/src/zh/UserGuide/Master/Tree/AI-capability/AINode_apache.md +++ b/src/zh/UserGuide/Master/Tree/AI-capability/AINode_apache.md @@ -23,15 +23,17 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点，内置业界领先的自研时序大模型，如清华自研时序模型 Timer 系列，可通过标准 SQL 语句进行调用，实现时序数据的毫秒级实时推理，可支持时序趋势预测、缺失值填补、异常值检测等应用场景。 +> V2.0.5及以后版本支持 + 系统架构如下图所示： -![](/img/h4.png) +![](/img/AINode-0.png) 三种节点的职责如下： -- **ConfigNode**：负责保存和管理模型的元信息；负责分布式节点管理。 +- **ConfigNode**：负责分布式节点管理和负载均衡。 - **DataNode**：负责接收并解析用户的 SQL请求；负责存储时间序列数据；负责数据的预处理计算。 -- **AINode**：负责模型文件的导入创建以及模型推理。 +- **AINode**：负责时序模型的管理和使用。 ## 1. 优势特点 @@ -56,11 +58,11 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点 - **推理（Inference）**：使用创建的模型在指定时序数据上完成该模型适用的时序分析任务。 - **内置能力（Built-in）**：AINode 自带常见时序分析场景（例如预测与异常检测）的机器学习算法或自研模型。 -![](/img/h3.png) +![](/img/AINode-new.png) ## 3. 安装部署 -AINode 的部署可参考文档 [部署指导](../Deployment-and-Maintenance/AINode_Deployment_apache.md#ainode-部署) 章节。 +AINode 的部署可参考文档 [AINode 部署](../Deployment-and-Maintenance/AINode_Deployment_apache.md)。 ## 4. 使用指导 @@ -79,14 +81,14 @@ AINode 对时序模型提供了模型创建及删除功能，内置模型无需 下方为模型注册的SQL语法定义。 ```SQL -create model using uri +create model using uri ``` SQL中参数的具体含义如下： -- model_name：模型的全局唯一标识，不可重复。模型名称具备以下约束： +- model_id：模型的全局唯一标识，不可重复。模型名称具备以下约束： - - 允许出现标识符 [ 0-9 a-z A-Z _ ] （字母，数字，下划线） + - 允许出现标识符 [ 0-9 a-z A-Z _ ]（字母，数字（非开头），下划线（非开头）） - 长度限制为2-64字符 - 大小写敏感 @@ -119,7 +121,7 @@ SQL中参数的具体含义如下： #### 示例 -在当前的example文件夹下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： +在[example 文件夹](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example)下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： ```YAML configs: @@ -142,12 +144,6 @@ attributes: # 可选项 为用户自定义的备注信息 IoTDB> create model dlinear_example using uri "file://./example" ``` -也可以从huggingFace上下载对应的模型文件进行注册 - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - SQL执行后会异步进行注册的流程，可以通过模型展示查看模型的注册状态（见模型展示章节），注册成功的耗时主要受到模型文件大小的影响。 模型注册完成后，就可以通过使用正常查询的方式调用具体函数，进行模型推理。 @@ -159,41 +155,38 @@ SQL执行后会异步进行注册的流程，可以通过模型展示查看模 ```SQL show models -show models +show models ``` 除了直接展示所有模型的信息外，可以指定model id来查看某一具体模型的信息。模型展示的结果中包含如下信息： -| **ModelId** | **State** | **Configs** | **Attributes** | -| ------------ | ------------------------------------- | ---------------------------------------------- | -------------- | -| 模型唯一标识 | 模型注册状态(INACTIVE,LOADING,ACTIVE,DROPPING) | InputShape, outputShapeInputTypes, outputTypes | 模型备注信息 | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|-----------|--------------|----------------| +| 模型ID | 模型类型 | 模型种类 | 模型状态 | -其中，State用于展示当前模型注册的状态，包含以下三个阶段 +- 模型状态机流转示意图如下 -- **INACTIVE**：模型处于不可用状态 -- **LOADING**：模型加载中状态 -- **ACTIVE**：模型处于可用状态 -- **DROPPING**:模型删除中状态 +![](/img/AINode-State-apache.png) -#### 示例 +**说明：** + +1. 启动 AINode，show models 只能看到 BUILT-IN 模型 +2. 用户可导入自己的模型，来源为 USER-DEFINED，可尝试从配置文件解析 ModelType，解析不到则为空 +3. 时序大模型权重不随 AINode 打包，AINode 启动时自动下载，下载过程中为 LOADING +4. 下载成功转变为 ACTIVE，失败则变成 INACTIVE + +**示例** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 删除模型 @@ -388,6 +381,11 @@ Total line number = 4 其中结果集中每行的标签对应每24行数据为一组，输入该异常检测模型后的输出。 + +### 4.5 时序大模型导入步骤 + +AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](../AI-capability/TimeSeries-Large-Model.md) + ## 5. 权限管理 使用AINode相关的功能时，可以使用IoTDB本身的鉴权去做一个权限管理，用户只有在具备 USE_MODEL 权限时，才可以使用模型管理的相关功能。当使用推理功能时，用户需要有访问输入模型的SQL对应的源序列的权限。 @@ -412,10 +410,10 @@ Total line number = 4 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 向 IoTDB 中导入 ETT 数据集 +用户可以使用tools文件夹中的`import-data.sh` 向 IoTDB 中导入 ETT 数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../ETTh1.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/ETTh1.csv ``` #### 步骤二：模型导入 @@ -482,10 +480,10 @@ Total line number = 48 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 导入数据集 +用户可以使用tools文件夹中的`import-data.sh` 导入数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### 步骤二：模型导入 @@ -544,10 +542,10 @@ Airline Passengers一个时间序列数据集，该数据集记录了1949年至1 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 导入数据集 +用户可以使用tools文件夹中的`import-data.sh` 导入数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### 步骤二：模型推理 diff --git a/src/zh/UserGuide/Master/Tree/AI-capability/AINode_timecho.md b/src/zh/UserGuide/Master/Tree/AI-capability/AINode_timecho.md index 91f167168..1caf1e969 100644 --- a/src/zh/UserGuide/Master/Tree/AI-capability/AINode_timecho.md +++ b/src/zh/UserGuide/Master/Tree/AI-capability/AINode_timecho.md @@ -23,15 +23,17 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点，内置业界领先的自研时序大模型，如清华自研时序模型 Timer 系列，可通过标准 SQL 语句进行调用，实现时序数据的毫秒级实时推理，可支持时序趋势预测、缺失值填补、异常值检测等应用场景。 +> V2.0.5.1及以后版本支持 + 系统架构如下图所示： -![](/img/h4.png) +![](/img/AINode-0.png) 三种节点的职责如下： -- **ConfigNode**：负责保存和管理模型的元信息；负责分布式节点管理。 +- **ConfigNode**：负责分布式节点管理和负载均衡。 - **DataNode**：负责接收并解析用户的 SQL请求；负责存储时间序列数据；负责数据的预处理计算。 -- **AINode**：负责模型文件的导入创建以及模型推理。 +- **AINode**：负责时序模型的管理和使用。 ## 1. 优势特点 @@ -56,11 +58,11 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点 - **推理（Inference）**：使用创建的模型在指定时序数据上完成该模型适用的时序分析任务。 - **内置能力（Built-in）**：AINode 自带常见时序分析场景（例如预测与异常检测）的机器学习算法或自研模型。 -![](/img/h3.png) +![](/img/AINode-new.png) ## 3. 安装部署 -AINode 的部署可参考文档 [部署指导](../Deployment-and-Maintenance/AINode_Deployment_timecho.md#AINode-部署) 章节。 +AINode 的部署可参考文档 [AINode 部署](../Deployment-and-Maintenance/AINode_Deployment_timecho.md) 章节。 ## 4. 使用指导 @@ -86,7 +88,7 @@ SQL中参数的具体含义如下： - model_id：模型的全局唯一标识，不可重复。模型名称具备以下约束： - - 允许出现标识符 [ 0-9 a-z A-Z _ ] （字母，数字，下划线） + - 允许出现标识符 [ 0-9 a-z A-Z _ ]（字母，数字（非开头），下划线（非开头）） - 长度限制为2-64字符 - 大小写敏感 @@ -119,7 +121,7 @@ SQL中参数的具体含义如下： #### 示例 -在当前的example文件夹下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： +在[example 文件夹](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example)下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： ```YAML configs: @@ -142,12 +144,6 @@ attributes: # 可选项 为用户自定义的备注信息 IoTDB> create model dlinear_example using uri "file://./example" ``` -也可以从huggingFace上下载对应的模型文件进行注册 - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - SQL执行后会异步进行注册的流程，可以通过模型展示查看模型的注册状态（见模型展示章节），注册成功的耗时主要受到模型文件大小的影响。 模型注册完成后，就可以通过使用正常查询的方式调用具体函数，进行模型推理。 @@ -164,38 +160,39 @@ show models 除了直接展示所有模型的信息外，可以指定model id来查看某一具体模型的信息。模型展示的结果中包含如下信息： -| **ModelId** | **State** | **Configs** | **Attributes** | -| ------------ | ------------------------------------- | ---------------------------------------------- | -------------- | -| 模型唯一标识 | 模型注册状态(INACTIVE,LOADING,ACTIVE,TRAINING,FAILED,DROPPING) | InputShape, outputShapeInputTypes, outputTypes | 模型备注信息 | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|-----------|--------------|----------------| +| 模型ID | 模型类型 | 模型种类 | 模型状态 | -其中，State用于展示当前模型注册的状态，包含以下三个阶段 +- 模型状态机流转示意图如下 -- **INACTIVE**：模型处于不可用状态 -- **LOADING**：模型加载中 -- **ACTIVE**：模型处于可用状态 -- **TRAINING**：模型处于微调中状态 -- **FAILED**：模型微调失败状态 -- **DROPPING**:模型删除中状态 +![](/img/AINode-State.png) -#### 示例 +**说明：** + +1. 启动 AINode，show models 只能看到 BUILT-IN 模型 +2. 用户可导入自己的模型，来源为 USER-DEFINED，可尝试从配置文件解析 ModelType，解析不到则为空 +3. 时序大模型权重不随 AINode 打包，AINode 启动时自动下载，下载过程中为 LOADING +4. 下载成功转变为 ACTIVE，失败则变成 INACTIVE +5. 用户启动微调，正在训练的模型状态为 TRAINING，训练成功变为 ACTIVE，失败则是 FAILED + +**示例** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| +| sundialx_1| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_2| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_4| timer-sundial| FINE-TUNED| TRAINING| +| sundialx_5| timer-sundial| FINE-TUNED| FAILED| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 删除模型 @@ -453,7 +450,7 @@ IoTDB> show models ### 4.6 时序大模型导入步骤 -AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](../AI-capability/TimeSeries-Large-Model) +AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](../AI-capability/TimeSeries-Large-Model.md) ## 5. 权限管理 @@ -479,10 +476,10 @@ AINode 目前支持多种时序大模型，部署使用请参考[时序大模型 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 向 IoTDB 中导入 ETT 数据集 +用户可以使用tools文件夹中的`import-data.sh` 向 IoTDB 中导入 ETT 数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../ETTh1.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### 步骤二：模型导入 @@ -549,10 +546,10 @@ Total line number = 48 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 导入数据集 +用户可以使用tools文件夹中的`import-data.sh` 导入数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### 步骤二：模型导入 @@ -611,10 +608,10 @@ Airline Passengers一个时间序列数据集，该数据集记录了1949年至1 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 导入数据集 +用户可以使用tools文件夹中的`import-data.sh` 导入数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### 步骤二：模型推理 diff --git a/src/zh/UserGuide/latest-Table/AI-capability/AINode_apache.md b/src/zh/UserGuide/latest-Table/AI-capability/AINode_apache.md index 18eff8ec6..194cbd783 100644 --- a/src/zh/UserGuide/latest-Table/AI-capability/AINode_apache.md +++ b/src/zh/UserGuide/latest-Table/AI-capability/AINode_apache.md @@ -23,15 +23,17 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点，内置业界领先的自研时序大模型，如清华自研时序模型 Timer 系列，可通过标准 SQL 语句进行调用，实现时序数据的毫秒级实时推理，可支持时序趋势预测、缺失值填补、异常值检测等应用场景。 +> V2.0.5及以后版本支持 + 系统架构如下图所示： -![](/img/h4.png) +![](/img/AINode-0.png) 三种节点的职责如下： -- **ConfigNode**：负责保存和管理模型的元信息；负责分布式节点管理。 +- **ConfigNode**：负责分布式节点管理和负载均衡。 - **DataNode**：负责接收并解析用户的 SQL请求；负责存储时间序列数据；负责数据的预处理计算。 -- **AINode**：负责模型文件的导入创建以及模型推理。 +- **AINode**：负责时序模型的管理和使用。 ## 1. 优势特点 @@ -56,11 +58,11 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点 - **推理（Inference）**：使用创建的模型在指定时序数据上完成该模型适用的时序分析任务。 - **内置能力（Built-in）**：AINode 自带常见时序分析场景（例如预测与异常检测）的机器学习算法或自研模型。 -![](/img/h3.png) +![](/img/AINode-new.png) ## 3. 安装部署 -AINode 的部署可参考文档 [部署指导](../Deployment-and-Maintenance/AINode_Deployment_apache.md#ainode-部署) 章节。 +AINode 的部署可参考文档 [AINode 部署](../Deployment-and-Maintenance/AINode_Deployment_apache.md)。 ## 4. 使用指导 @@ -79,14 +81,14 @@ AINode 对时序模型提供了模型创建及删除功能，内置模型无需 下方为模型注册的SQL语法定义。 ```SQL -create model using uri +create model using uri ``` SQL中参数的具体含义如下： -- model_name：模型的全局唯一标识，不可重复。模型名称具备以下约束： +- model_id：模型的全局唯一标识，不可重复。模型名称具备以下约束： - - 允许出现标识符 [ 0-9 a-z A-Z _ ] （字母，数字，下划线） + - 允许出现标识符 [ 0-9 a-z A-Z _ ]（字母，数字（非开头），下划线（非开头）） - 长度限制为2-64字符 - 大小写敏感 @@ -119,7 +121,7 @@ SQL中参数的具体含义如下： #### 示例 -在当前的example文件夹下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： +在 [example 文件夹](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example)下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： ```YAML configs: @@ -142,12 +144,6 @@ attributes: # 可选项 为用户自定义的备注信息 IoTDB> create model dlinear_example using uri "file://./example" ``` -也可以从huggingFace上下载对应的模型文件进行注册 - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - SQL执行后会异步进行注册的流程，可以通过模型展示查看模型的注册状态（见模型展示章节），注册成功的耗时主要受到模型文件大小的影响。 模型注册完成后，就可以通过使用正常查询的方式调用具体函数，进行模型推理。 @@ -159,41 +155,38 @@ SQL执行后会异步进行注册的流程，可以通过模型展示查看模 ```SQL show models -show models +show models ``` 除了直接展示所有模型的信息外，可以指定model id来查看某一具体模型的信息。模型展示的结果中包含如下信息： -| **ModelId** | **State** | **Configs** | **Attributes** | -| ------------ | ------------------------------------- | ---------------------------------------------- | -------------- | -| 模型唯一标识 | 模型注册状态(INACTIVE,LOADING,ACTIVE,DROPPING) | InputShape, outputShapeInputTypes, outputTypes | 模型备注信息 | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|-----------|--------------|----------------| +| 模型ID | 模型类型 | 模型种类 | 模型状态 | -其中，State用于展示当前模型注册的状态，包含以下三个阶段 +- 模型状态机流转示意图如下 -- **INACTIVE**：模型处于不可用状态 -- **LOADING**：模型加载中状态 -- **ACTIVE**：模型处于可用状态 -- **DROPPING**:模型删除中状态 +![](/img/AINode-State-apache.png) -#### 示例 +**说明：** + +1. 启动 AINode，show models 只能看到 BUILT-IN 模型 +2. 用户可导入自己的模型，来源为 USER-DEFINED，可尝试从配置文件解析 ModelType，解析不到则为空 +3. 时序大模型权重不随 AINode 打包，AINode 启动时自动下载，下载过程中为 LOADING +4. 下载成功转变为 ACTIVE，失败则变成 INACTIVE + +**示例** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 删除模型 @@ -245,8 +238,8 @@ SELECT * FROM forecast( 1. forecast 函数默认对输入表中所有列进行预测（不包含time列和partition by 的列）。 2. forecast 函数对于输入数据无顺序性要求，默认对输入数据按照时间戳（由 TIMECOL 参数指定时间戳的列名）做升序排序后，再调用模型进行预测。 -3. 不同模型对于输入数据的行数要求不同，输入数据少于最低行数要求时，则报错；输入数据高于最大行数要求时，则自动截取最后的满足要求的行数。 - - 在当前的 AINdoe 内置模型中，仅_sundial对输入行数有限制。_sundial 目前支持的最大输入行数为 2880，超过 2880 时，会自动截取输入数据的最后 2880 行。 +3. 不同模型对于输入数据的行数要求不同，输入数据少于最低行数要求时会报错。 + - 在当前的 AINdoe 内置模型中，Timer-XL 模型至少需要输入 96 行数据，Timer-Sundial 模型至少需要输入 16 行数据。 4. forecast 函数的返回结果列包含 input 表的所有输入列，列的数据类型与原表列的数据类型一致。若 preserve_input= true，则还包含 is_input 列（表征当前行是否为输入行） - 目前只支持对 INT32、INT64、FLOAT、DOUBLE 进行预测，否则报错：The type of the column [%s] is [%s], only INT32, INT64, FLOAT, DOUBLE is allowed 5. output_start_time 和 output_interval 只会影响输出结果集的时间戳列生成，均为可选参数。 @@ -254,7 +247,12 @@ SELECT * FROM forecast( - output_interval = (输入数据的最后一个时间戳 - 输入数据的第一个时间戳) / n - 1， 默认为输入数据的采样间隔 - 第 N 个输出行的时间为 output_start_time + (N - 1) * output_interval -#### 示例 +**示例：需要提前创建数据库及表** + +```sql +create database etth +create table eg (hufl FLOAT FIELD, hull FLOAT FIELD, mufl FLOAT FIELD, mull FLOAT FIELD, lufl FLOAT FIELD, lull FLOAT FIELD, ot FLOAT FIELD) +``` 我们所使用的的测试集的数据为[ETTh1-tab](/img/ETTh1-tab.csv)。 @@ -314,7 +312,7 @@ It costs 1.615s ### 4.5 时序大模型导入步骤 -AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](https://timecho.com/docs/zh/UserGuide/latest/AI-capability/TimeSeries-Large-Model.html) +AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](../AI-capability/TimeSeries-Large-Model.md) ## 5 权限管理 diff --git a/src/zh/UserGuide/latest-Table/AI-capability/AINode_timecho.md b/src/zh/UserGuide/latest-Table/AI-capability/AINode_timecho.md index 562cd2f20..0e98d5d2f 100644 --- a/src/zh/UserGuide/latest-Table/AI-capability/AINode_timecho.md +++ b/src/zh/UserGuide/latest-Table/AI-capability/AINode_timecho.md @@ -23,15 +23,17 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点，内置业界领先的自研时序大模型，如清华自研时序模型 Timer 系列，可通过标准 SQL 语句进行调用，实现时序数据的毫秒级实时推理，可支持时序趋势预测、缺失值填补、异常值检测等应用场景。 +> V2.0.5.1及以后版本支持 + 系统架构如下图所示： -![](/img/h4.png) +![](/img/AINode-0.png) 三种节点的职责如下： -- **ConfigNode**：负责保存和管理模型的元信息；负责分布式节点管理。 +- **ConfigNode**：负责分布式节点管理和负载均衡。 - **DataNode**：负责接收并解析用户的 SQL请求；负责存储时间序列数据；负责数据的预处理计算。 -- **AINode**：负责模型文件的导入创建以及模型推理。 +- **AINode**：负责时序模型的管理和使用。 ## 1. 优势特点 @@ -56,11 +58,11 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点 - **推理（Inference）**：使用创建的模型在指定时序数据上完成该模型适用的时序分析任务。 - **内置能力（Built-in）**：AINode 自带常见时序分析场景（例如预测与异常检测）的机器学习算法或自研模型。 -![](/img/h3.png) +![](/img/AINode-new.png) ## 3. 安装部署 -AINode 的部署可参考文档 [部署指导](../Deployment-and-Maintenance/AINode_Deployment_apache.md#ainode-部署) 章节。 +AINode 的部署可参考文档 [AINode 部署](../Deployment-and-Maintenance/AINode_Deployment_timecho.md) 章节。 ## 4. 使用指导 @@ -79,14 +81,14 @@ AINode 对时序模型提供了模型创建及删除功能，内置模型无需 下方为模型注册的SQL语法定义。 ```SQL -create model using uri +create model using uri ``` SQL中参数的具体含义如下： -- model_name：模型的全局唯一标识，不可重复。模型名称具备以下约束： +- model_id：模型的全局唯一标识，不可重复。模型名称具备以下约束： - - 允许出现标识符 [ 0-9 a-z A-Z _ ] （字母，数字，下划线） + - 允许出现标识符 [ 0-9 a-z A-Z _ ]（字母，数字（非开头），下划线（非开头）） - 长度限制为2-64字符 - 大小写敏感 @@ -119,7 +121,7 @@ SQL中参数的具体含义如下： #### 示例 -在当前的example文件夹下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： +在 [example 文件夹](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example)下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： ```YAML configs: @@ -142,12 +144,6 @@ attributes: # 可选项 为用户自定义的备注信息 IoTDB> create model dlinear_example using uri "file://./example" ``` -也可以从huggingFace上下载对应的模型文件进行注册 - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - SQL执行后会异步进行注册的流程，可以通过模型展示查看模型的注册状态（见模型展示章节），注册成功的耗时主要受到模型文件大小的影响。 模型注册完成后，就可以通过使用正常查询的方式调用具体函数，进行模型推理。 @@ -159,43 +155,44 @@ SQL执行后会异步进行注册的流程，可以通过模型展示查看模 ```SQL show models -show models +show models ``` 除了直接展示所有模型的信息外，可以指定model id来查看某一具体模型的信息。模型展示的结果中包含如下信息： -| **ModelId** | **State** | **Configs** | **Attributes** | -| ------------ | ------------------------------------- | ---------------------------------------------- | -------------- | -| 模型唯一标识 | 模型注册状态(INACTIVE,LOADING,ACTIVE,TRAINING,FAILED,DROPPING) | InputShape, outputShapeInputTypes, outputTypes | 模型备注信息 | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|-----------|--------------|----------------| +| 模型ID | 模型类型 | 模型种类 | 模型状态 | -其中，State用于展示当前模型注册的状态，包含以下三个阶段 +- 模型状态机流转示意图如下 -- **INACTIVE**：模型处于不可用状态 -- **LOADING**：模型加载中 -- **ACTIVE**：模型处于可用状态 -- **TRAINING**：模型处于微调中状态 -- **FAILED**：模型微调失败状态 -- **DROPPING**:模型删除中状态 +![](/img/AINode-State.png) -#### 示例 +**说明：** + +1. 启动 AINode，show models 只能看到 BUILT-IN 模型 +2. 用户可导入自己的模型，来源为 USER-DEFINED，可尝试从配置文件解析 ModelType，解析不到则为空 +3. 时序大模型权重不随 AINode 打包，AINode 启动时自动下载，下载过程中为 LOADING +4. 下载成功转变为 ACTIVE，失败则变成 INACTIVE +5. 用户启动微调，正在训练的模型状态为 TRAINING，训练成功变为 ACTIVE，失败则是 FAILED + +**示例** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| +| sundialx_1| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_2| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_4| timer-sundial| FINE-TUNED| TRAINING| +| sundialx_5| timer-sundial| FINE-TUNED| FAILED| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 删除模型 @@ -247,8 +244,8 @@ SELECT * FROM forecast( 1. forecast 函数默认对输入表中所有列进行预测（不包含time列和partition by 的列）。 2. forecast 函数对于输入数据无顺序性要求，默认对输入数据按照时间戳（由 TIMECOL 参数指定时间戳的列名）做升序排序后，再调用模型进行预测。 -3. 不同模型对于输入数据的行数要求不同，输入数据少于最低行数要求时，则报错；输入数据高于最大行数要求时，则自动截取最后的满足要求的行数。 - - 在当前的 AINdoe 内置模型中，仅_sundial对输入行数有限制。_sundial 目前支持的最大输入行数为 2880，超过 2880 时，会自动截取输入数据的最后 2880 行。 +3. 不同模型对于输入数据的行数要求不同，输入数据少于最低行数要求时会报错。 + - 在当前的 AINdoe 内置模型中，Timer-XL 模型至少需要输入 96 行数据，Timer-Sundial 模型至少需要输入 16 行数据。 4. forecast 函数的返回结果列包含 input 表的所有输入列，列的数据类型与原表列的数据类型一致。若 preserve_input= true，则还包含 is_input 列（表征当前行是否为输入行） - 目前只支持对 INT32、INT64、FLOAT、DOUBLE 进行预测，否则报错：The type of the column [%s] is [%s], only INT32, INT64, FLOAT, DOUBLE is allowed 5. output_start_time 和 output_interval 只会影响输出结果集的时间戳列生成，均为可选参数。 @@ -256,7 +253,12 @@ SELECT * FROM forecast( - output_interval = (输入数据的最后一个时间戳 - 输入数据的第一个时间戳) / n - 1， 默认为输入数据的采样间隔 - 第 N 个输出行的时间为 output_start_time + (N - 1) * output_interval -#### 示例 +**示例：需要提前创建数据库及表** + +```sql +create database etth +create table eg (hufl FLOAT FIELD, hull FLOAT FIELD, mufl FLOAT FIELD, mull FLOAT FIELD, lufl FLOAT FIELD, lull FLOAT FIELD, ot FLOAT FIELD) +``` 我们所使用的的测试集的数据为[ETTh1-tab](/img/ETTh1-tab.csv)。 @@ -379,7 +381,7 @@ IoTDB> show models ### 4.6 时序大模型导入步骤 -AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](https://timecho.com/docs/zh/UserGuide/latest/AI-capability/TimeSeries-Large-Model.html) +AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](../AI-capability/TimeSeries-Large-Model.md) ## 5 权限管理 diff --git a/src/zh/UserGuide/latest/AI-capability/AINode_apache.md b/src/zh/UserGuide/latest/AI-capability/AINode_apache.md index 9d92fb365..b21a5d281 100644 --- a/src/zh/UserGuide/latest/AI-capability/AINode_apache.md +++ b/src/zh/UserGuide/latest/AI-capability/AINode_apache.md @@ -23,15 +23,17 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点，内置业界领先的自研时序大模型，如清华自研时序模型 Timer 系列，可通过标准 SQL 语句进行调用，实现时序数据的毫秒级实时推理，可支持时序趋势预测、缺失值填补、异常值检测等应用场景。 +> V2.0.5及以后版本支持 + 系统架构如下图所示： -![](/img/h4.png) +![](/img/AINode-0.png) 三种节点的职责如下： -- **ConfigNode**：负责保存和管理模型的元信息；负责分布式节点管理。 +- **ConfigNode**：负责分布式节点管理和负载均衡。 - **DataNode**：负责接收并解析用户的 SQL请求；负责存储时间序列数据；负责数据的预处理计算。 -- **AINode**：负责模型文件的导入创建以及模型推理。 +- **AINode**：负责时序模型的管理和使用。 ## 1. 优势特点 @@ -56,11 +58,11 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点 - **推理（Inference）**：使用创建的模型在指定时序数据上完成该模型适用的时序分析任务。 - **内置能力（Built-in）**：AINode 自带常见时序分析场景（例如预测与异常检测）的机器学习算法或自研模型。 -![](/img/h3.png) +![](/img/AINode-new.png) ## 3. 安装部署 -AINode 的部署可参考文档 [部署指导](../Deployment-and-Maintenance/AINode_Deployment_apache.md#ainode-部署) 章节。 +AINode 的部署可参考文档 [AINode 部署](../Deployment-and-Maintenance/AINode_Deployment_apache.md)。 ## 4. 使用指导 @@ -79,14 +81,14 @@ AINode 对时序模型提供了模型创建及删除功能，内置模型无需 下方为模型注册的SQL语法定义。 ```SQL -create model using uri +create model using uri ``` SQL中参数的具体含义如下： -- model_name：模型的全局唯一标识，不可重复。模型名称具备以下约束： +- model_id：模型的全局唯一标识，不可重复。模型名称具备以下约束： - - 允许出现标识符 [ 0-9 a-z A-Z _ ] （字母，数字，下划线） + - 允许出现标识符 [ 0-9 a-z A-Z _ ]（字母，数字（非开头），下划线（非开头）） - 长度限制为2-64字符 - 大小写敏感 @@ -119,7 +121,7 @@ SQL中参数的具体含义如下： #### 示例 -在当前的example文件夹下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： +在[example 文件夹](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example)下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： ```YAML configs: @@ -142,12 +144,6 @@ attributes: # 可选项 为用户自定义的备注信息 IoTDB> create model dlinear_example using uri "file://./example" ``` -也可以从huggingFace上下载对应的模型文件进行注册 - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - SQL执行后会异步进行注册的流程，可以通过模型展示查看模型的注册状态（见模型展示章节），注册成功的耗时主要受到模型文件大小的影响。 模型注册完成后，就可以通过使用正常查询的方式调用具体函数，进行模型推理。 @@ -159,41 +155,38 @@ SQL执行后会异步进行注册的流程，可以通过模型展示查看模 ```SQL show models -show models +show models ``` 除了直接展示所有模型的信息外，可以指定model id来查看某一具体模型的信息。模型展示的结果中包含如下信息： -| **ModelId** | **State** | **Configs** | **Attributes** | -| ------------ | ------------------------------------- | ---------------------------------------------- | -------------- | -| 模型唯一标识 | 模型注册状态(INACTIVE,LOADING,ACTIVE,DROPPING) | InputShape, outputShapeInputTypes, outputTypes | 模型备注信息 | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|-----------|--------------|----------------| +| 模型ID | 模型类型 | 模型种类 | 模型状态 | -其中，State用于展示当前模型注册的状态，包含以下三个阶段 +- 模型状态机流转示意图如下 -- **INACTIVE**：模型处于不可用状态 -- **LOADING**：模型加载中状态 -- **ACTIVE**：模型处于可用状态 -- **DROPPING**:模型删除中状态 +![](/img/AINode-State-apache.png) -#### 示例 +**说明：** + +1. 启动 AINode，show models 只能看到 BUILT-IN 模型 +2. 用户可导入自己的模型，来源为 USER-DEFINED，可尝试从配置文件解析 ModelType，解析不到则为空 +3. 时序大模型权重不随 AINode 打包，AINode 启动时自动下载，下载过程中为 LOADING +4. 下载成功转变为 ACTIVE，失败则变成 INACTIVE + +**示例** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 删除模型 @@ -388,6 +381,11 @@ Total line number = 4 其中结果集中每行的标签对应每24行数据为一组，输入该异常检测模型后的输出。 + +### 4.5 时序大模型导入步骤 + +AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](../AI-capability/TimeSeries-Large-Model.md) + ## 5. 权限管理 使用AINode相关的功能时，可以使用IoTDB本身的鉴权去做一个权限管理，用户只有在具备 USE_MODEL 权限时，才可以使用模型管理的相关功能。当使用推理功能时，用户需要有访问输入模型的SQL对应的源序列的权限。 @@ -412,10 +410,10 @@ Total line number = 4 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 向 IoTDB 中导入 ETT 数据集 +用户可以使用tools文件夹中的`import-data.sh` 向 IoTDB 中导入 ETT 数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../ETTh1.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/ETTh1.csv ``` #### 步骤二：模型导入 @@ -482,10 +480,10 @@ Total line number = 48 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 导入数据集 +用户可以使用tools文件夹中的`import-data.sh` 导入数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### 步骤二：模型导入 @@ -544,10 +542,10 @@ Airline Passengers一个时间序列数据集，该数据集记录了1949年至1 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 导入数据集 +用户可以使用tools文件夹中的`import-data.sh` 导入数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### 步骤二：模型推理 diff --git a/src/zh/UserGuide/latest/AI-capability/AINode_timecho.md b/src/zh/UserGuide/latest/AI-capability/AINode_timecho.md index 91f167168..08bd7f078 100644 --- a/src/zh/UserGuide/latest/AI-capability/AINode_timecho.md +++ b/src/zh/UserGuide/latest/AI-capability/AINode_timecho.md @@ -23,15 +23,17 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点，内置业界领先的自研时序大模型，如清华自研时序模型 Timer 系列，可通过标准 SQL 语句进行调用，实现时序数据的毫秒级实时推理，可支持时序趋势预测、缺失值填补、异常值检测等应用场景。 +> V2.0.5.1及以后版本支持 + 系统架构如下图所示： -![](/img/h4.png) +![](/img/AINode-0.png) 三种节点的职责如下： -- **ConfigNode**：负责保存和管理模型的元信息；负责分布式节点管理。 +- **ConfigNode**：负责分布式节点管理和负载均衡。 - **DataNode**：负责接收并解析用户的 SQL请求；负责存储时间序列数据；负责数据的预处理计算。 -- **AINode**：负责模型文件的导入创建以及模型推理。 +- **AINode**：负责时序模型的管理和使用。 ## 1. 优势特点 @@ -56,11 +58,11 @@ AINode 是支持时序相关模型注册、管理、调用的 IoTDB 原生节点 - **推理（Inference）**：使用创建的模型在指定时序数据上完成该模型适用的时序分析任务。 - **内置能力（Built-in）**：AINode 自带常见时序分析场景（例如预测与异常检测）的机器学习算法或自研模型。 -![](/img/h3.png) +![](/img/AINode-new.png) ## 3. 安装部署 -AINode 的部署可参考文档 [部署指导](../Deployment-and-Maintenance/AINode_Deployment_timecho.md#AINode-部署) 章节。 +AINode 的部署可参考文档 [AINode 部署](../Deployment-and-Maintenance/AINode_Deployment_timecho.md) 章节。 ## 4. 使用指导 @@ -86,7 +88,7 @@ SQL中参数的具体含义如下： - model_id：模型的全局唯一标识，不可重复。模型名称具备以下约束： - - 允许出现标识符 [ 0-9 a-z A-Z _ ] （字母，数字，下划线） + - 允许出现标识符 [ 0-9 a-z A-Z _ ]（字母，数字（非开头），下划线（非开头）） - 长度限制为2-64字符 - 大小写敏感 @@ -119,7 +121,7 @@ SQL中参数的具体含义如下： #### 示例 -在当前的example文件夹下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： +在[example 文件夹](https://github.com/apache/iotdb/tree/master/integration-test/src/test/resources/ainode-example)下，包含model.pt和config.yaml文件，model.pt为训练得到，config.yaml的内容如下： ```YAML configs: @@ -142,12 +144,6 @@ attributes: # 可选项 为用户自定义的备注信息 IoTDB> create model dlinear_example using uri "file://./example" ``` -也可以从huggingFace上下载对应的模型文件进行注册 - -```SQL -IoTDB> create model dlinear_example using uri "https://huggingface.co/google/timesfm-2.0-500m-pytorch" -``` - SQL执行后会异步进行注册的流程，可以通过模型展示查看模型的注册状态（见模型展示章节），注册成功的耗时主要受到模型文件大小的影响。 模型注册完成后，就可以通过使用正常查询的方式调用具体函数，进行模型推理。 @@ -164,38 +160,39 @@ show models 除了直接展示所有模型的信息外，可以指定model id来查看某一具体模型的信息。模型展示的结果中包含如下信息： -| **ModelId** | **State** | **Configs** | **Attributes** | -| ------------ | ------------------------------------- | ---------------------------------------------- | -------------- | -| 模型唯一标识 | 模型注册状态(INACTIVE,LOADING,ACTIVE,TRAINING,FAILED,DROPPING) | InputShape, outputShapeInputTypes, outputTypes | 模型备注信息 | +| **ModelId** | **ModelType** | **Category** | **State** | +|-------------|-----------|--------------|----------------| +| 模型ID | 模型类型 | 模型种类 | 模型状态 | -其中，State用于展示当前模型注册的状态，包含以下三个阶段 +- 模型状态机流转示意图如下 -- **INACTIVE**：模型处于不可用状态 -- **LOADING**：模型加载中 -- **ACTIVE**：模型处于可用状态 -- **TRAINING**：模型处于微调中状态 -- **FAILED**：模型微调失败状态 -- **DROPPING**:模型删除中状态 +![](/img/AINode-State.png) -#### 示例 +**说明：** + +1. 启动 AINode，show models 只能看到 BUILT-IN 模型 +2. 用户可导入自己的模型，来源为 USER-DEFINED，可尝试从配置文件解析 ModelType，解析不到则为空 +3. 时序大模型权重不随 AINode 打包，AINode 启动时自动下载，下载过程中为 LOADING +4. 下载成功转变为 ACTIVE，失败则变成 INACTIVE +5. 用户启动微调，正在训练的模型状态为 TRAINING，训练成功变为 ACTIVE，失败则是 FAILED + +**示例** ```SQL IoTDB> show models - -+---------------------+--------------------+--------+--------+ -| ModelId| ModelType|Category| State| -+---------------------+--------------------+--------+--------+ -| arima| Arima|BUILT-IN| ACTIVE| -| holtwinters| HoltWinters|BUILT-IN| ACTIVE| -|exponential_smoothing|ExponentialSmoothing|BUILT-IN| ACTIVE| -| naive_forecaster| NaiveForecaster|BUILT-IN| ACTIVE| -| stl_forecaster| StlForecaster|BUILT-IN| ACTIVE| -| gaussian_hmm| GaussianHmm|BUILT-IN| ACTIVE| -| gmm_hmm| GmmHmm|BUILT-IN| ACTIVE| -| stray| Stray|BUILT-IN| ACTIVE| -| timer_xl| Timer-XL|BUILT-IN| ACTIVE| -| sundial| Timer-Sundial|BUILT-IN| ACTIVE| -+---------------------+--------------------+--------+--------+ ++---------------------+--------------------+--------------+---------+ +| ModelId| ModelType| Category| State| ++---------------------+--------------------+--------------+---------+ +| arima| arima| BUILT-IN| ACTIVE| +| custom| | USER-DEFINED| ACTIVE| +| timerxl| timer-xl| BUILT-IN| LOADING| +| sundial| timer-sundial| BUILT-IN| ACTIVE| +| sundialx_1| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_2| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx| timer-sundial| FINE-TUNED| ACTIVE| +| sundialx_4| timer-sundial| FINE-TUNED| TRAINING| +| sundialx_5| timer-sundial| FINE-TUNED| FAILED| ++---------------------+--------------------+--------------+---------+ ``` ### 4.3 删除模型 @@ -453,7 +450,7 @@ IoTDB> show models ### 4.6 时序大模型导入步骤 -AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](../AI-capability/TimeSeries-Large-Model) +AINode 目前支持多种时序大模型，部署使用请参考[时序大模型](../AI-capability/TimeSeries-Large-Model.md) ## 5. 权限管理 @@ -479,10 +476,10 @@ AINode 目前支持多种时序大模型，部署使用请参考[时序大模型 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 向 IoTDB 中导入 ETT 数据集 +用户可以使用tools文件夹中的`import-data.sh` 向 IoTDB 中导入 ETT 数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../ETTh1.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/ETTh1.csv ``` #### 步骤二：模型导入 @@ -549,10 +546,10 @@ Total line number = 48 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 导入数据集 +用户可以使用tools文件夹中的`import-data.sh` 导入数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### 步骤二：模型导入 @@ -611,10 +608,10 @@ Airline Passengers一个时间序列数据集，该数据集记录了1949年至1 #### 步骤一：数据导入 -用户可以使用tools文件夹中的`import-csv.sh` 导入数据集 +用户可以使用tools文件夹中的`import-data.sh` 导入数据集 ```Bash -bash ./import-csv.sh -h 127.0.0.1 -p 6667 -u root -pw root -f ../../data.csv +bash ./import-data.sh -ft csv -h 127.0.0.1 -p 6667 -u root -pw root -s /path/data.csv ``` #### 步骤二：模型推理