Adding_model.md

Adding Your Own Design

💡 A guide to add your own mode.

Adding your own model

To add your own in-network ML algorithm, there are two things to be considered, which are P4 file generation and M/A table entries generation. The main procedure can be generalized into creating a <model_name> folder under the directory ./src/models. Then, in the model folder, create a variation folder, e.g., Type_n. In the Type_n folder, create dedicated_p4.py file for model training and conversion, and table_generator.py for generating the model related tables. The following is a detailed explanation.

The table_generator.py file

In table_generator.py, there are two compulsory functions to be written, run_model(*) & test_tables(*):

run_model(*) function

• The overview of run_model(*) function

  def run_model(train_X, train_y, test_X, test_y, used_features):
      # 1. Model Training
      # 2. Model Testing
      # 3. Model Conversion (M/A table entries generation).
      return sklearn_test_y
  

• The input of run_model(*) function

   train_X # data frame/ndarray/list
           # training data
           # generate from <dataset_name>_dataset.py 
  
   train_y # data frame/ndarray/list
           # the training labels
           # generate from <dataset_name>_dataset.py 
  
   test_X # data frame/ndarray/list
          # testing data
          # generate from <dataset_name>_dataset.py 
  
   test_y # data frame/ndarray/list
          # testing labels
          # generate from <dataset_name>_dataset.py 
  
   used_features # int, 
                 # number of used features
                 # input in Planter.py 
  

• The output (return) of run_model(*) function

   sklearn_test_y # data frame/ndarray/list
                  # sklearn (baseline) inference result (output labels)
  

test_tables(*) function

• The overview of test_tables(*) function

  def test_tables(sklearn_test_y, test_X, test_y):
      # 1. Test the generated M/A table entries based on the pipeline
      #    logic.
      return 
  

• The input of test_tables(*) function

   sklearn_test_y # data frame/ndarray/list
                  # sklearn (baseline) inference result (output labels)
                  # generated by funtion test_tables(*)
  
   test_X # data frame/ndarray/list
          # testing data
          # generate from <dataset_name>_dataset.py 
  
   test_y # data frame/ndarray/list
          # testing labels
          # generate from <dataset_name>_dataset.py
  

The dedicated_p4.py file

In dedicated_p4.py, we can define functions to write M/A pipeline logic into P4 files. This file requires some key functions:

load_config(*) function

• The overview of load_config(*) function

  def load_config(fname):
      # 1. load the config files.
      return config, Planter_config
  

• The input of load_config(*) function

   fname # str
         # config file directory
         # generate from p4_generator.py 
  

• The output (return) of load_config(*) function

   config # dict
          # P4 generator's configs - Planter_config['p4 config']
   
   Planter_config # dict
                  # Planter's standard configs
  

add_model_intro(*) function

• The overview of add_model_intro(*) function

  def add_model_intro(fname, config):
      # 1. Write intro (header in comment format) in P4 files.
      with open(fname, 'a') as intro:
         intro.write("// ...\n")
      return 
  

• The input of add_model_intro(*) function

   fname # str
         # P4 file directory
         # generate from p4_generator.py 
  
   config # dict
          # P4 generator's configs - Planter_config['p4 config']
          # generated by function load_config(*)
  

separate_metadata(*) function

• The overview of separate_metadata(*) function

  def separate_metadata(fname, config):
      # 1. Write model related meta data.
      with open(fname, 'a') as file:
         file.write("...\n")
      return 
  

• The input of separate_metadata(*) function

   fname # str
         # P4 file directory
         # generate from p4_generator.py 
  
   config # dict
          # P4 generator's configs - Planter_config['p4 config']
          # generated by function load_config(*)
  

separate_logics(*) function

• The overview of separate_logics(*) function

  def separate_logics(fname, config):
      # 1. Write model related apply() logic in ingress pipeline.
      with open(fname, 'a') as file:
         file.write("...\n")
      return 
  

• The input of separate_logics(*) function

   fname # str
         # P4 file directory
         # generate from p4_generator.py 
  
   config # dict
          # P4 generator's configs - Planter_config['p4 config']
          # generated by function load_config(*)
  

separate_tables(*) function

• The overview of separate_tables(*) function

  def separate_tables(fname, config):
      # 1. Write model related tables, actions, and definitions in ingress.
      with open(fname, 'a') as file:
         file.write("...\n")
      return 
  

• The input of separate_tables(*) function

   fname # str
         # P4 file directory
         # generate from p4_generator.py 
  
   config # dict
          # P4 generator's configs - Planter_config['p4 config']
          # generated by function load_config(*)
  

create_load_tables(*) function

• The overview of create_load_tables(*) function

  def create_load_tables(fname, fjson, config, Planter_config, file_name):
      # 1. Prepare load data files for bfrt.
      #    varies from different targets
      return 
  

• The input of create_load_tables(*) function

   fname # str
         # P4 file directory
         # generate from p4_generator.py 
  
   fjson # str
         # M/A table file directory
         # generate from p4_generator.py 
  
   config # dict
          # P4 generator's configs - Planter_config['p4 config']
          # generated by function load_config(*)
  
   Planter_config # dict
                  # Planter's standard configs
                  # generate from anywhere 
  
   file_name # str
             # P4 file name
             # generate from p4_generator.py