[User Story] A novice user wants to understand the precondition for using the provided AI solutions on their data

[HuangYiran]

idea: two different condition arts should be considered:

• task
  -- if it’s one of the problem in the demo above,
  -- be sure that the features can be influenced

• data
  -- have a target
  -- have enough data (data size)
  -- should bring the data in the right form

[HuangYiran]

Methods: provide a notebook which,

• asks our custom to check the tasks that datafactory can deal with (with a link)
• includes a data format datachecker. It takes the data and target name as input and outputs whether the given data is in the right format or what is the problem of the given data.

The datachecker should include following function:

• check if the given data in the right form. For the relational dataset, the given data should contain only one csv file with: (i) the last column is the target column; (ii) all the other columns are the features; (iii) each row represents a data point and no relation between two points (two rows are iid? → exclude time series);
• feature and label in the same row?
• check the type of inputs (asking for customer input, when confusing, e.g., id is recognized as nummric or price is recognized as characteric)
• check the number of different value of each feature
• check consistency (e.g. no text in numerical feature, must be careful of ID-numbers)
• check the correlation between features (Multicollinearity check)
• check the label distribution (balance)

[HuangYiran]

Acceptance criterion:

• fulfill the function mentioned above, concretly:
  -- ?
• do it automatically
• if error, has clear feedback and suggesion

[HuangYiran]

collect situation:

• webstore: volumn prediction
• machine sensor:

don't forget the split sign of the data - autocheck, one column only
is index column included - atuocheck, correlation between first column and range (not a good method, better get the information from user)