ml concepts

[liezeltamon]

• loss function vs. cost function - loss is error for a data point and cost is the summary of that errors from data points

• momentum - gradient descent has enough momentum to skip local minima

• Transform test data using mean and sd used for transforming training data

• to deal with imbalanced labels - either 1) under sample dominant labels or 2) up sample rarer labels - which to do depends maybe on the number of rarer labels i.e. two rare so up sample rarer labels over under sample dominant labels bringing down to label of rarer labels

• Target variable needing scaling? - "unscaled target variables on regression problems can result in exploding gradients causing the learning process to fail", https://stats.stackexchange.com/questions/111467/is-it-necessary-to-scale-the-target-value-in-addition-to-scaling-features-for-re

• Dealing with correlated features