We generally find this question while starting up new project or we want to compare some algorithms to discriminate them (discrimination helps understand things better sometimes).
Although there is no definitive answer to this, I am writing here summaries from some posts. [0]
Factors to consider
-
Accuracy
- Most of people focus on accuracy but it practically it is not the only
-
Training time
- Naive Bayes and logistic regression are much faster than boosting or neural nets
-
Linearity
- LR and SVM are suitable when classes are linearly seperatble
- Of course SVM bypasses it via kernel trick but still not as much complex decision boundary as nueral nets
- Despite the risk of non linearity in data linear algorithms tends to work well in practice and are often used as starting point
- LR and SVM are suitable when classes are linearly seperatble
-
Number of parameters
- Parameters does affect training time and accuracy
- More parameters helps learning complex function, however it requires more data to prevent over-fitting
-
No of features
- When data point are not enough for no of features (text, NLP) SVM works well
Notes
- Try out linear/logistic regression, SVM first when you most dependent variables are numeric.
- SVM
- SVM suites more when no of data points are less for given no of features.
- SVM is linear classifier only. It just uses kernel trick to project linearly inseparable data on high dimension.
- SVM is solved by mathematical optimization problem unlike nueral nets. Hence tends to be bit faster.
- What is the difference between LR and SVM?
- LR has linear decision boundary while SVM can have non linear decision boundary.
- Reinforcement learning
- Analyses and optimized behavior of agent, (via feedback from environment)
- They try to discover different actions to maximize reward
- Trial-error and delayed reward distinguishes reinforcement learning from other ML algorithms
References
[1] : https://docs.microsoft.com/en-us/azure/machine-learning/studio/algorithm-choice
Data Stories 