Satyajit Das

@satyajit

active 1 day, 7 hours ago
  • Satyajit Das posted an update in the group Group logo of Machine LearningMachine Learning 1 week, 4 days ago

    Why Decisions tree is robust to outliers?
    For this you need to understand the Decisions tree algorithm and how each node splits the data space into pieces based on value of a feature and not on based on the distance between the values.

    The thumb rule that you need to follow is that any algorithms that computes the distance or assumes the normality, for them we need to scale the Features.
    for examples.

    The models where we use the below algos which measures the distances between the values.

    k-nearest neighbors –> Calculates the Euclidean Distance.
    PCA –> Calculates the variance between the features and skews towards the High Magnitude Features.
    Gradient Descents –> This is because “θ” will descend quickly on small ranges and slowly on large ranges also depends on distances measures.
    But Tree based models are not distance based models they depends on the values/weights of the Features not the ranges of the features that is why features scaling may not have much effect on these algorithms.
    similarly with “Naive Bayes”, “Linear Discriminant Analysis” and ofcourse “Random forest”.
    and that is why we say that “Tree based algos are robust to outliers” you must have heard this statements during interviews.

    So, your answer should be :-

    Because decision trees divide items by current values, and not on based on the absolute magnitude.

    http://www.stats.ox.ac.uk/~flaxman/HT17_lecture13.pdf
    https://medium.com/greyatom/why-how-and-when-to-scale-your-features-4b30ab09db5e#:~:text=Tree%20based%20models%20are%20not,weights%20to%20the%20features%20accordingly.

Skip to toolbar