- Here is my answer:
C4.5 is an algorithm that naturally applied on ID3 as a post process to prune the ID3 decision tree. After transforming the tree to if-then rules, C4.5 prune the rules. The difference between normal tree pruning, and what C4.5 does is that, one attribute can be removed from a node but it still exists in children nodes.
- Advantages of C4.5 over other Decision Tree systems:
The algorithm inherently employs Single Pass Pruning Process to mitigate overfitting.
It can work with both Discrete and Continuous Data
C4.5 can handle the issue of incomplete data very well
- However, Sklearn software uses CART algorithm instead of C4.5, Here is the explanation available in their website:
“C4.5 is the successor to ID3 and removed the restriction that features must be categorical by dynamically defining a discrete attribute (based on numerical variables) that partitions the continuous attribute value into a discrete set of intervals. C4.5 converts the trained trees (i.e. the output of the ID3 algorithm) into sets of if-then rules. This accuracy of each rule is then evaluated to determine the order in which they should be applied. Pruning is done by removing a rule’s precondition if the accuracy of the rule improves without it.
CART (Classification and Regression Trees) is very similar to C4.5, but it differs in that it supports numerical target variables (regression) and does not compute rule sets. CART constructs binary trees using the feature and threshold that yield the largest information gain at each node.
Scikit-learn uses an optimized version of the CART algorithm; however, Scikit-learn implementation does not support categorical variables for now.”
https://scikit-learn.org/stable/modules/tree.html#tree-algorithms-id3-c4-5-c5-0-and-cart