Graphical representation of using the decrease in Gini Index to determine the most important variables. 

University of Michigan - Ann Arbor

Cite helpful visuals for helping explain data science concepts

Helpful data science images

While bagging improves the prediction accuracy of decision trees, it reduces model interpretability because the ensemble can no longer be represented as a single tree. Variable importance measures address this by calculating the significance of predictors across all $$B$$ bagged trees. For regression trees, a variable's importance is determined by the total amount the Residual Sum of Squares (RSS) is decreased due to splits over that predictor, averaged over all $$B$$ trees. For classification trees, importance is measured by the total decrease in the Gini index caused by splits over the specific predictor, averaged across all $$B$$ trees. A larger average decrease indicates a more important predictor.

Variable Importance Measures in Bagged Decision Trees

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

https://www.textbook.ds100.org/ch/15/bias_cv.html



Image Reference: Train Test Error

Image Reference: Linear vs Logistic Regression

Link: [kaggle.com](https://www.kaggle.com/dansbecker/using-categorical-data-with-one-hot-encoding) 

One hot encoding picture

Link: [runestone.academy](https://runestone.academy/runestone/books/published/httlads/_images/DSPipeline1.svg) 

Image Reference: Data Science Pipeline

https://manisha-sirsat.blogspot.com/2019/04/confusion-matrix.html

Image Reference: Confusion Matrix

https://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5

ROC & AUC

https://www.intermediait.com/blog/2019/11/18/evaluating-model-performance-generalization-bias-variance-tradeoff-and-overfitting-vs-underfitting/

Workflow to reduce BIAS and VARIANCE

Recent Developments on Statistical and Neural Network Tools Focusing on Biodiesel Quality - Scientific Figure on ResearchGate. Available from: https://www.researchgate.net/figure/TOPOLOGY-OF-A-MULTILAYER-PERCEPTRON-NEURAL-NETWORK_fig1_274240858 [accessed 12 Feb, 2021]

Topology of a multilayer perceptron neural network

https://towardsdatascience.com/understanding-learning-rates-and-how-it-improves-performance-in-deep-learning-d0d4059c1c10

Learning Rate in Deep Learning

PART OF SPEECH TAGGER FOR ARABIC TEXT BASED SUPPORT VECTOR MACHINES: A REVIEW - Scientific Figure on ResearchGate. Available from: https://www.researchgate.net/figure/Methods-for-implementing-POS-tagger_fig1_331684946

Methods for implementing POS tagger


https://www.sentisum.com/library/nlp-and-text-mining

NLP and other relationships

https://medium.com/@tomyuz/a-sentiment-analysis-approach-to-predicting-stock-returns-d5ca8b75a42

Sentiment Analysis Process

Image Reference: Interpreting bias and variance

Gini index measures the node purity - total variance across all the K classes. A small value of Gini index implies that one class dominates the region.
$$
G=\sum_{k=1}^{K} \hat{p}_{m k}\left(1-\hat{p}_{m k}\right)
$$
where $p_{mk}$ denotes the proportion of training observations in the $m$th region that are from the $k$th class.
The index measures total variance across the $$K$$ classes. The index is minimized when $$p_{mk}$$ is close to zero or 1. 


Learn Before

Related