ML EXPLAINED

Articles

  • Importance of VIF in Linear Regression

    What is VIF

    Variance Inflation Factor (VIF) determines the multicollinearity amongst the independent variables (predictors). Multicollinearity is when there is a high correlation between your predictor variables, usually 0.8 or higher. This can adversely affect your regression analysis.

    How is it calculated?

    VIF of a predictor variable is calculated by regressing it against all other predictor variables. This gives the R2 value which can be plugged into this formula

    This will give the VIF value of a predictor.

    • VIF = 1, not correlated
    • VIF < 5, slightly correlated
    • VIF > 5, highly correlated

    These values are just guidelines and how high acceptable VIF values are depends on the problem statement.

    If you don’t want to use VIF and have very few predictor variables, one can plot a correlation matrix and remove the highly correlated variables.

    You might also wonder why do we calculate the p-value of predictor variables in Linear regression. Find out why here.

  • Null Hypothesis of Linear Regression Explained

    Ever wondered why we look for p-value less than 0.05 for the coefficients when looking at the linear regression results.

    Let’s quickly recap the basics of linear regression. In Linear Regression we try to estimate a best fit line for given data points. In case we have only one predictor variable and a target the linear equation will look something like

    Y = A + Bx

    Here A being the intercept and B being the slope or coefficient.

    The null hypothesis for linear regression is that B=0 and the alternate hypothesis is that B != 0.

    This is the reason why we look for p-value < 0.05 to reject the null hypothesis and establish that there exists a relationship between the target and the predictor variable.