Variance Inflation Factor in Python

As mentioned by others and in this post by Josef Perktold, the function's author, variance_inflation_factor expects the presence of a constant in the matrix of explanatory variables. One can use add_constant from statsmodels to add the required constant to the dataframe before passing its values to the function.

from statsmodels.stats.outliers_influence import variance_inflation_factorfrom statsmodels.tools.tools import add_constantdf = pd.DataFrame(    {'a': [1, 1, 2, 3, 4],     'b': [2, 2, 3, 2, 1],     'c': [4, 6, 7, 8, 9],     'd': [4, 3, 4, 5, 4]})X = add_constant(df)>>> pd.Series([variance_inflation_factor(X.values, i)                for i in range(X.shape[1])],               index=X.columns)const    136.875a         22.950b          3.000c         12.950d          3.000dtype: float64

I believe you could also add the constant to the right most column of the dataframe using assign:

X = df.assign(const=1)>>> pd.Series([variance_inflation_factor(X.values, i)                for i in range(X.shape[1])],               index=X.columns)a         22.950b          3.000c         12.950d          3.000const    136.875dtype: float64

The source code itself is rather concise:

def variance_inflation_factor(exog, exog_idx):    """    exog : ndarray, (nobs, k_vars)        design matrix with all explanatory variables, as for example used in        regression    exog_idx : int        index of the exogenous variable in the columns of exog    """    k_vars = exog.shape[1]    x_i = exog[:, exog_idx]    mask = np.arange(k_vars) != exog_idx    x_noti = exog[:, mask]    r_squared_i = OLS(x_i, x_noti).fit().rsquared    vif = 1. / (1. - r_squared_i)    return vif

It is also rather simple to modify the code to return all of the VIFs as a series:

from statsmodels.regression.linear_model import OLSfrom statsmodels.tools.tools import add_constantdef variance_inflation_factors(exog_df):    '''    Parameters    ----------    exog_df : dataframe, (nobs, k_vars)        design matrix with all explanatory variables, as for example used in        regression.    Returns    -------    vif : Series        variance inflation factors    '''    exog_df = add_constant(exog_df)    vifs = pd.Series(        [1 / (1. - OLS(exog_df[col].values,                        exog_df.loc[:, exog_df.columns != col].values).fit().rsquared)          for col in exog_df],        index=exog_df.columns,        name='VIF'    )    return vifs>>> variance_inflation_factors(df)const    136.875a         22.950b          3.000c         12.950Name: VIF, dtype: float64

Per the solution of @T_T, one can also simply do the following:

vifs = pd.Series(np.linalg.inv(df.corr().to_numpy()).diagonal(),                  index=df.columns,                  name='VIF')

I believe the reason for this is due to a difference in Python's OLS. OLS, which is used in the python variance inflation factor calculation, does not add an intercept by default. You definitely want an intercept in there however.

What you'd want to do is add one more column to your matrix, ck, filled with ones to represent a constant. This will be the intercept term of the equation. Once this is done, your values should match out properly.

Edited: replaced zeroes with ones

For future comers to this thread (like me):

import numpy as npimport scipy as spa = [1, 1, 2, 3, 4]b = [2, 2, 3, 2, 1]c = [4, 6, 7, 8, 9]d = [4, 3, 4, 5, 4]ck = np.column_stack([a, b, c, d])cc = sp.corrcoef(ck, rowvar=False)VIF = np.linalg.inv(cc)VIF.diagonal()

This code gives

array([22.95,  3.  , 12.95,  3.  ])

[EDIT]

In response to a comment, I tried to use DataFrame as much as possible (numpy is required to invert a matrix).

import pandas as pdimport numpy as npa = [1, 1, 2, 3, 4]b = [2, 2, 3, 2, 1]c = [4, 6, 7, 8, 9]d = [4, 3, 4, 5, 4]df = pd.DataFrame({'a':a,'b':b,'c':c,'d':d})df_cor = df.corr()pd.DataFrame(np.linalg.inv(df.corr().values), index = df_cor.index, columns=df_cor.columns)

The code gives

       a            b           c           da   22.950000   6.453681    -16.301917  -6.453681b   6.453681    3.000000    -4.080441   -2.000000c   -16.301917  -4.080441   12.950000   4.080441d   -6.453681   -2.000000   4.080441    3.000000

The diagonal elements give VIF.