Linear Regression in Python using Statsmodels

In this guide, I’ll show you how to perform linear regression in Python using statsmodels. I’ll use a simple example about the stock market to demonstrate this concept.

Here are the topics to be covered:

  • Background about linear regression
  • Review of an example with the full dataset
  • Review of the Python code
  • Interpretation of the regression results
  • Making predictions based on the regression results

About Linear Regression

Linear regression is used as a predictive model that assumes a linear relationship between the dependent variable (which is the variable we are trying to predict/estimate) and the independent variable/s (input variable/s used in the prediction).

Under Simple Linear Regression, only one independent/input variable is used to predict the dependent variable. It has the following structure:

Y = C + M*X

  • Y = Dependent variable (output/outcome/prediction/estimation)
  • C = Constant (Y-Intercept)
  • M = Slope of the regression line (the effect that X has on Y)
  • X = Independent variable (input variable used in the prediction of Y)

In reality, a relationship may exist between the dependent variable and multiple independent variables. For these types of models (assuming linearity), we can use Multiple Linear Regression with the following structure:

Y = C + M1*X1 + M2*X2 + …

An Example (with the Dataset to be used)

For illustration purposes, let’s suppose that you have a fictitious economy with the following parameters:

Use statsmodels to Perform Linear Regression in Python

The goal here is to predict/estimate the stock index price based on two macroeconomics variables: the interest rate and the unemployment rate.

We will use pandas DataFrame to capture the above data in Python. Before we dive into the Python code, make sure that both the statsmodels and pandas packages are installed. You may use the PIP method to install those packages.

The Python Code using Statsmodels

The following Python code includes an example of Multiple Linear Regression, where the input variables are:

  • Interest_Rate
  • Unemployment_Rate

These two variables are used in the prediction of the dependent variable of Stock_Index_Price.

Here is the complete syntax to perform the linear regression in Python using statsmodels (for larger datasets, you may consider to import your data):

from pandas import DataFrame
import statsmodels.api as sm

Stock_Market = {'Year': [2017,2017,2017,2017,2017,2017,2017,2017,2017,2017,2017,2017,2016,2016,2016,2016,2016,2016,2016,2016,2016,2016,2016,2016],
                'Month': [12, 11,10,9,8,7,6,5,4,3,2,1,12,11,10,9,8,7,6,5,4,3,2,1],
                'Interest_Rate': [2.75,2.5,2.5,2.5,2.5,2.5,2.5,2.25,2.25,2.25,2,2,2,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75],
                'Unemployment_Rate': [5.3,5.3,5.3,5.3,5.4,5.6,5.5,5.5,5.5,5.6,5.7,5.9,6,5.9,5.8,6.1,6.2,6.1,6.1,6.1,5.9,6.2,6.2,6.1],
                'Stock_Index_Price': [1464,1394,1357,1293,1256,1254,1234,1195,1159,1167,1130,1075,1047,965,943,958,971,949,884,866,876,822,704,719]        

df = DataFrame(Stock_Market,columns=['Year','Month','Interest_Rate','Unemployment_Rate','Stock_Index_Price']) 

X = df[['Interest_Rate','Unemployment_Rate']] # here we have 2 variables for the multiple linear regression. If you just want to use one variable for simple linear regression, then use X = df['Interest_Rate'] for example
Y = df['Stock_Index_Price']

X = sm.add_constant(X) # adding a constant

model = sm.OLS(Y, X).fit()
predictions = model.predict(X) 

print_model = model.summary()

This is the result that you’ll get once you run the Python code:

How to perform Linear Regression in Python using statsmodels

Interpreting the Regression Results

I highlighted several important components within the results:

  1. Adjusted. R-squared reflects the fit of the model. R-squared values range from 0 to 1, where a higher value generally indicates a better fit, assuming certain conditions are met.
  2. const coefficient is your Y-intercept. It means that if both the Interest_Rate and Unemployment_Rate coefficients are zero, then the expected output (i.e., the Y) would be equal to the const coefficient.
  3. Interest_Rate coefficient represents the change in the output Y due to a change of one unit in the interest rate (everything else held constant)
  4. Unemployment_Rate coefficient represents the change in the output Y due to a change of one unit in the unemployment rate (everything else held constant)
  5. std err reflects the level of accuracy of the coefficients. The lower it is, the higher is the level of accuracy
  6. P >|t| is your p-value. A p-value of less than 0.05 is considered to be statistically significant
  7. Confidence Interval represents the range in which our coefficients are likely to fall (with a likelihood of 95%)

Making Predictions based on the Regression Results

Recall that the equation for the Multiple Linear Regression is:

Y = C + M1*X1 + M2*X2 + …

So for our example, it would look like this:

Stock_Index_Price = (const coef) + (Interest_Rate coef)*X1 + (Unemployment_Rate coef)*X2

And this is how the equation would look like once we plug the coefficients:

Stock_Index_Price = (1798.4040) + (345.5401)*X1 + (-250.1466)*X2

Let’s suppose that you want to predict the stock index price, where you just collected the following values for the first month of 2018:

  • Interest Rate = 2.75 (i.e., X1= 2.75)
  • Unemployment Rate = 5.3 (i.e., X2= 5.3)

When you plug those numbers you’ll get:

Stock_Index_Price = (1798.4040) + (345.5401)*X1 + (-250.1466)*X2

Stock_Index_Price = (1798.4040) + (345.5401)*(2.75) + (-250.1466)*(5.3) = 1422.86

The predicted/estimated value for the Stock_Index_Price in January 2018 is therefore 1422.86.

The predicted value can eventually be compared with the actual value to check the level of accuracy. If, for example, the actual stock index price for that month turned out to be 1435, then the prediction would be off by 1435 – 1422.86 = 12.14

Disclaimer: this example should not be used as a predictive model for the stock market. It was based on a fictitious economy for illustration purposes only.

You may want to check the following tutorial that includes an example of multiple linear regression using both sklearn and statsmodels.

For further information about the statsmodels moduleplease refer to the statsmodels documentation.