Updated July 3, 2026:<\/strong> Reviewed for accuracy and refreshed metadata.<\/p>\n<\/div>\n In this class, we would build a logistic regression model.<\/p>\n We would cover the following topics:<\/p>\n Note<\/strong>: Logistic Regression is a Classification algorithm and hence discussed under Classification<\/p>\n You already learnt from Class 6 (Introduction to Classification) that given values of x, we need to determine the class, y.\u00a0 In other words, we aim to find the function that relates x and y. In the case of linear regression, this function is of the form:<\/p>\n y = f(x) = b0 + b1x<\/p>\n However, in the case of logistic regression, the output must be 0 or 1. So to achieve this, we need two things:<\/p>\n To solve the first issue, we would model the outputs as probabilities. For example, we have a dataset of bank customers with credit card debt and we want to predict which customers will default. The input variable would be the balance on the customers account. The function will be something like this<\/p>\n Pr(default = Yes | balance)<\/p>\n This is shortened as p(balance) and will always give a value between 0 and 1 (since probability values are in range of 0 and 1)<\/p>\n The solve the second issue we have to choose a threshold. For example, we may predict default = Yes for customers whose p(balance) > 0.5.<\/p>\n Before we go into the practical, I would just want you to know about the logistic function which is give below.<\/p>\n <\/p>\n Ok, let’s now go into the fun part!<\/p>\n <\/p>\n All the necessary modules we need are given below. I’m sure by now, you know what each of them are used for:<\/p>\n <\/p>\n We generate a dataset here using the make_classification function from sklearn.datasets.<\/p>\n You can actually change the number of samples to something more.<\/p>\n <\/p>\n Let’s now just do a scatter plot of this data. So we see why linear regression would not work quite well The output is given below:<\/p>\n <\/p>\n To perform the logistic regression, we would take two steps:<\/p>\n These three steps are given below in Python code<\/p>\n <\/p>\n We are interested in view the following metrics<\/p>\n <\/p>\n The confusion matrix is explained as given below:<\/p>\n You can also use the following code to check that a data value belongs to either class 0 or 1 (No or Yes).<\/p>\n <\/p>\n The code below creates a sigmoid plot of our dataset<\/p>\n The output of this code is given below:<\/p>\n This wraps up our class on Logistic Regression. I strongly recommend you watch the video for a clearer explanation.<\/p>\n <\/p>\n\n
1. The Logistic Regression Model<\/strong><\/h5>\n
\n
2. Import the Necessary Modules<\/strong><\/h5>\n
from<\/span> sklearn.datasets<\/span> import<\/span> make_classification\r\nfrom<\/span> matplotlib<\/span> import<\/span> pyplot as<\/span> plt\r\nfrom<\/span> sklearn.linear_model<\/span> import<\/span> LogisticRegression\r\nfrom<\/span> sklearn.model_selection<\/span> import<\/span> train_test_split\r\nfrom<\/span> sklearn.metrics<\/span> import<\/span> confusion_matrix\r\nimport<\/span> pandas<\/span> as<\/span> pd<\/span>\r\n<\/pre>\n3. Generate the Dataset<\/strong><\/h5>\n
# Generate and dataset for Logistic Regression<\/span>\r\nx, y =<\/span> make_classification(\r\n n_samples=<\/span>100<\/span>,\r\n n_features=<\/span>1<\/span>,\r\n n_classes=<\/span>2<\/span>,\r\n n_clusters_per_class=<\/span>1<\/span>,\r\n flip_y=<\/span>0.03<\/span>,\r\n n_informative=<\/span>1<\/span>,\r\n n_redundant=<\/span>0<\/span>,\r\n n_repeated=<\/span>0<\/span>\r\n)\r\n<\/pre>\n4. Visualize the Data Using ScatterPlot<\/strong><\/h5>\n
\n<\/p>\n# Create a scatter plot<\/span>\r\nplt.<\/span>scatter(x, y, c=<\/span>y, cmap=<\/span>'rainbow'<\/span>)\r\nplt.<\/span>title('Scatter Plot of Logistic Regression'<\/span>)\r\nplt.<\/span>grid( linestyle=<\/span>'--'<\/span>)\r\nplt.<\/span>show()\r\n<\/pre>\n![\"Logistic](\"https:\/\/kindsonthegenius.com\/data-science\/wp-content\/uploads\/2021\/09\/Screenshot-2021-09-30-at-21.41.30-300x212.png\")
<\/a>5. Perform Logistic Regression<\/strong><\/h5>\n
\n
# Split the dataset into training and test dataset<\/span>\r\nx_train, x_test, y_train, y_test =<\/span> train_test_split(x, y, random_state=<\/span>1<\/span>)\r\n\r\n# Create a Logistic Regression Object, perform Logistic Regression<\/span>\r\nlog_reg =<\/span> LogisticRegression()\r\nlog_reg.<\/span>fit(x_train, y_train)\r\n<\/pre>\n6. View the Metrics<\/strong><\/h5>\n
\n
# Show to Coeficient and Intercept<\/span>\r\nprint<\/span>(log_reg.<\/span>coef_)\r\nprint<\/span>(log_reg.<\/span>intercept_)\r\n\r\n# Perform prediction using the test dataset<\/span>\r\ny_pred =<\/span> log_reg.<\/span>predict(x_test)\r\n\r\n# Show the Confusion Matrix<\/span>\r\nconfusion_matrix(y_test, y_pred)\r\n<\/pre>\n# True positive: (top-left) (We predicted a positive result and it was positive)<\/span>\r\n# True negative: (lower-right) (We predicted a negative result and it was negative)<\/span>\r\n# False positive:(top-right) (We predicted a positive result and it was negative)<\/span>\r\n# False negative: (lower-left) (We predicted a negative result and it was positive)<\/span>\r\n<\/pre>\n# Check the actual probability that a data point belongs to a class<\/span>\r\nlr.<\/span>predict_proba(x_test)\r\n<\/pre>\n7. Create the Sigmoid Plot<\/strong><\/h5>\n
# Create and sort a dataframe containing our data<\/span>\r\ndf =<\/span> pd.<\/span>DataFrame({'x'<\/span>: x_test[:,0<\/span>], 'y'<\/span>: y_test})\r\ndf =<\/span> df.<\/span>sort_values(by=<\/span>'x'<\/span>)\r\n\r\n\r\n# The expit function, also known as the logistic function, is defined as expit(x) = 1\/(1+exp(-x)).<\/span>\r\n# It is the inverse of the logit function.<\/span>\r\nfrom<\/span> scipy.special<\/span> import<\/span> expit\r\nsigmoid_function =<\/span> expit(df['x'<\/span>] *<\/span> log_reg.<\/span>coef_[0<\/span>][0<\/span>] +<\/span> log_reg.<\/span>intercept_[0<\/span>]).<\/span>ravel()\r\nplt.<\/span>plot(df['x'<\/span>], sigmoid_function)\r\nplt.<\/span>scatter(df['x'<\/span>], df['y'<\/span>], c=<\/span>df['y'<\/span>], cmap=<\/span>'prism'<\/span>)\r\nplt.<\/span>show()\r\n<\/pre>\n![\"Sigmoid](\"https:\/\/kindsonthegenius.com\/data-science\/wp-content\/uploads\/2021\/09\/Screenshot-2021-09-30-at-22.02.41-300x200.png\")
<\/a>