Logistic Regression

11.1 From Linear Regression to Logistic Regression

In the last session we met a new technique for exactly this situation: logistic regression, LR for short. The whole discussion in this session continues that topic. Before we see the new machinery, we first rebuild the regression family we already know, so that the departure point is clear.

11.1.1 The Regression Models We Already Know

In a standard regression setup we have one dependent variable (the outcome we want to explain), written $y$, and a set of independent variables (the inputs we use to explain it), say $x_1,x_2,x_3$. In ordinary linear regression and multiple linear regression, the dependent variable $y$ and all the independent variables are continuous in nature: each can take any value inside an interval. For a problem with one dependent and two independent variables, the model we built has the form

$$y={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2$$

with the error term ignored for the modeling discussion (we treat the disturbance, the part of $y$ the model cannot explain, as zero). Writing the same idea for the expected value of $y$:

$$E(y)={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2$$

Every symbol has a job. Here:

• $y$ is the dependent variable, the outcome to be predicted;
• $x_1,x_2$ are the independent variables, the predictors we feed in;
• ${\beta }_0$ (beta-zero) is the intercept — the expected value of $y$ when both $x_1$ and $x_2$ are 0;
• ${\beta }_1,{\beta }_2$ are the regression coefficients, often called weights — they tell us how strongly each predictor moves the outcome;
• $E(y)$ is the expected value, the long-run average of $y$ given the $x$'s.

The interpretation is direct: a one-unit increase in $x_1$ changes $E(y)$ by ${\beta }_1$, a one-unit increase in $x_2$ changes $E(y)$ by ${\beta }_2$, and so on. If ${\beta }_1=2$, then raising $x_1$ by one unit raises $E(y)$ by 2 units. This straight, constant "change per unit" reading is exactly what we lose in logistic regression.

One variation we discussed earlier concerns the independent variables. When an independent variable is categorical — say a repair problem where a variable is coded with numerical values such as 0 and 1 — the model equation looks the same, but we have to take extra care in the interpretation. With a 0/1 input, a coefficient no longer means "one whole unit of the variable" but "the difference between the two categories." So the family so far covers:

• continuous dependent variables with continuous independent variables;
• continuous dependent variables with categorical independent variables (dummy coding).

In both cases $y$ itself stays continuous, and the straight-line logic holds.

11.1.2 A New Problem: a Dependent Variable That Takes Only 0 and 1

Now we flip the situation. Many practical, real-time problems have factors in which the dependent variable itself is categorical. We can decode $y$ with either 1 or 0 — think of 1 as "happy" and 0 as "sad", or 1 as "yes, the customer used the coupon" and 0 as "no, they did not". Once the dependent variable is categorical, the whole discussion changes. The way we estimated ${\beta }_0,{\beta }_1,{\beta }_2$ for linear regression — through the principle of least squares and the derivatives that follow from it — no longer carries over as-is.

Real-world: this setting is called a binary classification problem (you can also call it just a classification problem). It is one of the most common decision problems in business and industry: approve or reject, buy or skip, default or pay, use or not use. For the time being we concentrate only on the binary case, where $y$ takes exactly two values, 1 and 0.

When $y$ is 1 or 0 with several independent variables, you cannot blindly apply basic linear regression. Two things need care. First, the expression we write for the expected value of $y$ must be chosen to suit this kind of data — the formula itself has to respect what $y$ can be. Second, we need a model that produces answers inside the only range that makes sense for a probability: between 0 and 1.

11.1.3 Why a Linear Model Breaks Down

Picture the data. If we plot $y$ against the independent variables, all the observed $y$ values sit on two horizontal lines: the line $y=1$ for customers who used the coupon and the line $y=0$ for those who did not. If we fit any linear regression model to such a plot, there is every possibility that the fitted line will exceed $y=1$ on one side or drop below $y=0$ on the other. That is not acceptable for this problem: the estimate must be captured between 0 and 1, because the answer we are producing is a probability-like quantity that must lie in the interval $[0,1]$.

The visual takeaway: on a chart with the independent variable $x$ on the horizontal axis and $E(y)$ on the vertical axis scaled from 0 to 1, the data appear as two rows of dots — a lower row hugging 0 and an upper row hugging 1. A straight line drawn through that cloud will cut through both rows, but outside the cloud it keeps climbing past 1 or keeps sinking below 0. The curve we need instead starts near 0, rises smoothly, and flattens near 1: the S-shaped curve.

So the first thing to understand is: what type of model can predict $y$ within this range — either exactly 0, exactly 1, or something in between? Ordinary regression cannot do it. We need an S-shaped curve — the function we call the sigmoid function or logistic function — which will be the mathematical expression that takes care of this situation. The logistic function predicts $y$ between 0 and 1 no matter what the inputs are.

The contrast with linear regression is worth making explicit. The two models serve two different kinds of outcome:

When to pick which: use linear regression when the outcome is continuous and unbounded; use logistic regression the moment the outcome is a yes/no variable. The choice is driven by the dependent variable, not by convenience.

Real-world: this is not a textbook-only concern. Banks use the exact same setup to predict whether a credit card request is approved ($y=1$) or rejected ($y=0$); insurers predict whether a claim is fraudulent; telecom operators predict which customers will cancel (churn); email services classify a message as spam (1) or not (0). In every one of these domains the output must be a probability inside $[0,1]$, and the decision — approve, investigate, retain, block — comes from acting on that probability. That is the setting logistic regression was built for, and the rest of this lecture develops it on one running example.

11.2 The Logistic Regression Model

11.2.1 The S-Shaped Curve

The logistic regression model rests on one function: the sigmoid, also called the logistic function. Its graph is an S-shaped curve. The curve starts near 0 for small inputs, rises through a smooth bend, and flattens out near 1 for large inputs. This is exactly the shape we need: the model output can get arbitrarily close to 0 and arbitrarily close to 1, but never steps outside the $(0,1)$ interval.

Visualize the curve itself: on a chart with the linear combination on the horizontal axis and $E(y)$ on the vertical axis (scaled from 0 to 1), the curve begins hugging the 0 line on the left, bends upward through the point $(0,0.5)$ — where the input combination is exactly 0, the curve sits exactly at the halfway mark — and flattens against the 1 line on the right. The takeaway from the picture: the curve never leaves the band between 0 and 1, no matter how extreme the inputs are. It approaches 0 and 1 as asymptotes but never touches them.

The framing from the lecture: a normal regression will not work here, so we introduce a mathematical expression that takes care of this situation. Later in the course we come back to the math of why this particular expression is used — what the exponential term is doing and why people decided this is the best-suited model. For now the key point is that this one expression captures the nonlinear S-shaped behavior between $y$ and the independent variables.

11.2.2 The Model Equation and Its Two Equivalent Forms

11.2.3 Interpreting E(y) as the Probability of Success

Here is a point that must be remembered for the entire topic. In multiple linear regression we talk about the expected value of $y$, and $E(y)$ can take any value. In logistic regression the main objective — the way we interpret the expected value — is in terms of probability. For that reason we do not simply write $E(y)$; precisely, we write

$$E(y)=P(y=1\mid x_1,x_2)$$

the probability that $y$ equals 1 given the particular values of the independent variables. And whenever we talk about this expected value, the probability of interest is always the success — the happening. We always represent the event of interest as $y=1$. This is a fundamental rule of logistic regression: the event of interest is always $y=1$ given a particular set of independent variables, and the whole problem is developed in that context.

Real-world: this bounded-probability form is why logistic regression is the default model for response and risk scoring across industry. Credit bureaus score the probability that an applicant defaults; banks score the probability of card approval; marketing teams score the probability that a customer responds to a campaign; hospitals score the probability of readmission. In every case the score is a number between 0 and 1 that a decision rule converts into an action, and in every case the S-shape does the same job it does here.

11.3 Behavior of the Model: A Single-Variable Example

11.3.1 The Example Model

To see how the model behaves, shrink it to one dependent and one independent variable. Write the dependent variable as $y$ and the single independent variable as $x$. Then the expected value of $y$ — the probability that $y$ equals 1 given $x$ — is

$$E(y)=P(y=1\mid x)=\frac{e^{{\beta }_0+{\beta }_{1}x}}{1+e^{{\beta }_0+{\beta }_{1}x}}$$

where ${\beta }_0$ is the intercept and ${\beta }_1$ is the regression coefficient (the weight on $x$), exactly as in the two-variable model but with the second term removed.

11.3.2 Substituting x = 2

The walkthrough covers exactly this substitution — first replacing $x$ by 2 in the exponent, simplifying to $e^{-1}$, and then evaluating the ratio to get 0.27.

11.3.3 The Bounds: Output Always Stays Between 0 and 1

Now watch what happens at the extremes. If $x$ is close to 0, the exponent $-7+3x$ is a strongly negative number, the numerator $e^{-7+3x}$ becomes very small, and the expected value is very close to 0. On the other side, as $x$ increases, the expected value rises and approaches 1 — but it can never go beyond 1. Take any possible value of $x$ in between, and you get an expected value inside the interval $(0,1)$.

To see the whole journey in one picture, evaluate the same model across a range of $x$ values:

Read the column from top to bottom: the probability crawls near 0, accelerates through the middle (from 0.269 at $x=2$ to 0.881 at $x=3$), and then flattens near 1. The output is always between 0 and 1, no matter which row you look at.

This is the beauty of the expression, and the point about how the model captures probability: the more $x$ increases, the more the whole function moves toward 1 while always staying between 0 and 1. The model itself enforces the bounds; we never have to patch the answer after the fact.

Visualize the full curve once more, now with numbers: horizontal axis $x$ from 0 to 5, vertical axis $E(y)$ from 0 to 1. The curve is flat at the bottom near 0 for $x\le 1$, climbs through the point $(2,0.269)$, crosses the steepest part between $x=2$ and $x=3$ (the transition from 0.269 to 0.881), and then flattens toward 1. The midpoint of the rise sits near $x\approx 2.33$, where the exponent $-7+3x=0$ and the probability equals 0.5. The takeaway: the S-curve spends most of its length hugging 0 and 1, and concentrates all its change in a short middle stretch.

Real-world: this "S-curve between 0 and 1" behavior is exactly what risk-score models rely on in insurance and lending. A credit scorecard maps features (income, history, balances) through a logistic curve to a default probability; the scorecard is tuned so that the curve is steep in the region where decisions matter most — the middle — and flat where the answer is already clear. Watching the single-variable curve rise through 0.27 → 0.88 is watching that mechanism at its smallest scale.

11.4 The Simmons Stores Example: Problem Setup

The whole discussion of logistic regression is carried on a single example, taken from the textbook (the detailed problem description appears around page 727 of the text; the audio rendering of the store name comes through as "Simon", and the textbook example is Simmons Stores).

11.4.1 The Business Problem

The owner of Simmons Stores runs a national chain of women's apparel stores. He has 5,000 expensive copies of a catalog printed; each catalog includes a coupon that provides a 50-dollar discount on purchases of 200 dollars or more. Because the catalogs are expensive, he wants to send them only to customers who have a high probability of actually using the coupon in the catalog.

The core idea: instead of guessing, model the probability. What is the likelihood that a given customer uses the coupon? If we can estimate that probability, we can generalize the decision — which customers are worth the cost of a catalog.

Management believes that two variables might be helpful in predicting whether a customer who receives the catalog will use the coupon: annual spending at Simmons Stores, and whether the customer has a Simmons credit card.

11.4.2 The Variables

To carry out the study, Simmons conducted a pilot study: a random sample of 100 customers — 50 credit-card customers and 50 customers who do not have a Simmons credit card. Each of the 100 selected customers received the catalog, and at the end of the test period Simmons noted whether the customer used the coupon.

11.4.3 The Data Set

The data set has 100 observations of these three columns: spending ($X_1$), card ($X_2$), and coupon ($Y$). The first ten rows of the pilot study data look like this (annual spending in thousands of dollars):

Each row is one customer: how much they spent last year, whether they hold the store card, and whether they used the coupon in the test mailing. Notice that the coupon column contains only 0s and 1s — there is no middle value. The full hundred observations are available to work on from your end, so you can reproduce every output in this lecture yourself.

The key point about this data: the dependent variable is categorical in nature, so no ordinary linear or multiple linear regression will work here, and logistic regression is the tool to rely on. The 0/1 coupon column is exactly the kind of $y$ the S-curve was built to serve.

Real-world: this is direct-mail and catalog targeting, a classic marketing use of logistic regression: spend limited promotional money only on customers whose predicted response probability clears a bar. The same mechanics run modern advertising — email senders, mobile push campaigns, and coupon apps all score each customer's probability of responding and spend their message budgets only on the high-probability segment. The "5,000 expensive catalogs" is simply the budget that forces the targeting decision.

11.5 Estimating the Coefficients with Software

11.5.1 Why Software, Not Calculus

Why is the estimation harder here? In linear regression, minimizing the sum of squared errors leads to linear equations in the coefficients, and those equations have closed-form solutions. In logistic regression, the standard fitting approach (maximum likelihood) leads to nonlinear equations that have no such closed-form solution — software must search for the best values numerically. That is why the textbook and the lecture both route through software output: the search is the software's job, and reading the output is ours.

The important thing to note: the same approach was used even for multiple regression — the tool does the heavy lifting, and we read off the possible values of the coefficients. There is no need for extra effort in extracting them. Software is the tool, not the derivation.

11.5.2 The R Walkthrough: glm() with Family = Binomial

The example is worked in R (open-source software); the textbook shows the Minitab output — either way the numbers are the same, and there is no restriction on which tool you use. R was used here simply because it is open source and shareable.

The steps of the walkthrough:

1. Prepare the data. Take the Simmons data set, with columns for the customers: $y$ (coupon), $x_1$ (spending) and $x_2$ (card).
2. Convert the format. Convert the Excel data set into a CSV file, which is the general practice when working in R.
3. Read the data into R. Use read.csv with the file.choose() dialog, with the header argument set to true:

   LR <- read.csv(file.choose(), header = TRUE)

The loaded data set is called LR.

1. Fit the model with the glm function. glm stands for generalized linear model — the wider family that logistic regression belongs to. The dependent variable is coupon ($y$), the independent variables are spending ($x_1$) and card ($x_2$):

   fit <- glm(y ~ x1 + x2, data = LR, family = binomial)

The family is binomial because of the dichotomous nature of the event — having or not having the event (using or not using the coupon).

1. Display the results. Run summary(fit) to print the full model output.

LR <- read.csv(file.choose(), header = TRUE)     # pick the CSV file
fit <- glm(y ~ x1 + x2, data = LR, family = binomial)
summary(fit)

There was a small debugging moment in the session: an early version of the command left out the data reference, and that was caught and corrected.

11.5.3 Reading the Estimates: the Estimate Column

Real-world: glm() with family = binomial in R, and its equivalents in Minitab, SPSS and SAS, are the standard tools for fitting logistic regression in practice. Any of them is fine — and on the exam, any software output is acceptable, as we will see again in section 11.7. In industry, the same commands run inside credit-risk engines, marketing response models, and medical prediction tools; the analyst's job is rarely to fit the model from scratch, but to read the output correctly and explain it — exactly what this section trains.

11.6 Estimating Probabilities from the Model

We use the fitted equation to estimate the probability of using the coupon for a particular type of customer. The whole exercise is substitution: wherever $x_1$ appears, put the customer's spending in thousands; wherever $x_2$ appears, put their card status. Then simplify the exponent, evaluate the exponential, and read off a probability between 0 and 1.

11.6.1 A 2,000-Dollar Spender Without a Simmons Card

Suppose the customer spends 2,000 dollars annually and does not have a Simmons credit card. In terms of the variables: $x_1=2$ (spending in thousands) and $x_2=0$ (no card).

In probability terms: about 18% of customers of this type — 2,000 dollars annual spending, no Simmons card — are estimated to use the coupon.

11.6.2 A 2,000-Dollar Spender With a Simmons Card

Now the same customer but holding the card: $x_1=2$ and $x_2=1$. The only difference from the previous calculation is that the card term now contributes $1.09(1)$ instead of $1.09(0)$.

About 41% of customers who spend 2,000 dollars annually and hold the Simmons credit card are estimated to use the coupon.

11.6.3 Reading the Two Estimates Together

Comparing the two numbers already gives a first inference: the probability of using the coupon is much higher — roughly double — when the customer uses the Simmons credit card. So with the possible combinations $x_1=2$ with $x_2=0$ and $x_2=1$, we already get a feel for how to comment on the probability of using the coupon.

This can be extended. Since $x_2$ only takes 1 or 0, the interesting variation is annual spending. If spending moves from one level to another, how do the probabilities move? The full tabulated exercise spreads $x_1$ over 1, 2, 3, 4, 5, 6 and 7 (thousands of dollars) for both $x_2=0$ and $x_2=1$:

Reading the table: every row rises as spending grows, and the with-card row sits above the without-card row at every spending level. At the lowest end (1,000 dollars, no card) the probability is about 14%; at the highest end (7,000 dollars, with card) it is about 79%. The probabilities spread widely across that range, which is the information a manager needs to decide who gets a catalog.

At this initial level the expression itself guarantees that every $\hat{y}$ lies between 0 and 1 — you cannot get more than that — which is the first big payoff of the logistic form. Compare the two columns at any single spending level and the card effect is visible; trace any row across spending levels and the spending effect is visible. The table is the model doing its job across the whole range of customers.

Visualize the table as two rising curves on one chart: horizontal axis annual spending (thousands of dollars), vertical axis estimated probability from 0 to 1. The with-card curve starts higher (0.33 at 1,000) and runs above the without-card curve (0.14 at 1,000) at every point; both rise smoothly and gradually steepen across the middle spending range. The vertical gap between the two curves is roughly the same width across the whole range — a hint of the constant card effect that the odds ratio will measure precisely in section 11.8.

Real-world: this substitution exercise is the day-to-day job of a response-model analyst. Given a fitted model, a marketing team scores each customer in the database — plug in their spending and card status, read off the probability, rank the list, and mail only the top segment. The same pattern runs in credit: plug in an applicant's features, read off a default probability, and route the application. The probability table above is the scorecard of a small direct-mail campaign.

11.7 Significance Testing in Logistic Regression

11.7.1 Overall Significance: The Chi-Square Test

Before making generalized conclusions, we need the same validity check we did for multiple linear regression: the overall significance of the model, and the individual significance of each independent variable. In multiple linear regression, overall significance used the F distribution and individual significance used the t distribution. In logistic regression the logic is the same, but the statistic changes.

11.7.2 Getting the Chi-Square from R Output

The Minitab output prints the chi-square directly, but the simple R output does not make it quite as straightforward. In the R output, the chi-square value is the difference between the null deviance and the residual deviance:

$${\chi }^2=\text{null deviance}-\text{residual deviance}$$

A small difference separates the two output styles: with the R output you find the chi-square yourself by subtracting the two deviances; with Minitab it is printed for you. The p-value in the R output is also given, and both outputs lead to the same conclusion — the overall model is significant.

11.7.3 Individual Significance of Each Variable

The second check: is spending significant? Is card significant? For that, we refer to the p-values of the individual coefficients, the same p-value approach we introduced earlier for these tests. For each independent variable $x_i$, the hypotheses are

$$H_0:{\beta }_i=0$$

$$H_a:{\beta }_i\ne 0$$

In the standard output, each coefficient carries its own test statistic and p-value; the textbook's deviance table shows the test for each coefficient as a chi-square value with 1 degree of freedom — equivalent to the normal-based z test the p-value approach relies on. For the Simmons Stores example:

• spending ($x_1$): chi-square value 7.56, p-value 0.006 — significant at the 0.05 level, so ${\beta }_1\ne 0$;
• card ($x_2$): chi-square value 6.41, p-value 0.011 — significant at the 0.05 level, so ${\beta }_2\ne 0$.

Both independent variables are statistically significant — both contribute to the model. So before making any further conclusions we have a clear picture: overall significance is established, and both individual variables pass the significance check as well.

A small related story from the grading side: a mistake committed while grading a quiz was described in hypothesis-testing language. Rejecting the null hypothesis when the null hypothesis is true is a type I error — here, giving zero marks for answers that were correct even though the question itself was fine. The reverse mistake — the question is wrong but marks are still given — is a type II error, from the other side. Both are being rectified before the marks are finalized. The story is worth keeping because it maps the formal definitions onto a concrete situation: a type I error is a false alarm (declaring something wrong when it is right), and a type II error is a missed signal (declaring something fine when it is not).

Real-world: this two-stage significance check is standard practice in applied modeling. Before a bank deploys a credit scorecard or a hospital publishes a prediction model, the same questions are asked: does the model as a whole beat the intercept-only baseline, and does each feature earn its place? Regulators and audit teams expect exactly this evidence — the chi-square test statistic and per-variable p-values — as part of a model's validation documentation.

11.8 Odds and the Odds Ratio

The odds ratio is the key measure for interpreting logistic regression, and it is a core exam concept. To define it, we first need the odds in favor of an event.

11.8.1 Odds in Favor of an Event

11.8.2 Computing the Odds for Both Customer Types

Take the two customer types from the probability estimates. For the customer who spends 2,000 dollars and has the Simmons card ($x_1=2,x_2=1$), the estimated probability is 0.4102.

For the customer who spends 2,000 dollars and does not have the card ($x_1=2,x_2=0$), the estimated probability is 0.1881.

Each step is a direct substitution: the probability in the numerator, one minus that probability in the denominator, and the division carried out. The odds for the card-holding customer (0.6956) are much larger than the odds for the customer without the card (0.2318).

11.8.3 The Odds Ratio and What It Means

The odds ratio is simply the ratio of the two odds:

$$\text{odds ratio}=\frac{\text{odds}(2,1)}{\text{odds}(2,0)}=\frac{0.6956}{0.2318}\approx 3.00$$

The output gives this as 3.004. The interpretation is the standard one: the estimated odds in favor of using the coupon for a customer who spent 2,000 dollars last year and has a Simmons credit card are about three times greater than the estimated odds in favor of the coupon for a customer who spent 2,000 dollars and does not have a Simmons credit card.

Notice what this buys us. Earlier we sensed from the probabilities that card holders are more likely to use the coupon. Instead of running through the whole probability table, one small number — the odds ratio of about 3 — states the whole observation. This is the very interesting thing: the odds ratio measures the impact of one independent variable on the dependent variable, and it does it with a single digestible number.

11.8.4 A Shortcut: e to the Power of the Coefficient

A shortcut avoids the whole odds-computation exercise. The estimated odds ratio for a variable is just $e$ raised to that variable's coefficient:

$$\text{odds ratio for }{x}_i=e^{{\beta }_i}$$

Why does this identity hold? It falls straight out of the model, so we can derive it instead of taking it on faith. Write $z={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2$. The odds in favor of $y=1$ are

$$\text{odds}=\frac{P(y=1\mid x_1,x_2)}{1-P(y=1\mid x_1,x_2)}=\frac{\frac{e^z}{1+e^z}}{1-\frac{e^z}{1+e^z}}=\frac{\frac{e^z}{1+e^z}}{\frac{1}{1+e^z}}=e^z$$

The second line follows because $1-\frac{e^z}{1+e^z}=\frac{1+e^z-e^z}{1+e^z}=\frac{1}{1+e^z}$. So the odds equal the exponential of the linear combination. Now increase $x_1$ by one unit, holding everything else fixed: $z$ becomes $z+{\beta }_1$, and the odds become $e^{z+{\beta }_1}=e^z\cdot e^{{\beta }_1}$. The odds ratio is the new odds divided by the old odds:

$$\text{odds ratio}=\frac{e^{z+{\beta }_1}}{e^z}=e^{{\beta }_1}$$

That is the shortcut, derived: a one-unit increase in a variable multiplies the odds by $e$ to the power of that variable's coefficient. For a change of $c$ units in one variable, the same argument gives $e^{c{\beta }_i}$.

So there are two routes to the same answer: the full managerial table of probabilities and odds (one approach), or the very small shortcut of raising $e$ to the regression coefficient. The numbers agree, and the interpretation is the part that matters.

11.8.5 Properties: Range and Invariance to the Other Variable

Two properties worth noting. First, the odds ratio inherits its range from the odds itself: since odds is $p$ divided by $(1-p)$, it runs from 0 to infinity. Second, and this is the deep reason the odds ratio is so clean: the ratio is the same no matter which value of the other variable you start from. The odds ratio focuses on a one-unit increase in only one independent variable, keeping the others constant. If you change $x_1$ from 2,000 to 3,000, or 3,000 to 4,000, or 4,000 to 5,000 — the ratio for $x_2$ is still about 3.

Check the invariance with numbers from the table in section 11.6. At $x_1=3$: the with-card probability is 0.4948, so the odds are $0.4948/0.5052=0.9794$; the without-card probability is 0.2460, so the odds are $0.2460/0.7540=0.3263$. The ratio $0.9794/0.3263\approx 3.00$ — the same 3.00 we found at the 2,000-dollar baseline. The invariance is exact, not a coincidence: it follows directly from the shortcut, because the baseline value $e^z$ cancels in the ratio and only the coefficient difference $e^{{\beta }_2}$ survives.

Visualize the invariance as two parallel odds curves on one chart: horizontal axis annual spending, vertical axis odds of coupon use. The with-card curve ($e^z$ with the card term included) is a constant vertical multiple of the without-card curve ($e^z$ without it) — the gap between the curves is exactly the factor $e^{{\beta }_2}\approx 3$ at every spending level. One number, one constant gap: that is the whole picture.

Real-world: because of this invariance, odds ratios are how logistic regression results are reported and compared in practice — one number per variable, with a confidence interval. Most packages print a 95% confidence interval next to each odds ratio; for the Simmons model the interval for spending is about 1.09 to 1.81 and for card about 1.26 to 7.17 — neither interval contains 1, which backs up the significance conclusion of section 11.7. In medical studies, odds ratios are the standard currency of reporting treatment effects ("patients on the drug had 2.1 times the odds of recovery"); in marketing, they quantify how much each customer attribute shifts the response odds. Confidence intervals also help when interpreting the odds ratio: an interval straddling 1 signals a variable that might have no real effect.

11.9 Managerial Decision Making with the Model

11.9.1 Setting a Probability Threshold

The probability estimates are not just numbers — they feed decisions. Suppose management decides, based on experience, that a customer is worth a catalog if the estimated probability of using the coupon is 0.4 or more. That is a benchmark you set yourself: send the catalogs only to those customers whose probability of using the coupon is at least 0.4, regardless of whether they have a credit card.

With such a rule in place, the question becomes: which types of customers cross the bar?

11.9.2 The Two Recommendations

Reading the probability table (section 11.6) against the threshold 0.4, two recommendations fall out.

Real-world note on the borderline case: the textbook adds that the cardless 5,000-dollar customer sits at 0.3927 — so close to 0.4 that the firm may choose to revise the strategy and include 5,000-dollar-and-above cardless customers anyway. Thresholds are managerial dials, not laws of nature; the model makes the trade-off visible instead of hidden.

This is how the model becomes a targeting policy. Instead of trial and error over every combination of customer attributes, the fitted logistic equation plus a probability threshold produces a clean rule: who gets the catalog and who does not. The odds ratio then serves as the supporting evidence — the single-number justification behind the rule.

11.9.3 Why Interpretation Is Not Direct

Throughout the topic, one contrast recurs: in a simple linear regression, a one-unit increase in $x$ maps straight to a change in the dependent variable — you can interpret the coefficient directly. In logistic regression that is not true. The expression is nonlinear because of the exponential term, so you cannot straight away say what a one-unit increase in $x$ does to $y$ or to the probability. The interpretation is not as direct as the way we assumed in the linear case. That is precisely why the odds ratio exists: it is the simple measure we rely on to talk about the relationship between an independent variable and the dependent variable in this nonlinear model.

11.9.4 Next Steps

The discussion of logistic regression will be concluded in a short span next time, with one or two more worked examples — a new example apart from Simmons Stores — and then some remaining output concepts such as the AIC value. After that, the course moves to dimension-reduction techniques: principal component analysis (PCA) and linear discriminant analysis (LDA). The plan is a conceptual understanding first, then a numerical example to justify and conclude.

A note for preparation: those who already know some linear algebra can anticipate a better understanding of PCA and LDA — behind the fascinating names there is mathematics, and some idea of the math really helps in understanding these techniques in a better way. These are algorithms you have been using as part of machine learning and artificial intelligence courses.

Real-world: this threshold logic is how logistic models run in production across industry. A bank sets a cut-off on its default-probability score and approves above it, denies below it; an insurance company prices or declines policies by a probability cut-off; an online retailer scores every visitor's purchase probability and decides who gets a discount code. The pattern is always the same: model the probability, set the bar by business judgment, act on the crossing — exactly the Simmons Stores exercise at the scale of millions of customers.

Exam Guidance Summary

• Software outputs: on the exam, any software output is acceptable — R, Minitab, SPSS, SAS — whatever you are comfortable with. The outputs you used in the exam yourself are fine; you can use any such information. There is no restriction on which tool's output you read the coefficients from.
• Coefficient estimation by calculus is out of scope. You are expected to read ${\beta }_0,{\beta }_1,{\beta }_2$ from the Estimate column of the output (here −2.14, 0.34, 1.09) and write the fitted logistic equation, not to derive the estimates. No partial differential equations, no maximum-likelihood algebra.
• Be ready to substitute values into the fitted equation and read off probabilities — e.g., $x_1=2$ and $x_2=0$ gives about 0.1881, $x_1=2$ and $x_2=1$ gives about 0.4102 — and to convert probabilities into odds with $p/(1-p)$.
• Overall significance in logistic regression uses the chi-square statistic (13.63 here, p < 0.05, with 2 degrees of freedom). In R, compute it as null deviance minus residual deviance (134.60 − 120.97). Individual significance uses the p-values of the coefficients.
• The odds ratio is a core exam concept: definition $p/(1-p)$, the shortcut $e$ to the power of the coefficient, and the interpretation — the odds in favor of the event are about three times greater (3.004), not the probability itself. The odds ratio is the same at any baseline of the other variable.
• Managerial reasoning: setting a probability threshold (e.g., 0.4) and deriving recommendations — send catalogs to customers spending 2,000 dollars or more with the card, or 6,000 dollars and above without the card.
• Remember the vocabulary: the event of interest in logistic regression is always $y=1$ — the success, the happening — and $E(y)$ is interpreted as the probability of that event.
• Mid-semester exams (regular and makeup) are under evaluation. A quiz marks issue is being corrected: zero marks given for correct answers (a type I error — rejecting the null when the null is true) and the opposite grading mistake (a type II error) are both being fixed before the marks are finalized.
• Next time: two more logistic regression examples and the AIC value, then PCA and LDA — a conceptual understanding followed by a numerical example. Some linear algebra background will help.

Key Industry Applications

• Catalog and direct-mail targeting (Simmons Stores): model the probability that a customer uses a coupon, set a probability threshold, and spend expensive promotional resources only on customers who clear the bar. The manager's rule from the example: catalogs go to customers spending 2,000 dollars or more with the credit card, or 6,000 dollars and above without it.
• Logistic regression as a binary classification tool: problems with a 1/0 dependent variable are framed as classification problems, which is the same modeling idea used across industry for decision and prediction tasks with two outcomes — approve or reject a credit card, flag a transaction as fraud or not, classify an email as spam or not.
• Standard software tooling: glm() with family = binomial in R (open-source), and the equivalent procedures in Minitab, SPSS and SAS, are the everyday tools for fitting logistic regression in practice. R computes the estimates, deviances and p-values; Minitab prints the chi-square and the odds ratio directly.
• Odds ratios as the reporting currency: because the odds ratio for a one-unit change in one variable is the same at any value of the other variables, it is the number companies quote when communicating the impact of a predictor — in marketing (how much a feature shifts response odds), in insurance and lending (risk factors), and in medical reporting (treatment effects). Confidence intervals around the odds ratio show whether the effect is credible.
• Dimension reduction (PCA and LDA) is previewed as the next industrial technique: principal component analysis and linear discriminant analysis are algorithms already used in machine learning and artificial intelligence courses, and linear algebra knowledge is the key to understanding them.