Residual Analysis, Dummy Variables, and Logistic Regression

10.1 Regression Recap: The Toolkit from the Previous Session

10.1.1 Simple and Multiple Linear Regression

The discussion opened by reconnecting with the regression analysis built in the previous session. Two fundamental techniques were introduced there: simple linear regression — one dependent variable explained by a single independent variable — and multiple linear regression — one dependent variable explained by several independent variables at once. Both are predictive tools: they describe how the response changes as the predictors change, and they allow us to forecast the response at new predictor values.

The simple linear regression model relates the dependent variable $y$ to one independent variable $x$ through a linear equation with an error term:

$$y={\beta }_0+{\beta }_{1}x+\epsilon$$

In words: "beta naught plus beta 1 x plus some epsilon." Here ${\beta }_0$ is the intercept — the value of $y$ when $x$ is zero — ${\beta }_1$ is the slope — how much $y$ changes for a one-unit increase in $x$ — and $\epsilon$ is the error term, the part of $y$ the line cannot explain. The error term is essential: without it the equation would claim every data point sits exactly on the line, which never happens with real data.

The multiple linear regression version expresses the expected value of $y$ as a linear combination of $p$ independent variables:

$$E(y)={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2+\cdots +{\beta }_p{x}_p$$

In words: "the expected value of y equals beta naught plus beta 1 x1 plus beta 2 x2, and so on, up to p independent variables." Each ${\beta }_j$ is a population coefficient, and in practice the Greek letters are replaced by sample estimates $b_0,b_1,b_2,\dots$ found through least squares — the approach that picks the estimates so that the sum of the squared vertical deviations of the observations from the fitted line is as small as possible. In the earlier session the least-squares formulas were not re-derived from calculus; the class took the support of the software output and went straight to interpretation.

10.1.2 Coefficient of Determination and Adjusted R Square

Two very important metrics were discussed for judging regression quality. The first is the coefficient of determination, written $R^2$. It measures the amount of variation in the dependent variable that is explained by the independent variables — or, equivalently, by the estimated regression equation. The $R^2$ value takes care of that proportion-of-explained-variation question. Recall the decomposition of variation from the earlier discussion: the total variation $SST$ splits into the part the model explains, $SSR$, and the part left over, $SSE$, and the coefficient of determination is the explained share:

$$R^2=\frac{SSR}{SST}=1-\frac{SSE}{SST}$$

Because $SSR$ can never exceed $SST$, the value of $R^2$ always sits between 0 and 1 — the fraction of the total variation the regression explains.

The second is the adjusted R square. The motivating observation was this: generally speaking, the more independent variables you add, the more $R^2$ increases. Moving from simple to multiple linear regression, you can easily witness $R^2$ going up. But there are situations where you add redundant independent variables — variables that contribute nothing to the dependent variable — and plain $R^2$ does not flag this: its nature is to keep climbing as variables are added. The adjusted $R^2$ is the measure that responds. The moment you add a redundant variable that contributes nothing to the dependent variable, you can see a change in the adjusted $R^2$.

The practical recommendation that followed was strong and unambiguous: whenever you work on regression problems — in course work or in your own analysis — it is always preferable to report the adjusted $R^2$ rather than simply quoting the plain $R^2$ value, because adjusted $R^2$ makes sense even when redundant variables are present.

10.1.3 Tests of Overall and Individual Significance

Two tests of significance were also introduced in the previous session. Whenever you talk about the overall significance of a regression model — whether simple or multiple — the F test reported in the ANOVA table is the place to look: it tells whether the model as a whole is significant. Alongside that, the t test output tells the individual significance — whether a particular coefficient is contributing to the model on its own. So: ANOVA/F for the model, t for each predictor.

When to pick which: run the F test first to see whether the model explains anything at all; then use the t tests to see which individual predictors matter.

10.1.4 Multicollinearity

The last recap point was the concept of multicollinearity, flagged as a very, very important aspect. The motivating scenario: in practice you are usually not working with two or three variables — you can be working with hundreds of variables in a specific problem, and that is where multicollinearity becomes a real concern.

The idea: we may say we are working on a dependent variable $y$ with two independent variables $x_1$ and $x_2$. But calling them "independent variables" does not mean they are statistically independent.

What happens when many highly correlated variables enter a model? You cannot get any extra information from including those related variables. So it should be avoided. The fundamental measure used to detect the problem is the correlation coefficient: you take the correlation matrix (correlation coefficient matrix) for all 100 variables, or however many there are, and you look for pairs of variables that are highly correlated — highly dependent. If such a pair exists, you retain only one of them, $x_1$ or $x_2$, depending on your problem: even if you add the other variable, you are not going to fetch any new information.

As a working rule, statisticians have settled on a threshold: at about 0.7, you can say a potential multicollinearity problem exists among those variables, and you should attempt to redefine your problem or redefine the model by retaining few variables and eliminating the rest.

10.2 Residual Analysis: Judging a Model by Its Errors

10.2.1 The Regression Model and the Residual

The agenda for this session started with a point about how conclusions should be substantiated. Numerical justifications and outputs are one way to address the appropriateness of a model — simple or multiple — but it is always recommended to infer, apart from the analysis-of-variance output, with the help of plots. The same logic that works for simple linear regression carries over to multiple regression along similar lines.

Recall the model: $y={\beta }_0+{\beta }_{1}x+\epsilon$, fitted as $\hat{y}=b_0+b_{1}x$. The residual is the difference between the observed value and the predicted value:

$$e_i=y_i-{\hat{y}}_i$$

In words: the residual for observation $i$ is "the observed value minus the predicted value." It is the vertical gap between the data point and the fitted line — the part of that observation the model could not explain. The observed value $y_i$ is the actual measurement for observation $i$; the predicted value ${\hat{y}}_i$ comes from plugging that observation's $x_i$ into the fitted equation.

There is an important attitude point here: getting the output is one aspect, and discussing it further is another. In Excel or SPSS outputs, all these plots can be produced with a single click — but producing the output is not the same as interpreting it. The click is the easy part; looking at the plots, checking them against the known patterns, and justifying the assumptions is the part that carries the analysis.

10.2.2 The Error Assumptions

Beyond fitting the model, there is a very important discussion to carry out: the error analysis. It fetches clarity into the entire discussion. The assumptions on the error term $\epsilon$ are:

$$E(\epsilon )=0\text{and}\mathrm{Var}(\epsilon )={\sigma }^2$$

In words: "the expected value of epsilon is zero," and "the variance of epsilon is sigma square" — the amount of variation is constant, which is what we call homogeneity of variance (the technical name is homoscedasticity). There are situations where the variance is not constant — that condition is called heteroscedasticity — but that was named and deferred, not developed.

The third assumption completes the set: the errors should behave independently and identically distributed with the normal distribution:

$$\epsilon \sim \text{i.i.d. }\mathcal{N}(0,{\sigma }^2)$$

So the error analysis justifies three things: (1) the errors have mean zero, (2) they capture an equal amount of variation (constant variance), and (3) they follow a normal distribution. To justify these aspects, different plots are available — all visible in Excel and SPSS outputs. Two plots address the homogeneity of variance aspect: the residual plots. Two more plots capture the information that the error follows a normal distribution. Many such justifications are required before suggesting a specific model and making conclusions.

10.2.3 Residual Plots, the Pizza Example, and the Three Classic Patterns

The running example for this part was the Armand's Pizza problem used earlier in the discussion of the coefficient of determination, where the complete variation was decomposed into its parts. The fitted line there was:

$$\hat{y}=60+5x$$

In words: "y hat is something like 60 plus 5x." Here $x$ is the size of the student population (in thousands) around a pizza parlor and $y$ is the quarterly sales (in thousands of dollars); a parlor in an area of $x$ thousand students is predicted to sell $60+5x$ thousand dollars of pizza per quarter. The residual for any observation is the observed value minus the predicted value, and a zero residual means a perfect fit with no deviation.

For this problem two residual plots were drawn. The first plots the residual against the independent variable $x$. The second plots the residual against the predicted values $\hat{y}$. Both plots convey information about the constancy of the variation. (In simple regression the two plots show the same pattern, so either one can be used; in multiple regression, where there are several $x$'s to choose from, the plot of residuals against the predicted values is the one more widely used.)

Visual intuition. Imagine a scatter diagram whose horizontal axis is $x$ (students in thousands) and whose vertical axis is the residual (in thousands of dollars of sales). Draw a horizontal reference line through zero. The ten points scatter above and below that line: the first point sits at $x=2$, $e=-12$ (below the line), the second at $x=6$, $e=15$ (above the line), and so on. The landmark to look for is the band: the points should form a roughly horizontal, shapeless band of about the same width at every $x$. That is what the pizza residual plots look like — no widening, no curve. Takeaway: the residual band hugs the zero line with constant width, which is the picture of constant variance.

In these plots there is a central line representing zero residuals: points above the line are positive residuals, points below it are negative residuals. On a raw plot alone you may not be able to conclude anything — which is why the discussion compared the plots against the three patterns that generally appear in residual studies:

• Panel A — constant variation: there is no unusual pattern. When your residual plot looks like this, you have a good pattern: the assumption $\mathrm{Var}(\epsilon )={\sigma }^2$ is being taken care of.
• Panel B — as $x$ increases, the variation keeps increasing (a funnel shape). Here there is no constant variation: the assumption of constant variance is not supported — a heteroscedasticity-type problem.
• Panel C — a completely different pattern, where the observations behave like a curvilinear curve: this signals a curvilinear specification problem — the linear form itself is wrong.

The conclusion for the pizza problem: both residual plots resembled Panel A, which justifies that the appropriateness of the regression model holds. Along with $R^2$, adjusted $R^2$, and the significance procedures, the residual plots give an idea of model quality — this is what one must know about residual plots.

10.2.4 Standardized Residual Plots

The next pair of plots addresses the normality assumption. Heavy calculations had been done (by the software) to produce them, and the discussion is deliberately visual: no theoretical depth, just inspection.

The standardized residual plot draws the standardized residuals against $x$. A standardized residual is the residual divided by its standard deviation: since the mean of the residuals is zero, standardizing means simply dividing each residual $e_i$ by its standard deviation,

$$\text{standardized residual}=\frac{y_i-{\hat{y}}_i}{s_{y_i-{\hat{y}}_i}}$$

where $s_{y_i-{\hat{y}}_i}$ is the estimated standard deviation of residual $i$. The key fact to know: if all the standardized residuals lie between +2 and −2, you have a reliable justification that the errors follow a normal distribution.

Visual intuition. Horizontal axis: the independent variable $x$. Vertical axis: the standardized residual, which carries no units (it is measured in standard deviations). The chart is framed by two horizontal landmark lines at +2 and −2. Points should sit inside the frame, scattered around zero, with no points breaking out beyond the lines and no funnel or curve. Takeaway: an all-inside-the-frame scatter is the visual fingerprint of normal errors.

10.2.5 Normal Probability Plots

On a similar note, the normal plot — also called the QQ plot (quantile-quantile plot) — is the companion check. All the observations should be clustered towards the line drawn at around a 45-degree angle. If everyone is very close to that line, with no huge variations, then this visual trait also explains that the error follows a normal distribution.

Visual intuition. The QQ plot pairs each ordered standardized residual with the value it would take if the data came from a normal distribution (the normal score). Plot the normal scores on the horizontal axis and the standardized residuals on the vertical axis. If the residuals are normal, the points fall close to the 45-degree line through the origin — the line where observed equals expected. Substantial curvature away from the line, or points straying far from it, is evidence against normality. Takeaway: points hugging the 45-degree line mean the normal assumption holds; systematic curvature means it does not.

The overall message: this error analysis gives sound clarity before you present any specific report, and the same logic prevails for multiple regression as well.

10.2.6 Where to Read More

For more detail, the textbook reference is: sections 14.8 and 15.8 (chapter 14, section 8, and chapter 15, section 8) contain the discussions of these residual plots, worked out in detail with the full problem discussion. If you want more insights on residual analysis, go through those specific sections.

10.3 Qualitative Independent Variables: Dummy Variables

10.3.1 The Johnson Filtration Problem

The multiple linear regression discussion so far serves the purpose when all predictors are numeric — but what if one of them is categorical? The Johnson Filtration problem was the vehicle for this. Johnson provides a maintenance service for water filtration systems throughout southern Florida. Customers contact Johnson with requests for maintenance service on their water filters. The Johnson manager wants to predict the repair time necessary for each maintenance request.

So the repair time in hours is the dependent variable, $y$. It is believed to be related to two factors: (1) the number of months since the last maintenance service, written MSLS, which is called $x_1$; and (2) the type of repair — which may be electrical or mechanical — called $x_2$. The data table showed 10 service centers, with the months since last service and the repair times in hours.

With these quantifications in hand, a simple linear regression $\hat{y}=b_0+b_1{x}_1$ using only the months since last service would serve the purpose — $x_1$ is a quantitative independent variable. But $x_2$ is not quantitative: the type of repair carries no numbers, and no numerical quantification exists for it. If you go with simple linear regression alone, you are simply ignoring the type-of-repair information entirely — the whole piece of information is useless to the model, even though it contributes to repair time.

10.3.2 Coding a Categorical Variable as a Dummy

How do you include a categorical (qualitative) independent variable such as type of repair in a multiple linear regression? The approach: you decode or code the categories with numbers. For this example, electrical repairs are coded as 1 and mechanical repairs as 0. The entire $x_2$ column is renamed into a sequence of ones and zeros — wherever an electrical repair is there, the entry is 1; otherwise 0. This is what is called the dummy variable concept.

The choice of 1 and 0 was made purely for ease of explanation. You could equally use 1 and 2, or 3 and 4 — but then you would have to interpret accordingly. The point: whenever you involve qualitative independent variables, code them with numbers; the combination of 1 and 0 keeps the interpretation simple, with 1 standing for "there exists an electrical repair" and 0 for a mechanical repair.

10.3.3 The Estimated Regression Equation

With the problem transformed, the multiple linear regression becomes:

$$\hat{y}=b_0+b_1{x}_1+b_2{x}_2$$

In words: "b0 plus b1 x1 plus b2 x2," where $x_1$ is MSLS and $x_2$ is the coded type of repair. The discussion compared two outputs: one that ignores the type of repair (because at first you do not know what to do with it) and one that includes the coded variable — and you can see the $R^2$ increasing in the second output, the behavior already expected from the $R^2$ discussion. In the book's output for this problem, $R^2$ jumps from about 53% with only the months variable to about 86% once the type of repair is added — the categorical information earns its place in the model.

The fitted equation used for interpretation was:

$$\hat{y}=0.9304+0.3876x_1+1.26x_2$$

In words: "y hat is 0.9304 plus 0.3876 times MSLS plus 1.26 times the type of repair."

10.3.4 Interpreting the Two Models

This is where interpretation demands care, because the categorical predictor splits the fitted equation into two specific models. Whenever you want to talk about an electrical repair, replace $x_2$ with 1; for a mechanical repair, replace $x_2$ with 0.

• Mechanical repair ($x_2=0$): the problem collapses to the simple linear form, $\hat{y}=b_0+b_1{x}_1$. Numerically:

$$\hat{y}=0.9304+0.3876x_1$$

• Electrical repair ($x_2=1$): the whole problem becomes $\hat{y}=(b_0+b_2)+b_1{x}_1$. Numerically:

$$\hat{y}=2.193+0.3876x_1$$

Carefully observing these two equations gives the key impressions:

• The slope $b_1$ is the same in both contexts — whether mechanical or electrical, there is no change in the slope. Thinking of the line $y=mx+c$, where $m$ talks about the slope and $c$ talks about the intercept: the slope does not change across repair types.
• The intercept changes: for mechanical repair it is $b_0$; for electrical repair it is $b_0+b_2$. So the coefficient $b_2$ is what separates the mean repair times of electrical and mechanical repairs.

The coefficient $b_2$ drives the conclusion about mean repair times:

• If $b_2$ is positive: the mean repair time for an electrical repair will be greater than the mean repair time for a mechanical repair.
• If $b_2$ is negative: the mean repair time for an electrical repair is less than the mean repair time for a mechanical repair.
• If $b_2=0$: there is no difference between the repair times across the two types.

10.3.5 What the Graph Shows

Visual intuition. Draw a scatter plot with months since last service ($x_1$) on the horizontal axis and repair time in hours ($y$) on the vertical axis. Mark each mechanical repair with an M and each electrical repair with an E. Superimpose the two fitted lines: the mechanical line $\hat{y}=0.9304+0.3876x_1$ and the electrical line $\hat{y}=2.193+0.3876x_1$. Both rise with the same slope $0.3876$, so the lines are parallel; the electrical line sits everywhere $1.26$ hours above the mechanical line. The landmarks: on the vertical axis the mechanical line starts at 0.93 and the electrical line at 2.19 — a vertical gap of 1.26 that holds at every value of $x_1$. Takeaway: the entire difference between the two categories is captured in the shift of the intercept.

Graphically, the two fitted lines are parallel: the same slope $b_1$ with a vertical separation of $b_2$ between the mechanical line (intercept $b_0$) and the electrical line (intercept $b_0+b_2$). The entire difference between the two categories is captured in the shift of the intercept.

10.3.6 Exam Notes

This is where the dummy-variable trick pays off in practice: repair-time prediction for maintenance fleets, customer-service planning, and any industry model that mixes numeric drivers with categorical factors (machine type, region, shift, supplier). Categorical predictors are everywhere in applied work, and the dummy coding shown here is the standard way to give the model access to them.

10.4 Logistic Regression: When the Dependent Variable Is Dichotomous

10.4.1 Why Linear Regression Fails for Zero–One Outcomes

So far the analysis covered multiple linear regression with quantitative variables, and then the dummy-variable extension for a categorical independent variable. But there are situations where the dependent variable itself is categorical — a dichotomous response that takes only two possible values, 0 and 1.

If you apply simple or multiple linear regression to such a $y$, the predictions can exceed the range of 0 and 1 — $y$ may come out more than 1 or less than 0 — and that is completely meaningless when $y$ takes only the values 0 and 1. Linear regression is simply not helpful here, and this is not advisable. Because of this behavior of the dependent variable, the old reading of a coefficient also fails: "a one-unit increase in $x_1$ will change $y$ by $b_1$" is not the way to talk about these models. Everything is better interpreted in terms of probabilities instead.

10.4.2 Real Examples: Credit Card Approval and Spam Detection

Two motivating situations were given. First, a credit card request: whether the bank approves the credit card or rejects it — the decision takes only two possible values, approving or rejecting, so $y$ is mapped to 0 and 1. Second, spam detection: whether a given email is spam or not — again $y$ takes two values, with many factors modeled against it. These days logistic regression is also used as a classification technique.

The summary rule: logistic regression is the appropriate regression analysis to conduct when the dependent variable is dichotomous. Like all regressions it is still a predictive regression, but some mathematical violations are expected in it — which is exactly why a different expression is needed and why the interpretation cannot run the same way.

10.4.3 The Logistic Regression Equation

The expression used to capture this zero–one behavior is the sigmoidal function — the S-shaped curve you observe between 0 and 1. The logistic regression equation is:

$$p=\frac{e^{{\beta }_0+{\beta }_1{x}_1+\cdots +{\beta }_p{x}_p}}{1+e^{{\beta }_0+{\beta }_1{x}_1+\cdots +{\beta }_p{x}_p}}$$

In words: "e power beta 0 plus beta 1 x1 plus beta p xp, divided by 1 plus e power beta 0 plus beta 1 x1 plus beta p xp." Here $p$ denotes the probability of the positive outcome, and the term in the exponent is the linear combination of the predictors. Whenever you are asked to write a logistic regression equation, you should express everything in this form.

If there is only one independent variable, the expected value ranges from 0 to 1, and the S-shaped expression best suits that behaviour: as the value of $x$ keeps increasing, you approach 1; and in the other direction, you approach 0.

Visual intuition. Horizontal axis: the single independent variable $x$. Vertical axis: $p$, the probability of the positive outcome, scaled from 0 to 1. The curve is a stretched S: it runs nearly flat near 0 for small $x$, climbs steeply through the middle, then flattens again near 1 for large $x$. Its landmarks are the two horizontal asymptotes — $p=0$ and $p=1$ — which the curve approaches but never touches. For the parameter values $b_0=-7$, $b_1=3$, the curve gives about 0.02 at $x=1$, about 0.27 at $x=2$, and about 0.88 at $x=3$ — the steep middle stretch is where a small change in $x$ changes the probability a lot. Takeaway: the S-curve is the probability shape — bounded by 0 and 1, steepest in the middle.

The mathematical aspects — why we take this exponential function and how it takes care of the zero–one $y$ values — were flagged as a point to be developed in the next session.

When to pick which: use linear regression for a continuous response; switch to logistic regression the moment the response is a 0–1 outcome.

10.4.4 The Expected Value as a Probability

The single most important interpretive principle: in logistic regression the expected value is always interpreted as a probability. Precisely:

$$E(y)=P(y=1\mid x_1,x_2,\dots ,x_p)$$

In words: the expected value of $y$ given the set of independent variables is the probability that $y$ equals 1 given those variables. Every subsequent discussion in logistic regression — everything involving the p values and probability values — rides on this identification. Even the very basic expression of the expected value itself already indicates that it is talking about the probability that $y=1$ given the set of independent variables.

10.4.5 Student Questions and Answers

10.5 The Simon's Coupon Example: Fitting Logistic Regression in R

10.5.1 The Data

The worked example came from the book: a catalog retailer (called Simon's in the discussion; the book names it Simmons Stores), which runs a national chain of women's apparel stores, tracked whether customers used the coupon they received. The full data file has 100 customers; only the sample of the first 10 catalog recipients was shown for illustration, and the same data file was promised to be shared.

The variables to keep track of — relate $x_1$, $x_2$, and $y$ to these descriptions in all further discussion:

• $y$ — the coupon column: 1 indicates the customer used the coupon, 0 indicates the customer did not use the coupon. This is the dependent variable.
• $x_1$ — the amount spent at Simon's last year, in thousand dollars (annual spending).
• $x_2$ — the credit card information, coded as 0 and 1 (whether the customer has a credit card: 1 = has, 0 = does not).

The first ten rows of the data, exactly as they appear in the book:

For a model with these predictors, the multiple linear regression form $E(y)={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2$ will not serve the purpose: $y$ is not continuous, it takes only 0 and 1, and the linear predictions can exceed 1 or fall below 0. So everything is expressed in terms of probabilities, and the logistic regression equation takes over.

10.5.2 The R Workflow

The demonstration was run live in R, because the data file was not at hand at that moment. The steps, exactly as shown:

data <- read.csv(file.choose(), header = TRUE)
fit <- glm(y ~ x1 + x2, data = data, family = binomial)
summary(fit)

The live demo used 10 sample observations, while the full output discussed came from the complete 100-observation file.

10.5.3 The Fitted Equation

From the output, the fitted coefficients were read off directly:

$$b_0=-2.146,b_1=0.3416,b_2=1.0987$$

Because this is a logistic regression, the estimated regression equation is written in the exponential form, replacing ${\beta }_0,{\beta }_1,{\beta }_2$ with the fitted values:

$$\hat{p}=\frac{e^{-2.146+0.3416x_1+1.0987x_2}}{1+e^{-2.146+0.3416x_1+1.0987x_2}}$$

In words: "e power b0 plus b1 x1 plus b2 x2 over 1 plus e power b0 plus b1 x1 plus b2 x2," with the coefficients replaced by the numbers above. At this point the equation was merely written down; the interpretation of the coefficients — what they contribute — was the next step.

10.5.4 Reading the Output Differently

The key caution for this problem: the coefficients ${\beta }_0,{\beta }_1$ will not give you the direct information they give in linear regression. There, a one-unit increase in $x_1$ influences $y$ in a directly readable way; here, the same logic does not work because of the probability-based behaviour of the dependent variable. With logistic regression you need much more careful observation to conclude anything about the predictors.

The discussion previewed what comes next: odds and the odds ratio — the concepts that will carry the interpretation of $b_1$ and $b_2$ — along with the basic interpretations that are very much required and very much different from ordinary regression concepts. The managerial questions posed for the Simon's problem will be answered from this perspective.

10.5.5 Student Questions and Answers

10.6 A Real Research Application: Consanguinity and Autism Spectrum Disorder

10.6.1 The 2014 Short Report

Before the technical treatment of logistic regression, a personal research experience was shared as motivation — a short report published in 2014 in a peer-reviewed periodical (a Wiley publication; the journal name did not come through exactly in the discussion, but the report itself was shared for reference). The point of the story: even a research team could rely on the very technique being introduced today, mapped onto a real medical question.

The research concerned autism spectrum disorder, a disorder in kids, and its association with consanguinity in India — the question of whether parents being related to each other (for example, cousins) is associated with autism in the child. Consanguinity (from "con-" meaning together and "sanguis" meaning blood) refers to a biological relationship between the parents; the study asked whether that family structure changes the odds that a child develops the disorder. The team was interdisciplinary: biology faculty, a mathematician, and some doctors. One researcher, while pursuing her PhD, had carried out research on the autism spectrum in the context of India.

The statistical work used — at that point in time — simple logistic regression: many factors were considered, and whether the individual child has consanguinity or not, and other such attributes, were mapped to dichotomous $y$ values and regressed against the different factors, exactly in the spirit of the current discussion. The team substantially justified the discussion and proved the association with consanguinity, which was anyway true.

10.7 Roadmap for the Next Session

10.7.1 Odds, Odds Ratios, and Interpretation

The next session will continue with the Simon's credit card problem from this juncture: the basic concepts, properties, and interpretations of the coefficients — how odds and the odds ratio matter and how they will be addressed. The managerial questions posed for the problem will then be answered. The discussion was expected to take about half an hour to forty minutes.

10.7.2 Dimensionality Reduction: PCA and LDA

After the interpretation of logistic regression, the focus will shift to two dimensionality-reduction techniques: principal component analysis (PCA) and linear discriminant analysis (LDA). An element of linear algebra will be required — eigenvalues and eigenvectors will come into the picture, especially for PCA. The view offered: to understand these reduction techniques, linear algebra gives much more insight than merely calculating. Because of the non-mathematics background of many in the audience, the material will be framed so that it is neither too mathematical nor too far away from mathematics.

Exam Guidance Summary

Mid-semester paper. The evaluation of the mid-semester scripts is underway. The key (solutions with the marking scheme) will be shared by Monday — go through it, relate it to your marks, and raise any queries. If you feel a correct answer was not credited, that is what a rechecking request is for. The framing came from the two types of error in hypothesis testing: if the question is correct and zero is awarded, that is one error committed; if the question is wrong and marks are awarded, that is the other error. Both errors are being taken care of; scripts submitted by mail will also be considered. The sample-size determination problem in quiz one reportedly had a wrong option — for that particular problem, the majority will get credit for that point.

Quizzes. Each quiz allows only one attempt; no further attempts are allowed. A huge duration is given, so complete the quiz within the stipulated time.

Assignment. The assignment component works in groups: groups of 7 to 10 (the total strength is around 400–440, so the final size will depend on how many turn up), a very basic dataset, a simple analysis, and a single report submitted as a whole team. There is not much variation in what can be expected, since everyone performs at their own level — the best you present, the best credits you get. A drive link will be shared where each group adds the ID numbers so the marks can be reflected for the entire group.

Exam content signals. For the exam, expect fitted regression outputs that you must analyze — an estimated equation like $b_0+0.340x_1$ given in the output (digits as they appear in the output), with the task of writing the regression equation and interpreting the coefficients. For regression work, remember: report adjusted $R^2$ rather than plain $R^2$; use the ANOVA F test for overall significance and the t test for individual significance; check residual plots against the three classic patterns (Panel A constant variance, Panel B increasing variance, Panel C curvilinear); standardized residuals within +2 to −2 and normal plot points close to the 45-degree line support the normality assumption; and for dummy variables, know that the slope stays the same while the intercept shifts by the dummy coefficient.

Key Industry Applications

• Real-world: Johnson Filtration — a maintenance service for water filtration systems throughout southern Florida — predicts repair time in hours from months since last service and type of repair (electrical vs. mechanical), using a dummy variable.
• Real-world: Simon's catalog coupon targeting — predicting from spending and credit card status whether a customer will use a coupon (dichotomous $y$), modeled with logistic regression in R.
• Real-world: credit card approval decisions — approve or reject, a two-decision problem mapped to 0 and 1.
• Real-world: spam detection — classifying an email as spam or not spam; logistic regression used as a classification technique.
• Real-world: medical/epidemiological research — the 2014 short report on consanguinity and autism spectrum disorder in India, which used simple logistic regression to map dichotomous outcomes to risk factors.
• Real-world: software practice — logistic regression is fit directly in R (the glm function with family = binomial) and SPSS; Excel has no complete built-in logistic output; Excel data should be converted to CSV before reading into R.
• Reference: textbook sections 14.8 and 15.8 contain the detailed residual analysis discussion.