*Covariance* measures the *direction* of a linear relationship between two variables, $X$ and $Y$. It tells you whether they move in the same direction, opposite directions, or independently. - **Positive covariance** ($\mathrm{Cov}(X,Y)>0$): as $X$ increases, $Y$ tends to increase. Both move in the same direction. - **Negative covariance** ($\mathrm{Cov}(X,Y)<0$): as $X$ increases, $Y$ tends to decrease. They move in opposite directions. - **Zero covariance** ($\mathrm{Cov}(X,Y)=0$): no *linear* relationship exists. The variables may still have a nonlinear connection. Covariance is the raw, un-normalized measure. It tells you direction only — never strength. A covariance of 100 could signal a tight relationship or a loose one, depending entirely on the scales of $X$ and $Y$.

The sample covariance between $X$ and $Y$ is: $$\mathrm{Cov}(X,Y)=\frac{{\sum }_{i=1}^n(X_i-\bar{X})(Y_i-\bar{Y})}{n-1}$$ Here's how to read it, term by term: - $X_i-\bar{X}$ is the deviation of the $i$-th $X$ value from the mean of all $X$'s. - $Y_i-\bar{Y}$ is the deviation of the $i$-th $Y$ value from the mean of all $Y$'s. - Multiply these two deviations for each data point. When both are on the same side of their means (both positive or both negative), the product is positive. When they're on opposite sides, the product is negative. - Sum all of these products across all $n$ data points. - Divide by $n-1$ (sample covariance; for population covariance, divide by $n$). The sign of the result — positive, negative, or zero — is the directional signal.

The *correlation coefficient* (Pearson's $r$) measures both the direction and the *strength* of a linear relationship. It normalizes covariance by dividing by the product of the two standard deviations, forcing the result into the interval $[-1,+1]$. Interpretation of $r$: - $r=+1$: perfect positive linear relationship. All points lie exactly on an upward-sloping line. - $r=-1$: perfect negative linear relationship. All points lie exactly on a downward-sloping line. - $r=0$: no linear relationship — but a nonlinear relationship may still exist. - $|r|\ge 0.75$: strong linear correlation. - $0<|r|<0.75$: moderate to weak linear correlation. The critical nuance: $r=0$ means *no linear relation*, not "no relation at all." Two variables can have $r=0$ yet be perfectly connected through a curve.

The Pearson correlation coefficient is the normalized covariance: $$r=\frac{\mathrm{Cov}(X,Y)}{{\sigma }_X{\sigma }_Y}$$ where ${\sigma }_X=\sqrt{\frac{\sum (X_i-\bar{X}{)}^2}{n-1}}$ and ${\sigma }_Y=\sqrt{\frac{\sum (Y_i-\bar{Y}{)}^2}{n-1}}$ are the sample standard deviations. Expanded form — useful for direct computation from raw data: $$r=\frac{{\sum }_{i=1}^n(X_i-\bar{X})(Y_i-\bar{Y})}{\sqrt{{\sum }_{i=1}^n(X_i-\bar{X}{)}^2{\sum }_{i=1}^n(Y_i-\bar{Y}{)}^2}}$$ Why the range $[-1,+1]$? The denominator ${\sigma }_X{\sigma }_Y$ is the maximum possible absolute value the numerator can take (by the Cauchy-Schwarz inequality). When the numerator equals the denominator, you get $\pm 1$. When it's a fraction of it, you get something between $-1$ and $+1$. Both forms are algebraically identical. Use whichever is more convenient given your data.

Given $k$ variables $X_1,X_2,\dots ,X_k$, the *variance-covariance matrix* (or simply the *covariance matrix*) $\Sigma$ is a $k\times k$ matrix where: $${\Sigma }_{ij}=\mathrm{Cov}(X_i,X_j)$$ In expanded form for $k=3$: $$\Sigma =\left[\begin{array}{ccc}\mathrm{Var}(X_1)& \mathrm{Cov}(X_1,X_2)& \mathrm{Cov}(X_1,X_3)\\ \mathrm{Cov}(X_2,X_1)& \mathrm{Var}(X_2)& \mathrm{Cov}(X_2,X_3)\\ \mathrm{Cov}(X_3,X_1)& \mathrm{Cov}(X_3,X_2)& \mathrm{Var}(X_3)\end{array}\right]$$

Three defining properties of the covariance matrix: 1. **Symmetric.** $\mathrm{Cov}(X_i,X_j)=\mathrm{Cov}(X_j,X_i)$, so $\Sigma ={\Sigma }^T$. The matrix equals its own transpose. 2. **Diagonal entries are variances.** $\mathrm{Cov}(X_i,X_i)=\mathrm{Var}(X_i)$. So the diagonal of $\Sigma$ contains the variance of each variable — a measure of how spread out each variable is on its own. 3. **Positive semi-definite.** For any non-zero vector $\mathbf{a}$, ${\mathbf{a}}^T\Sigma \mathbf{a}\ge 0$. This is a mathematical guarantee that the matrix behaves properly — you'll never get a negative variance from any linear combination of the variables.

**Principal Component Analysis (PCA):** PCA is a dimensionality reduction technique. It finds new axes (principal components) that capture the directions of maximum variance in the data. The computation works directly on the variance-covariance matrix: 1. Compute the covariance matrix $\Sigma$ of the (centered) data. 2. Find the eigenvalues ${\lambda }_1\ge {\lambda }_2\ge \cdots \ge {\lambda }_k$ and corresponding eigenvectors ${\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_k$ of $\Sigma$. 3. The eigenvectors are the principal components — new uncorrelated directions. 4. The eigenvalue ${\lambda }_i$ tells you how much variance the $i$-th principal component captures. Everything in PCA flows from the covariance matrix. If you understand $\Sigma$, you understand PCA.

**Correlation** answers three questions about X and Y: - Are they related? (is $r$ significantly different from 0?) - In which direction? (positive or negative?) - How strongly? (how close is $|r|$ to 1?) **Regression** takes the next step. It answers: - What is the exact mathematical form of the relationship? - If X changes by one unit, by how much does Y change on average? - Given a new value of X, what is my best prediction for Y? The sequence is: correlation first (does a relationship exist?), then regression (what is its equation?).

The purpose of regression is **prediction**. Once you have the equation — say, $\hat{Y}=5+2X$ — you can plug in any X and get a predicted Y. Correlation cannot do this. A correlation of $r=0.9$ tells you the relationship is strong, but it does not tell you what value of Y to expect when $X=10$. Regression also serves a second purpose: **explanation**. The coefficients tell you how much Y changes per unit change in X, holding other factors constant (in multiple regression). This is why regression is the workhorse of scientific inference — it quantifies effects.

A regression is *linear* if the parameters ($W_0,W_1,W_2,\dots$) appear linearly — not raised to powers, not inside nonlinear functions like $\sin$ or $\log$. The model: $$Y=W_0+W_{1}X$$ is linear. So is $Y=W_0+W_{1}X+W_2{X}^2$ — it's "polynomial regression" but still *linear in the parameters* $W_0,W_1,W_2$. The word "linear" in "linear regression" refers to linearity in the parameters, not linearity in $X$. A regression is *nonlinear* when the parameters themselves appear inside nonlinear functions. Examples: $Y=W_0{e}^{W_{1}X}$, $Y=\frac{W_0}{1+W_{1}X}$, or $Y=W_0\sin (W_{1}X)$. In this course, "linear regression" means the parameters enter linearly. That includes polynomial regression with terms like $X^2,X^3$.

*Simple linear regression* models the relationship between one independent (predictor) variable $X$ and one dependent (response) variable $Y$ as: $$Y=W_0+W_{1}X$$ - $W_0$ is the **intercept** — the value of $Y$ when $X=0$. It sets the vertical position of the line. - $W_1$ is the **slope** — the change in $Y$ for a one-unit increase in $X$. Positive $W_1$ means an upward-sloping line; negative means downward. - "Simple" = one predictor. With multiple predictors, it becomes *multiple linear regression*.

Given a scatter of $n$ data points $(X_i,Y_i)$, infinitely many lines could pass through or near them. Each line corresponds to a different choice of $(W_0,W_1)$. The question: which pair $(W_0,W_1)$ is best? The intuitive answer: the line that passes *closest to the maximum number of points*. More formally: the line that minimizes the total prediction error. For a line with parameters $(W_0,W_1)$, the predicted value at $X_i$ is: $${\hat{Y}}_i=W_0+W_1{X}_i$$ The error (also called the *residual*) at point $i$ is the vertical gap between the actual $Y_i$ and the predicted ${\hat{Y}}_i$: $$e_i=Y_i-{\hat{Y}}_i=Y_i-(W_0+W_1{X}_i)$$ The goal: choose $W_0$ and $W_1$ to make the total error across all $n$ points as small as possible. But what does "total error" mean? You can't just sum the raw errors — positive and negative errors would cancel. The next section tackles this.

Squaring the errors gives the *Sum of Squared Errors* (SSE): $$\text{SSE}=\sum _{i=1}^n(Y_i-{\hat{Y}}_i{)}^2=\sum _{i=1}^n(Y_i-(W_0+W_1{X}_i){)}^2$$ SSE is also called the *residual sum of squares* (RSS) or the *error sum of squares*. **Three reasons squaring wins:** 1. **No cancellation.** Squares are always $\ge 0$. Positive and negative errors both contribute. A large error (in either direction) contributes a lot. 2. **Differentiable everywhere.** The function $f(e)=e^2$ is smooth — no kinks, no corners. You can take derivatives at every point, making calculus-based optimization clean. 3. **Convexity — the key property.** The SSE function, as a function of $W_0$ and $W_1$, is a convex quadratic. It curves upward everywhere, like a bowl. It has exactly one minimum — the global minimum. Take the derivative, set it to zero, solve, and you are guaranteed to be at the bottom of the bowl. No need to check second derivatives or worry about getting stuck in a local dip. The professor emphasized convexity as *the* reason SSE dominates. With a general function, you must: compute first derivatives → find critical points → compute second derivatives → classify each as min, max, or saddle. The SSE function saves you from all of that.

Taking the average of the squared errors gives the *Mean Squared Error*: $$\text{MSE}=\frac{1}{n}\sum _{i=1}^n(Y_i-{\hat{Y}}_i{)}^2$$ MSE is just SSE divided by $n$. Since dividing by a positive constant doesn't change where the minimum occurs, minimizing SSE and minimizing MSE give the exact same optimal $W_0,W_1$. The choice between them is about interpretability — MSE is in "squared-error-per-data-point" units, which is easier to compare across datasets of different sizes.

The loss function to minimize: $$L(W_0,W_1)=\sum _{i=1}^n(Y_i-W_0-W_1{X}_i{)}^2$$ To find the optimal $W_0$ and $W_1$, compute the two partial derivatives, set each to zero, and solve the resulting system.

Differentiate $L$ with respect to $W_0$, treating $W_1$ as constant: $$\begin{array}{rl}\frac{\partial L}{\partial W_0}& =\sum _{i=1}^{n}2(Y_i-W_0-W_1{X}_i)\cdot \frac{\partial }{\partial W_0}(Y_i-W_0-W_1{X}_i)\\ & =\sum _{i=1}^{n}2(Y_i-W_0-W_1{X}_i)\cdot (-1)\\ & =-2\sum _{i=1}^n(Y_i-W_0-W_1{X}_i)\end{array}$$ Set equal to zero and simplify: $$\begin{array}{rl}-2\sum (Y_i-W_0-W_1{X}_i)& =0\\ \sum (Y_i-W_0-W_1{X}_i)& =0\\ \sum Y_i-\sum W_0-W_1\sum X_i& =0\\ \sum Y_i-n{W}_0-W_1\sum X_i& =0\end{array}$$ Rearranging gives the **first normal equation**: $$\boxed{\sum _{i=1}^n{Y}_i=n{W}_0+W_1\sum _{i=1}^n{X}_i}\text{\hspace{1em}(1)}$$ Professor's verbal walkthrough: "We differentiate L with respect to W₀. Chain rule: 2 times the bracket times the derivative of inside. The derivative of inside with respect to W₀ is −1. Set equal to 0, remove the −2. You get: sum of Y equals n times W₀ plus W₁ times sum of X."

Differentiate $L$ with respect to $W_1$, treating $W_0$ as constant: $$\begin{array}{rl}\frac{\partial L}{\partial W_1}& =\sum _{i=1}^{n}2(Y_i-W_0-W_1{X}_i)\cdot \frac{\partial }{\partial W_1}(Y_i-W_0-W_1{X}_i)\\ & =\sum _{i=1}^{n}2(Y_i-W_0-W_1{X}_i)\cdot (-X_i)\\ & =-2\sum _{i=1}^n{X}_i(Y_i-W_0-W_1{X}_i)\end{array}$$ Set equal to zero and simplify: $$\begin{array}{rl}-2\sum X_i(Y_i-W_0-W_1{X}_i)& =0\\ \sum X_i{Y}_i-W_0\sum X_i-W_1\sum X_i^2& =0\end{array}$$ Rearranging gives the **second normal equation**: $$\boxed{\sum _{i=1}^n{X}_i{Y}_i=W_0\sum _{i=1}^n{X}_i+W_1\sum _{i=1}^n{X}_i^2}\text{\hspace{1em}(2)}$$ Professor's verbal walkthrough: "Differentiating with respect to W₁, the chain rule brings down an Xᵢ from the inside derivative. So it is equivalent to multiplying the original equation throughout by Xᵢ and then summing."

You now have two linear equations in two unknowns $(W_0,W_1)$: $$\{\begin{array}{l}n{W}_0+\left(\sum X_i\right)W_1=\sum Y_i\\ \left(\sum X_i\right)W_0+\left(\sum X_i^2\right)W_1=\sum X_i{Y}_i\end{array}$$ All the summation terms — $\sum X_i,\sum Y_i,\sum X_i^2,\sum X_i{Y}_i$ — you compute directly from your data. Then solve. **Matrix form:** $$\underset{A}{\underbrace{\left[\begin{array}{cc}n& \sum X_i\\ \sum X_i& \sum X_i^2\end{array}\right]}}\underset{\mathbf{w}}{\underbrace{\left[\begin{array}{c}{W}_0\\ W_1\end{array}\right]}}=\underset{\mathbf{b}}{\underbrace{\left[\begin{array}{c}\sum Y_i\\ \sum X_i{Y}_i\end{array}\right]}}$$ The solution: $$\mathbf{w}=A^{-1}\mathbf{b}$$ This is exactly what regression libraries do internally: build the $2\times 2$ matrix $A$, compute its inverse, and multiply by $\mathbf{b}$ to get $(W_0,W_1)$. **Direct (shortcut) formulas.** Solving the system algebraically without matrix inversion gives the well-known formulas: $$W_1=\frac{\sum (X_i-\bar{X})(Y_i-\bar{Y})}{\sum (X_i-\bar{X}{)}^2}=\frac{\sum X_i{Y}_i-n\bar{X}\bar{Y}}{\sum X_i^2-n{\bar{X}}^2}$$ $$W_0=\bar{Y}-W_1\bar{X}$$ The slope $W_1$ is the ratio of the sample covariance (numerator) to the sample variance of $X$ (denominator). The intercept $W_0$ centers the line so it passes through the point of means $(\bar{X},\bar{Y})$.

The loss function $L(W_0,W_1)=\sum (Y_i-W_0-W_1{X}_i{)}^2$ is a convex quadratic. The Hessian matrix (matrix of second partial derivatives) is: $$H=\left[\begin{array}{cc}\frac{{\partial }^{2}L}{\partial W_0^2}& \frac{{\partial }^{2}L}{\partial W_0\partial W_1}\\ \frac{{\partial }^{2}L}{\partial W_1\partial W_0}& \frac{{\partial }^{2}L}{\partial W_1^2}\end{array}\right]=\left[\begin{array}{cc}2n& 2\sum X_i\\ 2\sum X_i& 2\sum X_i^2\end{array}\right]$$ This matrix is positive definite (all its eigenvalues are positive), which confirms global convexity. The critical point from setting first derivatives to zero is guaranteed to be the unique global minimum. No second-derivative test is needed. This is a deliberate design choice — SSE was selected specifically because it has this property.

The normal equations give a **closed-form solution**: one computation, exact answer (up to numerical precision). This contrasts with **gradient descent**, an iterative method that takes small steps downhill until it converges. | Aspect | Normal Equations | Gradient Descent | |---|---|---| | How it works | Solve $A\mathbf{w}=\mathbf{b}$ directly | Iteratively update $\mathbf{w}:=\mathbf{w}-\alpha \nabla L$ | | Result | Exact (to machine precision) | Approximate (converges gradually) | | Speed with few features | Very fast | Slower | | Speed with many features | Slow — matrix inversion is $O(k^3)$ | Faster — each step is $O(nk)$ | | Requires learning rate? | No | Yes — must tune $\alpha$ | | Works for non-convex loss? | No | Yes | For linear regression with a modest number of features, the normal equations are the gold standard. For models with thousands of features or non-convex loss functions (like neural networks), gradient descent is the only practical option.

An alternative that avoids solving the system of equations: use the pre-derived shortcut formulas. First compute the means: $$\bar{X}=\frac{\sum X}{n}=\frac{92}{8}=11.5,\bar{Y}=\frac{\sum Y}{n}=\frac{949}{8}=118.625$$ Then the slope: $$W_1=\frac{\sum XY-n\bar{X}\bar{Y}}{\sum X^2-n{\bar{X}}^2}=\frac{12510-8(11.5)(118.625)}{1368-8(11.5{)}^2}=\frac{12510-10913.5}{1368-1058}=\frac{1596.5}{310}=5.15$$ Then the intercept: $$W_0=\bar{Y}-W_1\bar{X}=118.625-5.15(11.5)=118.625-59.225=59.4$$ Same results. The direct formula is computationally cleaner — it's what most software uses internally.

*Multiple linear regression* extends simple linear regression to $k$ predictor variables $X_1,X_2,\dots ,X_k$: $$Y=W_0+W_1{X}_1+W_2{X}_2+\cdots +W_k{X}_k$$ - $W_0$ is the intercept — the predicted $Y$ when all $X_j=0$. - $W_j$ (for $j\ge 1$) is the *partial slope* — the change in $Y$ for a one-unit increase in $X_j$, holding all other predictors constant. - "Multiple" = multiple input features. The model is still linear in the parameters $W_j$.

The same least-squares logic scales naturally. With $k$ predictors, the model is: $$Y=W_0+W_1{X}_1+W_2{X}_2+\cdots +W_k{X}_k$$ The loss function is still SSE: $$L=\sum _{i=1}^n(Y_i-W_0-W_1{X}_{i1}-W_2{X}_{i2}-\cdots -W_k{X}_{ik}{)}^2$$ Taking partial derivatives with respect to each $W_j$ gives $k+1$ normal equations: - Differentiating with respect to $W_0$ gives an equation with $\sum Y$ on the left. - Differentiating with respect to $W_1$ brings down $X_1$ — equivalent to multiplying by $X_{i1}$ and summing. - Differentiating with respect to $W_2$ brings down $X_2$ — same pattern. - And so on for all $k$ predictors. In matrix form: $\mathbf{w}=(X^{T}X{)}^{-1}{X}^T\mathbf{y}$, where $X$ is the $n\times (k+1)$ design matrix (first column of 1's for $W_0$), and $\mathbf{y}$ is the $n\times 1$ response vector. The matrix $A=X^{T}X$ is $(k+1)\times (k+1)$.

Three quantities partition the total variation in $Y$: $$\begin{array}{rlrl}\text{SST}& =\sum _{i=1}^n(Y_i-\bar{Y}{)}^2& & \text{Total Sum of Squares — total variation in }Y\\ \text{SSR}& =\sum _{i=1}^n({\hat{Y}}_i-\bar{Y}{)}^2& & \text{Regression Sum of Squares — variation explained by the model}\\ \text{SSE}& =\sum _{i=1}^n(Y_i-{\hat{Y}}_i{)}^2& & \text{Error Sum of Squares — variation left unexplained}\end{array}$$ These three are related by the fundamental identity: $$\text{SST}=\text{SSR}+\text{SSE}$$ Every bit of variation in $Y$ is either explained by the regression (SSR) or left as error (SSE). Nothing is lost and nothing is double-counted.

$R^2$ is the proportion of total variation explained by the model: $$R^2=\frac{\text{SSR}}{\text{SST}}=1-\frac{\text{SSE}}{\text{SST}}$$ Both forms are algebraically identical (since SST = SSR + SSE). The first form expresses the fraction *explained*. The second says "1 minus the fraction *unexplained*." Use whichever is more intuitive. $R^2$ always falls in $[0,1]$ for models fit with least squares (it can technically be negative for models fit on data not used in training — but that's beyond ISM scope).

Divide the first normal equation $\sum Y=n{W}_0+W_1\sum X$ by $n$: $$\bar{Y}=W_0+W_1\bar{X}$$ This proves the regression line always passes through $(\bar{X},\bar{Y})$. Now here's the baseline logic: - If you knew nothing about $X$, your best guess for any $Y$ would be the mean $\bar{Y}$. The total error of this naive strategy is SST. - If you use the regression line, your error is SSE. - The difference, SST − SSE = SSR, is how much the regression *improves* over the naive guess. - $R^2=\frac{\text{SST}-\text{SSE}}{\text{SST}}=\frac{\text{SSR}}{\text{SST}}$ is the fraction of the naive error that the regression eliminates.

When you add more predictors, $R^2$ never decreases — it always stays the same or increases, even if the new predictor is pure noise. This creates an illusion of improvement. **Adjusted $R^2$** fixes this by penalizing the number of predictors: $$R_{\text{adj}}^2=1-\frac{\text{SSE}/(n-k-1)}{\text{SST}/(n-1)}$$ where $k$ is the number of predictors. - Use plain $R^2$ for **simple linear regression** (one predictor). - Use **Adjusted $R^2$** for **multiple linear regression** (multiple predictors). Adjusted $R^2$ only increases when the new predictor genuinely improves the model more than you'd expect from random chance. If adding a useless predictor, Adjusted $R^2$ actually *decreases*.

In a real project, you never know the true relationship in the data. The decision process: 1. **Start with simple linear regression.** It is the cheapest, fastest, and most interpretable model. Always. 2. **Check its performance.** If $R^2$ (or another metric) is acceptable — say 80% — and the customer is satisfied, **stop**. There is no need for complexity. 3. **If the customer is not satisfied**, move to polynomial regression. Try degree 2 (quadratic: $Y=W_0+W_{1}X+W_2{X}^2$). Check performance. 4. **If still not enough**, try degree 3 (cubic). Continue increasing the degree. 5. **Stop when the customer is happy** with the performance-cost tradeoff — or when you hit diminishing returns.

A high-degree polynomial might give 98% accuracy vs. linear regression's 80%. But: - **More computation.** Higher-degree polynomials need more training time and more memory. - **More complexity.** Harder to explain to stakeholders. "Sales = 50 + 3×price" is clear. "Sales = 50 + 3×price − 0.1×price² + 0.002×price³" is not. - **More risk of overfitting.** A wiggly polynomial may nail the training data but fail miserably on new data. The business decision: propose the linear model with its cost and performance. The customer may accept 80% because the budget doesn't allow a more expensive model. If they want better accuracy, *then* invest in the polynomial.

If your data has a **timestamp** — a date, a year, a quarter, a minute — do not treat it as ordinary regression. The way you interrogate the data changes fundamentally. - **Regression:** assumes a direct relationship $Y=f(X)$. You ask: "what X predicts Y?" - **Time series:** the value at time $t$ depends on past values. You ask: "what happened before that predicts what happens next?" In regression, you randomly split into train/test. In time series, you must respect chronological order — earlier data trains, later data tests. Shuffling destroys the temporal signal.

| Aspect | Prediction (Regression) | Forecasting (Time Series) | |---|---|---| | Data structure | $(X_i,Y_i)$ pairs — no time component | Sequence indexed by time $Y_1,Y_2,\dots ,Y_t$ | | What you use | Relationship $Y=f(X)$ | Past values of $Y$ itself | | Train/test split | Random | Chronological (earlier → train, later → test) | | Key question | "What X explains Y?" | "What does the past say about the future?" | | Example | Predict salary from education and experience | Predict next month's sales from past 36 months of sales | In regression, you predict. In time series, you *forecast*. The terminology signals the different approach.

The same principle applies in Natural Language Processing (NLP). When translating "I had a good coffee in the morning," you cannot shuffle the words randomly. The sequence carries meaning — "morning the in coffee good a had I" is nonsense. Sequence models — RNNs (Recurrent Neural Networks), LSTMs, Transformers — exist precisely to respect input order. They process tokens one after another, maintaining a memory of what came before. Time series models respect temporal order in the same way. The core insight is identical: when order matters, your model must respect it.

Every time series can be decomposed into four components: **1. Trend ($T_t$):** The long-term direction. Over many years, is the series generally going up, down, or staying flat? - Identify by looking at the overall slope from start to end, ignoring short-term wiggles. - Examples: increasing global average temperature (upward trend), declining landline phone subscriptions (downward trend), stable annual rainfall (flat trend). - Trend can be linear (constant slope) or nonlinear (accelerating or decelerating). **2. Seasonality ($S_t$):** Regular, predictable patterns that repeat at a fixed, known interval — typically within a year. - Period is fixed and known: 12 months for annual seasonality, 4 quarters, 7 days for weekly patterns, 24 hours for daily patterns. - Examples: AC sales drop every winter and rise every summer (annual seasonality). Rush-hour traffic peaks at 8 AM and 6 PM (daily seasonality). Retail sales spike in December (holiday seasonality). - Seasonality is the most predictable component — it will repeat next year because it did last year. **3. Cyclicality ($C_t$):** Longer-term wave-like patterns *without* a fixed period. - Unlike seasonality, cycles don't have a known length. Business cycles (boom and recession) may last 5–10 years but aren't clockwork. - Examples: economic recessions and expansions, technology adoption cycles (hype → trough → plateau), real estate market cycles. - Harder to predict than seasonality because the timing is irregular. **4. Irregularity ($I_t$):** Random, unpredictable spikes or dips. Also called noise, residuals, or the error component. - One-off events: a pandemic, a natural disaster, a sudden regulatory change, a product recall. - Example: a company's variable pay trends upward most years, then suddenly drops to zero during a company-wide freeze — that single anomalous dip is irregularity. - By definition, irregularity cannot be predicted. You can only acknowledge it and model around it.

The four components combine in one of two ways: **Additive model** (components are added): $$Y_t=T_t+S_t+C_t+I_t$$ Used when the magnitude of seasonal fluctuations doesn't change with the level of the series. Each component is in the original units of $Y$. **Multiplicative model** (components are multiplied): $$Y_t=T_t\times S_t\times C_t\times I_t$$ Used when seasonal fluctuations grow or shrink with the trend. For example, as a company grows, its December sales spike gets larger in absolute terms even though it's still roughly the same percentage increase. In practice, the multiplicative model is often converted to additive by taking logarithms: $\log Y_t=\log T_t+\log S_t+\log C_t+\log I_t$.