An entity is a real-world object — a person, a car, a transaction, a flower. A collection of entities is called an entity set. Each characteristic or property of an entity is an attribute.

Synonyms you must know (they all mean the same thing):

• For a data object: record, point, case, sample, entity, instance, vector, observation, pattern, event
• For an attribute: feature, dimension, characteristic, variable, field

One row = one data object. One column = one attribute. A collection of rows = a dataset.

Nominal and ordinal are collectively called categorical (or qualitative). Interval and ratio are collectively called numerical (or quantitative).

A nominal attribute has values that are symbols or names. "Nominal" comes from the Latin nomen (name). Each value is a category, a code, or a state. Nominal attributes are also called categorical attributes.

Examples: hair color (black, brown, blonde, red, gray), marital status (single, married, divorced, widowed), occupation, ZIP code, ID numbers, jersey numbers.

An ordinal attribute has values with a meaningful order or ranking, but the magnitude between successive values is unknown.

Examples include drink size (small, medium, large), shirt size (XS, S, M, L, XL), and grades (A, A−, B). Also satisfaction surveys (0 = very dissatisfied to 4 = very satisfied), professional ranks, and military ranks.

An interval-scaled attribute is measured on a scale of equal-size units. Values have order, can be positive/zero/negative, and allow you to quantify differences. But there is no true zero — you cannot say one value is a multiple of another.

Examples: calendar year, temperature in °C or °F, marks, CGPA, credit score.

A ratio-scaled attribute has an inherent zero point. Zero means the absence of that quantity. You can say one value is a multiple of another.

Examples include length (20 cm is twice 10 cm), weight (80 kg is double 40 kg), and number of children (4 is double 2, zero means no children). Also temperature in Kelvin (40 K is double 20 K), exam percentage (50% is double 25%), and age.

A discrete attribute has a finite or countably infinite set of values. A continuous attribute can take any real-number value between its minimum and maximum.

• Discrete: hair color, ZIP codes, ID numbers, number of children
• Continuous: temperature, height, weight, percentage (74.5% is possible)

A binary attribute is a nominal attribute with exactly two categories (0 or 1).

• Binary symmetric: Both states carry equal weight. Either can be coded as 0 or 1. Example: gender (male/female).
• Binary asymmetric: One state is more important (usually the rarer one), coded as 1. Example: HIV test result — positive = 1 (rare, important), negative = 0.

• Temporal resolution: data collected hourly vs. yearly
• Spatial resolution: a map showing individual buildings vs. entire regions
• Value resolution (measurement precision): 100 vs. 100.00 — more decimal places = finer gradation

Higher resolution enables discovery of more patterns, but requires more capable systems.

• Relational data: Standard structured format in relational databases. Each row is a record; each column is an attribute.
• Transactional data: Each record is a set of items (a "market basket"). Used for association analysis and market basket analysis — finding which items are frequently bought together. Example: a retail bill — customer X bought items A, B, C for price Y on date Z.
• Document data: Each document is a vector of term frequencies. Used for TF-IDF calculation and cosine similarity. Example: Document 1 = {trend: 5, data: 10, story: 4, mining: 8}.
• Data matrix: All attributes are numeric — the dataset is an $m\times n$ matrix suitable for linear algebra operations.
• Sparse data matrix: A data matrix where most entries are zero (document-term matrices, transaction data).

Data represented as nodes (entities) and edges (relationships). Examples: social networks (Jack and Mary are friends → an edge connects them), the World Wide Web (pages linked by hyperlinks), chemical compounds (atoms connected by bonds). Graph databases like Neo4j store this data.

• Sequential/Temporal data: Records have timestamps. Example: retail transactions with purchase times — enables patterns like "candy sales peak before Halloween."
• Time series data: A set of data points indexed in time order at regular intervals. Example: stock market charts, monthly profit percentages.
• Sequence data: Order matters, but no timestamps. Example: genomic data (DNA sequences — A, T, G, C), text (word order in a sentence).
• Spatial data: Data with geographic components — latitude, longitude, state, country.
• Clickstream data: The order of user clicks through a website. Example path: Home → Product → Checkout → Thank You.

Good data must be: correct, usable, complete, trustable, and consistent. The textbook (Tan, Steinbach & Kumar) identifies four major data quality issues: (1) measurement and data collection errors, (2) noise and artifacts, (3) outliers, and (4) missing, inconsistent, and duplicate data.

The textbook defines noise as "the random component of a measurement error" — it may involve distortion of a value or addition of spurious objects. Signal processing techniques can sometimes reduce noise, but the best approach is robust algorithms that tolerate it.

An outlier is data that does not fit with the rest — but it is NOT necessarily an error. Example: analyzing income of 1,000 people — one is Bill Gates. His income is much higher, but he is a valid data point from the same population. He is an outlier, not noise.

Duplicate data represents loss of information, not additional information. If two records exist for the same entity with conflicting values, you lose the true information. Either fix with the domain expert or delete both records.

The textbook notes that deduplication is often necessary when the same real-world entity appears multiple times under slightly different names (e.g., "J. Smith" and "John Smith").

Examples: birthday format DD-MM-YY in one record, MON-DD-YYYY in another; age = 42 but birthday = 3/7/2010; ratings as 1/2/3 in some records and A/B/C in others. Fix by checking with the domain expert for the correct format.

• Wrong/fake data: Intentionally incorrect (e.g., everyone's birthday = January 1). No automated method detects this without domain knowledge.
• Biased data: Purposefully unrepresentative. Example: survey data only from people who responded — sampling bias.

Data preprocessing involves two major phases:

1. Data Engineering: Converting raw data into prepared (clean) data — aggregation, cleansing, instance selection, partitioning.
2. Feature Engineering: Tuning prepared data to create features expected by the ML model — may involve combining features to create new ones.

The workflow: Raw data → Data Engineering → Prepared data → Feature Engineering → ML-ready features.

1. Smaller datasets → less memory, faster processing
2. High-level view instead of low-level — change of scope/scale
3. Aggregated quantities (averages, totals) have less variability than individual objects — more stable behavior

For values that are errors (e.g., salary = −10):

1. Delete the record (last resort — data is precious)
2. Replace with the measure of central tendency (mean/median/mode)

Given a numerical column with 12 sorted values.

From the worked example:

• Sorted data (positions 1–12): 6th = 12, 7th = 13
• $Q_2=\frac{12+13}{2}=12.5$

• $Q_1=\frac{11+11}{2}=11$

• $Q_3=\frac{14+15}{2}=14.5$

$$\text{IQR}=Q_3-Q_1=14.5-11=3.5$$

$$\text{Lower bound}=Q_1-1.5\times \text{IQR}=11-1.5\times 3.5=11-5.25=5.75$$ $$\text{Upper bound}=Q_3+1.5\times \text{IQR}=14.5+1.5\times 3.5=14.5+5.25=19.75$$

The training data must be a representative sample of the population — it should have approximately the same properties (mean, standard deviation, distribution shape) as the original data. A non-representative sample produces incorrect results regardless of sample size.

• Sampling with replacement: After selecting an item, put it back. Every item always has the same probability of being selected. For 7 items: always $1/7$.
• Sampling without replacement: After selecting, keep it out. Probabilities change: first draw $1/7$, second $1/6$, third $1/5$.

• Without replacement — when each sample must be unique (e.g., selecting 100 students from 1,000 for a project).
• With replacement — when repetition is allowed (e.g., bootstrapping to estimate model uncertainty, or when the dataset is too small and you need many different training subsets).

• Simplest version: Equal number from each group. 4 groups, need 40 people → 10 from each.
• Proper version: Proportional to group percentage. Population: 48.7% male, 51.3% female. To select 1,000: 487 males, 513 females.

Stratified sampling is the most commonly used method because it guarantees representation of every subgroup.

An imbalanced dataset has one class heavily underrepresented. Example: HIV test results — 95 negative, 5 positive out of 100. The rare class (positive) is what you want to analyze, but it's severely underrepresented. Many classifiers fail on imbalanced data.

1. Undersample the majority class — reduce majority-class records.
2. Oversample the minority class — create synthetic data for the rare class.

Use when you know the upper and lower bounds of the data.

Formula — to normalize a value $v$ to a new range $[\text{new\_min},\text{new\_max}]$:

$$v^{\prime }=\frac{v-\text{min}}{\text{max}-\text{min}}\times (\text{new\_max}-\text{new\_min})+\text{new\_min}$$

The fraction $\frac{v-\text{min}}{\text{max}-\text{min}}$ maps $v$ to $[0,1]$. Then multiply by the desired range width and add the new minimum to shift.

Use when you do NOT know the min and max, but you know (or can compute) the mean $\mu$ and standard deviation $\sigma$. The result has mean = 0 and standard deviation = 1.

$$v^{\prime }=\frac{v-\mu }{\sigma }$$

The textbook notes that the mean and standard deviation are strongly affected by outliers. A robust alternative replaces the mean with the median and the standard deviation with the absolute standard deviation: ${\sigma }_A={\sum }_{i=1}^m|x_i-\mu |$ where $\mu$ is either the mean or median.

The simplest method. Divide by a power of 10 to shift the decimal point:

$$v^{\prime }=\frac{v}{10^k}$$

Where $k$ is chosen so that the largest absolute value becomes less than 1. Example: values in thousands (3000, 6000) → use $k=4$: $3000/10^4=0.3$, $6000/10^4=0.6$.

The Iris dataset (Fisher, 1936) is the most famous dataset in machine learning. Contents:

• 3 species: Iris setosa, Iris versicolor, Iris virginica
• 50 records per species (150 total)
• 4 features: sepal length, sepal width, petal length, petal width (all in cm)
• 1 target label: species
• 3 duplicate records exist

• `iris.shape` → returns (rows, columns)
• `iris.head()` → first 5 rows
• `iris.info()` → attribute types (float64 for numerical, object for categorical)
• `iris.describe()` → the five-point summary plus mean and standard deviation for all numerical columns

For any numerical column, the five-point summary is (in order): min, Q1, Q2 (median), Q3, max. Along with mean and standard deviation, this is what `describe()` returns. The five-point summary must be memorized and recallable instantly for the exam.

Check duplicates with `duplicated()`, drop with `drop_duplicates()`. In Iris, duplicates are exact (no inconsistency), so dropping the second occurrence is straightforward.

Using Seaborn's `boxplot` for each attribute:

• Sepal length shows outliers (black dots outside whiskers)
• Whiskers represent the 1.5 × IQR bounds
• Decision to keep or remove outliers depends on domain expertise

Source: UCI Machine Learning Repository. Columns: symboling, normalized-losses, make, fuel-type, number-of-doors, body-style, drive-wheels, engine-location, wheel-base, length, width, height, curb-weight, engine-type, num-of-cylinders, engine-size, fuel-system, bore, stroke, compression-ratio, horsepower, peak-rpm, city-mpg, highway-mpg, price (target).

1. Add headers — Raw data had no column headers; provided in a separate file.
2. Identify missing values — Many "?" characters throughout.
3. Replace "?" with NaN — Convert all "?" to proper NaN so Python recognizes them as missing.
4. Impute numerical columns with mean: `normalized-losses`, `bore`, `stroke`, `horsepower`, `peak-rpm`.
5. Impute categorical column with mode: `number-of-doors` → majority of cars have 4 doors.
6. Handle missing target label: `price` had missing values → drop those entire records.
7. Data type conversions — Convert columns to appropriate types.
8. Create derived attribute — Convert `city-mpg` (miles per gallon) to liters per 100 km.