EDA & Visualization

Learning objectives

By the end you can:

1. Explain what exploratory data analysis is and why it must happen (numerically, not just visually) before any model is fit or any chart is drawn, and name the three systematic checks (distribution, outliers, correlation) that make up a disciplined first look at a dataset.
2. Compute a full five-number summary plus mean, sample standard deviation, and a moment-based skewness in code (NumPy), stating precisely which convention each statistic uses (sample vs. population denominator, linear-interpolation quantiles) and reading what a nonzero skew implies about a distribution's shape.
3. Choose a histogram's bin count by a principled rule (Freedman–Diaconis, with a Sturges fallback) rather than an arbitrary default, and explain (with a concrete before/after) how the choice of bins changes the story a histogram appears to tell.
4. Detect outliers with Tukey's IQR fences, implement the mask in code, and reason about when a flagged point is a genuine extreme worth keeping versus a data error worth investigating, never an automatic delete.
5. Compute a Pearson correlation matrix in code as a descriptive summary of co-movement, build the heatmap it feeds, and state precisely why "the correlation is 0.42" is not the same claim as "the correlation is statistically significant" (that test is QM1.02's, not this module's).
6. State and apply the dataviz grammar this module needs in code: map a comparison type (ranking, trend over time, distribution, relationship between two variables) to its correct chart type, and implement that mapping with matplotlib's object-oriented API (fig, ax = plt.subplots()), not the bare pyplot state machine.
7. Identify and eliminate the specific chartjunk failures this module grades against: an unsorted ranking bar chart, a dual (twinx) y-axis comparing unlike scales, a missing or unit-free axis label, and a generic, non-committal title, and implement the fix (sort, rebase to a common index, label, title the comparison) in code.
8. Build a rolling mean and volatility band to smooth a noisy time series, and explain why plotting raw daily noise over a multi-year window is usually the wrong exhibit.
9. Build the numeric machinery behind a box-plot comparison (matplotlib's own five-number-plus-whiskers convention) so that comparing several groups' distributions in one exhibit is a computed fact, not a matplotlib default you trusted blindly.
10. (Productivity objective: R10 duality.) Use an AI coding copilot to accelerate writing pandas/matplotlib EDA boilerplate once you have already decided, from this module's grammar, which chart the comparison calls for, and verify its output by running this module's own hidden tests and confirming every matplotlib call it used actually exists.

The duality, stated once (R10). Objectives 1–9 are the understanding objective the mastery gate rewards: you pass by demonstrating you can compute the numbers and choose the chart yourself, closed-book. Objective 10 is the productivity payoff you keep afterward, once you already know which chart a comparison calls for, a copilot legitimately speeds up the ax.set_* boilerplate. A tool can draft the code; it cannot decide what the chart should say, and that decision is the entire content of this module.

Prerequisites & connections

Builds on. This node assumes real fluency with the three modules immediately before it in the quant-ds branch. DA1.01 (NumPy & Vectorized Computation) is where the array operations this module leans on constantly: np.mean, np.std with an explicit ddof, np.quantile, vectorized arithmetic over a whole column at once, stopped being new; this module does not re-teach vectorization, it applies it to real EDA questions. DA1.02 (pandas I. Data Wrangling) is where loading a CSV, filtering rows, and grouping by a key became routine; every exercise and worked example below loads one of the program's bundled datasets and immediately filters it (df[df.ticker == '...'], df[df.fiscal_year == ...]) exactly the way DA1.02 taught. DA1.03 (pandas II. Cleaning, Time Series & the Excel↔Python Bridge) is the closest and most load-bearing neighbor: this module assumes the data arriving in each exercise is already clean (no stray nulls, correctly typed dates): DA1.03 is where that cleaning discipline was built, and this module's rolling-window work (DA1.04-e09) is the direct continuation of DA1.03's time-series indexing. If any of the three feels shaky, the shakiness will surface here as a wrong sample-vs-population ddof, a groupby that returns the wrong shape, or a rolling window that silently includes a row it shouldn't: go back, not around.