Market & Fundamental Data Pipelines + Vendor Landscape

Branch: Quant Developer (quant-dev) · Finance galaxy, Quant-Code region · Node QD1.01 · ~18 focused hours · Gate: quiz ≥ 85% + code-pack ≥ 85% (hidden tests green on at least 85% of exercises)

Every quant strategy you will ever build in this branch (a signal, a backtest, a risk model, the capstone itself) is a function of a table of numbers you did not personally collect. Nobody in this program is going to hand you a clean, timeless, universally-agreed price history. What you get, in the real world and in this module's bundled datasets alike, is a vendor's raw feed: a close price per ticker per day, reported the way the exchange printed it that day, with every corporate event still baked invisibly into the number, every company that ever failed already scrubbed from today's convenient "current constituents" list, and every fundamental figure dated by the moment someone chose to file it rather than the moment it became true. If you feed that raw table straight into a return calculation, a factor score, or a backtest, you will get an answer (code rarely refuses to run) and the answer will be wrong in ways that are almost always invisible until it is too late, because a bad data pipeline doesn't crash, it just quietly lies with total confidence. This module is where you learn to notice, and then to fix, the three specific ways that lie shows up: a stock split or dividend that mechanically moves the price without moving one rupee or dollar of value, a fundamentals table that lets you "know" a number before it was ever filed, and a universe of "current" companies that has quietly forgotten every one that went bankrupt along the way. Fix those three things correctly, every time, and you have a pipeline. Miss any one of them, and every downstream number (however sophisticated the model that produced it) is decorated nonsense.

Here is the whole module compressed into one sentence you will spend eighteen hours earning the right to believe: a data pipeline's job is not to fetch numbers, it is to make sure that every number downstream means exactly what your model thinks it means, on exactly the date your model thinks it knew it. Everything below (split and dividend adjustment, point-in-time discipline, survivorship control) is one instance or another of that single idea. A raw close after a 2-for-1 split does not mean "the stock fell 50%"; it means "the company issued one more share for every share outstanding," and your pipeline's job is to make the two facts stop looking identical to a return calculation. A revenue figure filed six weeks after the fiscal year ended does not mean "the market knew this on day one of the new year"; it means the market was blind to it until the filing date, and your pipeline's job is to make sure no backtest is ever allowed to peek past that date. A company that got delisted in year six of a ten-year study does not mean "irrelevant, since it isn't around to ask about today"; it means its investors lived through a real, often catastrophic, real-money outcome, and your pipeline's job is to make sure that outcome is still in the average when you compute one. None of this is exotic quant theory, it is data engineering discipline, and it is the discipline every number in the rest of this branch, and every number a real desk trades on, silently depends on.

You are not the first person to have this problem, which is exactly why this module also maps the terrain: who sells clean, adjusted, point-in-time, survivorship-controlled data (for a price), who publishes something usable for free, and which of those choices is right for India, for the US, and for a strategy that trades both. Vendor selection is itself a skill, a wrong or misunderstood choice here poisons everything you build in QD1.02 through QD2.05, and this module's second half is a deliberately concrete, dated, and honestly-hedged map of that landscape, so that "which data source do I actually use" stops being a guess.

Learning objectives

By the end you can:

1. Explain, precisely, why a raw vendor price feed cannot be fed directly into a return calculation across a stock split or a cash dividend, and state the single sentence that separates a mechanical price change from an economic one: nothing happened to the value of the company; something happened to the accounting of how that value is sliced.
2. Compute, by hand and in code, a split-adjusted price series from a raw close series and a split ratio, dividing every pre-split price by the ratio so that a return computed across the split boundary reads correctly as flat, not as a phantom crash.
3. Compute, by hand and in code, a dividend-adjusted (total-return) price series from a raw close series and a cash dividend amount, using the standard 1 − dividend / previous_close factor, and explain precisely why ignoring dividends silently understates every long-horizon return you will ever report.
4. Generalize both of the above into a single back-adjustment algorithm that composes an arbitrary sequence of splits and dividends, computing each event's own adjustment factor from raw (not already-adjusted) prices and applying the cumulative product working backward from the most recent date, and explain why that "compute locally, apply cumulatively" order matters.
5. Build a data-quality scanner that flags suspiciously large single-day price moves as candidate signs of an un-adjusted corporate action, and correctly state the technique's real limits (false positives on genuine large moves, false negatives on small-ratio splits).
6. Define point-in-time (PIT) data precisely, the distinction between when a fact became true (a fiscal period ended) and when it became knowable (it was filed/published), and build a point-in-time filter and an "as-of" lookup over a fundamentals table, using nothing more exotic than string comparison on YYYY-MM-DD dates.
7. Define survivorship bias precisely, demonstrate numerically the gap between a survivors-only average return and the true full-universe average that includes every company that was ever in the universe (including the ones that failed), and explain why that gap is not a rounding error but routinely tens of percentage points.
8. Synthesize adjustment, point-in-time discipline, and survivorship control into one function that builds a point-in-time, survivorship-controlled, split/dividend-adjusted panel as of any query date, the actual deliverable a real quant data pipeline exists to produce.
9. Name and correctly classify the recurring mess in real vendor feeds beyond corporate actions and PIT/survivorship, missing trading days, ticker renames/reuses, currency and unit-scale traps (₹ crore vs ₹, cents vs dollars), and exchange-calendar/timezone mismatches between NSE and NYSE, and state the standard fix for each.
10. Map the India and US market-data landscape: name the major paid institutional vendors, the free/official sources, and the survivorship-bias-free specialist providers, and correctly match a data need (real-time vs. historical, fundamentals vs. prices, retail-budget vs. institutional-budget) to an appropriate source in each market.
11. (Productivity objective: R10 duality.) Use an AI coding assistant to accelerate exactly the parts of a pipeline you already understand by hand, drafting a first-pass adjustment function, auditing code for a PIT or survivorship leak you specify, while treating every AI-drafted number as unverified until it reproduces your own by-hand calculation and passes this module's hidden tests.

The duality, stated once (R10). Objectives 1–10 are the understanding objective: the gate (quiz ≥ 85%, code-pack ≥ 85% with hidden tests green) rewards being able to build, by hand and in code, a pipeline that gets adjustment, point-in-time, and survivorship all correct, on data you have never seen, since the hidden tests exist precisely to catch a solution that only works on the visible example. Objective 11 is the productivity objective: knowing how a working quant developer actually uses an AI copilot on this kind of code, and where that copilot cannot be trusted. A copilot can draft a plausible-looking back_adjust function in five seconds. It cannot tell you, on its own authority, whether that function is correct, only your own hand-traced example and this module's hidden tests can do that, and a subtly wrong adjustment function is exactly the kind of bug that looks fine on every visible case and silently corrupts every number downstream of it.

Prerequisites & connections

Builds on. Nothing hard-gated. QD1.01 is the first node of the quant-dev branch and, per the program's day-one entry rule, it unlocks from the very start, no other quant-code module, and no finance-phase progress, is required to begin. Everything you need is built from scratch in this module: plain Python lists and dictionaries, string comparison on dates, and arithmetic you can check on a calculator. That said, this module is written assuming you can already program at the level CS1.01–CS1.02 teach (functions, lists, dicts, loops), if you have never written a Python function before, do CS1.01 first; it is a soft, non-blocking recommendation (R11), not a gate, but the code pack below will feel considerably smoother with that foundation already in hand. If you have already done DA1.01/DA1.02 (NumPy and pandas), you will recognize every idea here as the hand-built version of something pandas gives you a one-line method for (pd.merge_asof for point-in-time joins, in particular), this module deliberately builds the mechanism in plain Python first, because a pandas one-liner that you don't understand is exactly the kind of code that hides a silent bug, and the whole point of this module is to stop silent bugs from hiding.

Feeds forward. Everything else in the quant-dev branch depends on this module's output existing and being trustworthy. QD1.02 (Signal & Factor Construction and Evaluation) takes the point-in-time, survivorship-controlled, adjusted panel this module teaches you to build and computes value/momentum/quality/low-volatility scores on top of it, a factor computed on an un-adjusted or survivorship-biased panel is not a factor, it is noise with a confident-looking Sharpe ratio. QD1.03 (Building a Backtesting Engine) is where the discipline this module teaches by hand, "never let information from after your as-of date leak into a computation", becomes an automated guard: its no_lookahead harness exists to catch, mechanically, exactly the class of bug Worked Examples 4 and 6 below walk through by hand. QD1.04 (Portfolio Optimization), QD2.01–QD2.04, and the QD2.05 capstone all inherit this module's panel as their starting input; the capstone's own grading rubric explicitly credits a point-in-time, survivorship-controlled data pipeline as a scored dimension, not a footnote. Get this module right and every later module in the branch gets easier; get it wrong and every later module inherits a bug you will spend far longer finding downstream than you would have spent fixing it here.

The binding C-3 boundary, read this before you read the "look-ahead bias" material in QM1.03. The Finance-Plus QM branch (specifically QM1.03, Time-Series, Monte Carlo & Intro ML) already teaches the statistical failure mode of using future information in a model: shuffle a time-ordered dataset before a train/test split, and some "training" observations land after some "test" observations in real time, the look-ahead bias that breaks an honest out-of-sample evaluation, fixed with a chronological or walk-forward split. That is a concept about model evaluation, and this module does not re-derive it (R1), if the phrase "look-ahead bias" or "chronological split" is unfamiliar, QM1.03 §9 is the concept-level treatment, and this module assumes you can cross-reference it. What THIS module owns, and what QM1.03 does not teach, is the data-engineering discipline one layer underneath: how do you build the dataset itself so that a look-ahead bug can't even get a foothold before a model ever sees it? Point-in-time filtering and survivorship control are that discipline, they are about the panel, not the split. QD1.03's no_lookahead harness is the automated guard that later catches a look-ahead violation inside a backtest; this module is what makes the data feeding that backtest honest in the first place. Three different layers, three different modules, no overlap: QM1.03 teaches you to recognize the disease conceptually; this module teaches you to build data that doesn't carry it; QD1.03 teaches you to build a machine that automatically checks for it.

A second explicit cross-link, read CS2.05 first if you haven't. CS2.05 (APIs, HTTP, JSON & Web Scraping) already built the resilient-fetcher pattern, retries, exponential backoff, rate-limit respect, defensive JSON extraction, robots.txt/Terms-of-Service ethics, for talking to a real vendor API. This module does not re-teach any of that (R1); CS2.05's own Prerequisites section already names this module as where its skills "meet their natural home," and the hand-off is intentional and reciprocal. What this module assumes you already have, once you actually go build a production fetcher against a real vendor, is CS2.05's fetcher; what THIS module teaches is what you do to the data after it lands, adjustment, PIT discipline, survivorship control, regardless of which vendor or which fetching mechanics got it to you. If your mental model of "the pipeline" is "the part that calls the API," widen it: that is CS2.05's job. This module is everything that happens next.

Other cross-links, briefly. CS2.03 (Databases & SQL) is where you would actually store a growing point-in-time history properly, the standard production pattern is "never overwrite a row, only ever append a new vintage with its own as-of stamp," which is a database design problem CS2.03 already equips you to reason about; this module builds the logic that such a store needs to serve correctly, not the storage engine itself. CS2.04 (Systems & Data-Intensive Thinking) covers columnar formats (Parquet) and the storage/compute trade-offs a real market-data warehouse lives inside; this module is agnostic to storage format entirely, everything here is expressed over plain Python lists and dicts precisely so the logic is visible without a storage-engine's furniture in the way. QM1.01/QM1.02 (probability, regression) are not required and not referenced; nothing here is inferential statistics.

Core teaching content

1. What a data pipeline actually is, and why market data is uniquely hostile to naive code

A data pipeline, in the sense this module means it, is the sequence of steps between "a vendor's raw feed" and "a table your model can trust": ingest (get the raw bytes. CS2.05's job), validate (catch garbage before it propagates, §5 below), adjust (correct for corporate actions, §2–4), index for point-in-time correctness (never let a fact appear before it was knowable, §6), control for survivorship (never let a company silently vanish from history just because it isn't around today, §7), and serve (answer "what did the world look like, honestly, as of this date": §8). Most introductions to programming, including this branch's own CS1.01–CS1.02, teach you to write a function that is correct given clean input. Market data is unusual among the datasets you will meet in this program precisely because the input is never clean in the ways that matter here, and the ways it is dirty do not look like dirt, a raw closing price of ₹610 the day after a 2-for-1 split is not malformed, missing, or out of range; it is a completely well-formed number that means something different from what a naive reader assumes. This is the module's central warning, worth holding before a single line of code: the bugs in this module do not crash your program. They produce a plausible, well-typed, silently wrong number, and every technique below exists to convert an invisible error into a visible, fixable one.

5. A pipeline's data-quality net: catching what the vendor's feed didn't flag

In a perfect world, every vendor feed ships a clean, separate corporate-actions table alongside the prices, and your back_adjust function simply consumes it. In the real world, that table is sometimes late, sometimes wrong, and sometimes (for a small-cap name from a scrappier feed) simply absent. A production pipeline needs a second line of defense: a heuristic scanner that flags candidate un-adjusted events for a human (or a downstream reconciliation step) to check. The idea is blunt but genuinely useful: any single-day percentage move whose absolute value exceeds a threshold (a common default is 15%) is worth a second look, because a real, ordinary trading day essentially never moves a liquid large-cap that much, while an un-adjusted 2-for-1 split moves it by exactly 50% and a 3-for-1 split by exactly 66.7%. Run this scanner on the very same NILAM and BRPEAK series from §2: flag_unadjusted_jumps([1200.00, 1180.00, 1220.00, 610.00, 605.00, 615.00], 0.15) returns [3] (precisely the split day) and the BRPEAK series flags the same single index for the same reason. This is Exercise 4.

Be honest about the technique's limits, because an over-confident heuristic is its own hazard: it will produce false positives on a name that genuinely gapped 20% on an earnings surprise or a merger announcement (real moves, not adjustment bugs, a human still has to look), and it will produce false negatives on a small-ratio split (a 3-for-2 split moves the price by only 33%, and a 5% stock dividend by only 5%, both can hide under a 15% threshold entirely). The scanner is a net, not a verdict: it narrows a full history down to a short list worth checking against the vendor's actual corporate-actions table (or a second, independent vendor), which is exactly how a real desk's data-quality process works, automate the narrowing, keep a human or a cross-check in the loop for the judgment.

6. Point-in-time discipline: the difference between true and knowable

Move now from prices to fundamentals, and to the single idea that most separates a research-grade dataset from a naive one. Every fundamentals record carries (at minimum) two dates that are easy to conflate and dangerous to conflate: the fiscal period it describes (when the underlying business activity actually happened) and the filing date (when the figure was actually published and became knowable to anyone outside the company). This program's own bundled sec-fundamentals and india-fundamentals datasets carry exactly this shape: ticker, fiscal_year, revenue, ..., filed_date, precisely so this distinction is not hypothetical. In the real world, a company's fiscal year might end March 31st, but its results are rarely filed the next morning: Indian listed companies typically file within roughly 45–60 days of period-end under SEBI's listing regulations, and US companies file a 10-K roughly 60–90 days (accelerated filers faster) after fiscal year-end, a 10-Q roughly 40–45 days after quarter-end. Point-in-time (PIT) data means: a figure is only usable in a computation dated on or after the day it was actually filed, never before, no matter how "obviously true" it seems in hindsight that the fiscal year had already ended.

Here is the exact numeric trap this creates, using NILAM's (illustrative, synthetic) fundamentals history: FY2022 revenue ₹3,600, filed 2022-05-18; FY2023 revenue ₹4,200, filed 2023-05-20; FY2024 revenue ₹4,800, filed 2024-05-19. Suppose your model needs "the latest known revenue as of 2023-08-01." The correct, point-in-time-safe answer restricts to records with filed_date <= as_of first, which, as of 2023-08-01, includes only FY2022 and FY2023 (FY2024's filing date of 2024-05-19 is nearly a year in the future relative to this query), then takes the most recent of those, landing on FY2023's ₹4,200. The tempting, wrong shortcut, max(records, key=lambda r: r["fiscal_year"]), with no point-in-time filter at all, simply grabs whichever fiscal year is largest across the entire table, landing on FY2024's ₹4,800, a figure that would not exist in the world for another nine months. A backtest that used ₹4,800 "as of" August 2023 has, silently, let the future leak into the past, and the two numbers are not close: ₹4,800 vs. ₹4,200 is a 14.3% overstatement, entirely from one missing date check. This exact pair, a correct filter_point_in_time/latest_as_of and the tempting max(..., key=fiscal_year) shortcut, is Exercises 5 and 6, and the gap between the two numbers is precisely "what breaks numerically" when point-in-time discipline is skipped.

Because every date here is a zero-padded ISO string ("YYYY-MM-DD"), ordinary string comparison (<=) already sorts chronologically, you do not need a date-parsing library for this module's exercises, though a real production system would typically use one (datetime.date, or pandas.Timestamp) for the calendar arithmetic (adding days, handling month-end) that comparison alone cannot do. Hold onto the vocabulary: a query that respects filing lag is a point-in-time or as-of query; a naive query that silently uses today's complete table as if it had always looked that way is sometimes called using as-reported-today or, more bluntly, a vintage-blind table, and the difference between the two is the entire reason pandas.merge_asof exists as a first-class tool in the pandas library specifically for point-in-time joins (a forward pointer for once you reach DA1.02).

{
  "skill": "Auditing a data-pipeline function for a specific, named class of bug (a point-in-time or survivorship leak) rather than a generic 'is this good code' review",
  "use": "Once you can already state precisely what a PIT or survivorship violation looks like in code (a missing filed_date check, a universe list with no delisted names in it), ask an AI assistant to review a function specifically for that failure mode — a narrow, checkable question gets a far more useful answer than 'review my code'.",
  "tools": ["AI coding assistants — Claude, ChatGPT, GitHub Copilot", "This module's own filter_point_in_time / survivorship_bias_gap functions, as ground truth to test the review's claims against"],
  "prompt": "Here is a function that looks up the latest known fundamental value for a ticker as of a given date: <PASTE YOUR CODE>. Specifically check: (1) does it filter by filed_date <= as_of BEFORE picking the latest fiscal_year, or does it pick the latest fiscal_year first? (2) is there any path where a record with a future filed_date could be returned? Explain any issue found in terms of what a backtest using this function would silently do wrong.",
  "verify": "Construct your own small example with a record whose filed_date is AFTER your as_of date (like NILAM's FY2024 above) and manually check the function's output against what the correct point-in-time answer should be. An AI review can miss the exact failure mode it wasn't specifically asked about, or claim a fix is correct without actually tracing it against a concrete date — your own worked example is the only real judge, and this module's hidden tests are the automated version of that same check.",
  "diy": "You must already be able to construct a concrete example that WOULD leak (a future filed_date, as done above) before you can judge whether an AI's review — or its proposed fix — actually catches it. Without that, you have no way to tell a correct review from a plausible-sounding wrong one.",
  "market": "US"
}

8. Synthesis: the point-in-time, survivorship-controlled, adjusted panel

Put all three disciplines together and you have the actual deliverable this module exists to teach you to build: given a raw universe (every candidate company's price history, its corporate-action events, and its listing window) and a query date, produce the panel that honestly reflects only what existed and only what was knowable as of that date, with prices correctly adjusted using only the corporate actions that had themselves already happened by that date.

That last clause deserves its own sentence, because it is a genuine, debatable nuance rather than a settled rule, and you should know both sides of the debate rather than accept one silently: should a "current" adjusted price use every split and dividend in a company's entire history (including ones after your query date), on the reasoning that a split ratio is a purely mechanical fact that carries no real information once it has happened, and using the fully-adjusted-to-today number is simply more convenient for cross-period comparison? Or should it use only the corporate actions known as of the query date, on the stricter reasoning that a genuinely point-in-time-honest panel should never reach past its own as-of date for any input, corporate actions included, so the same discipline applies uniformly everywhere rather than carving out a "this one input is exempt" special case? Many real vendors and practitioners take the first (convenience) view for splits and dividends specifically, precisely because (unlike a fundamentals restatement or a future price) a split ratio is public, mechanical, and non-informative. This module's own capstone exercise takes the second, stricter view deliberately, because a module whose entire thesis is "look-ahead creeps in through inputs you didn't think to check" should not itself quietly carve out an exception, and because the stricter rule generalizes cleanly to every other PIT problem you will meet in this branch, where "well, this one input doesn't really count as leaking the future" is exactly the rationalization that produces a real bug. Know that the debate exists; know which side this module lands on, and why; and know that a real production system's choice here should be a documented, deliberate decision, never a silent default.

With that settled, build_pit_panel(universe, since, as_of) (Exercise 8) works as follows. For every candidate company: it must have been listed for the entire window (entry on or before since, and (if it ever exited) not before as_of); it must have valid price data on both query dates; then its adjusted return from since to as_of is computed using back_adjust, restricted to only the events with an index at or before as_of's own position in its date series. A company failing either membership check is simply omitted from that query's output, not zeroed out, not flagged, just absent, exactly the way a name that hadn't IPO'd yet, or had already delisted, genuinely was absent from the investable universe on that date.

Trace why this single function actually needs everything this module has built. Take a small illustrative universe: AAA, listed since 2023, whose raw closes over four dates are [100.00, 102.00, 50.00, 51.00] with a 2-for-1 split taking effect at the third date; and BBB, which only listed on the fourth of those four dates. Query since="2024-01-01", as_of="2024-04-01" (the first and last of the four dates). BBB is immediately excluded, it wasn't listed yet at since. AAA's raw closes fell from 100 to 51, a naive, unadjusted return of 51/100 − 1 = −49%, which looks like a near-total loss. But once you back-adjust using the split known by as_of: [50.0, 51.0, 50.0, 51.0], the true return from the adjusted since-value (50.0) to the adjusted as_of-value (51.0) is 51/50 − 1 = +2%. The difference between −49% and +2% is not a rounding error; it is the entire economic reality of the stock's actual performance, recovered only because adjustment, membership, and the as-of query all had to work together correctly. Run the same construction on a richer India universe (NILAM with a split, KUSHAGRA delisted mid-window, SARASWATI listed only partway through) and a US companion (BRPEAK, PENWICK, CLRRDG in the same three roles), and you get panels that correctly include exactly the names that were genuinely investable across the full window and correctly exclude the rest, this module's code pack hidden tests check exactly this, on data you have not seen while writing your solution.

9. The rest of the mess: what else a real vendor feed gets wrong

Corporate actions, point-in-time discipline, and survivorship are the three failure modes this module drills to the point of numeric fluency, because they are the three that most silently corrupt a downstream model. A production pipeline also has to defend against a longer tail of smaller, more mundane traps, each worth being able to name on sight even though none of them gets its own graded exercise here:

• Missing trading days and mismatched calendars. NSE and NYSE do not observe the same holidays, and even within one exchange, a single ticker can have a genuinely missing row for a day it was halted. Joining an India series and a US series by raw calendar date without first reconciling each market's own trading calendar silently misaligns rows, a "same-day" comparison that is quietly comparing Tuesday in Mumbai to Monday in New York.
• Ticker renames and reuse. A company can change its ticker symbol (a rebrand, a listing migration) without changing its underlying identity, and (more dangerously) an exchange can, after enough time has passed, reassign an old, now-vacated ticker symbol to a completely unrelated new company. A pipeline that joins purely on ticker symbol, with no stable internal company identifier, will eventually stitch together two different companies' histories as if they were one continuous series. The standard fix is a stable internal ID (this program's own bundled finance-db.sqlite, built for CS2.03, uses exactly this pattern: an internal company_id that tickers map to, never the reverse).
• Currency and unit-scale traps. ₹ crore (10 million) vs. ₹ lakh (100 thousand) vs. plain ₹ is a uniquely Indian-market landmine with no US equivalent, and it sits alongside the equally real US trap of a per-share figure reported in cents rather than dollars by one API and dollars by another. A pipeline that silently assumes a fixed scale across sources will occasionally produce a number that is off by a clean factor of 100,000 or 100, plausible-looking on its own, catastrophic the moment it's compared against a correctly-scaled figure from elsewhere.
• Timezone and "as-of" ambiguity for a still-trading session. A close price for "today" from a live feed queried mid-session is not the close at all (it's whatever the last print happened to be) and a naive pipeline that doesn't distinguish "final settled close" from "latest available print" will occasionally treat an intraday snapshot as if it were the day's real, final number.

None of these get a dedicated code-pack exercise, they are named here so you recognize them the first time one bites, the same way this module's earlier sections turned "the number looks fine" into "the number needs a specific check" for splits, dividends, point-in-time, and survivorship.

Common mistakes & how experts think differently

1. "The price series looks fine, so the return calculation is fine." A raw price series with an un-adjusted split in it is perfectly well-formed (every number is a plausible, correctly-typed price) and it will pass any check that only looks at individual values in isolation. The expert's habit is to check the derivative (the day-over-day percentage change), not just the level, because that is where a mechanical event announces itself as an impossible-looking jump; §5's scanner exists precisely to automate this habit.
2. "Adjusted for splits is the same as adjusted." These are two different things with a real gap between them, and conflating them silently and permanently understates every long-horizon total return. The expert always asks, explicitly, "adjusted for what, exactly?" of any vendor field named adj_close or similar, because the same field name means different things at different vendors.
3. "If the fiscal year already ended, the number is fair game." This is the single most common point-in-time error, and it is seductive precisely because it feels like common sense, of course the number existed once the year was over. It didn't exist to the market until it was filed, and a backtest that uses it earlier than that has quietly handed its model information nobody actually had. The expert's reflex, drilled in §6, is to check filed_date, not fiscal_year, before ever treating a fundamental as "known."
4. "I only need today's constituent list, it's the current, correct one." A "current" list is correct for exactly one thing: telling you who is in the universe today. It is actively wrong for any historical question, because every company that failed along the way has been silently removed from it by the mere fact of no longer existing. The expert never builds a historical study on a present-tense list; they build (or buy) a historical membership record with entry and exit dates, per §7.
5. "The gap from survivorship bias is probably small, a few percent, tops." It routinely is not. This module's own two worked universes showed 82 and 81 percentage points of naive-average inflation from a universe of only five names each, and these are not cherry-picked extremes; a universe where a meaningful fraction of constituents fail over a multi-year window will show a comparably large gap almost by construction, because a −100% outcome pulls an average down far more than a +100% outcome pulls it up in absolute-percentage terms once you're averaging a mix of the two.
6. "I'll adjust for corporate actions using whatever's in the price file: vendors handle that." Some do, reliably; many, especially free or budget sources, do not, or handle it inconsistently across different security types. The expert treats "does this vendor's adjustment actually match what I'd compute independently on a name I can check by hand" as a real verification step before trusting a feed at scale, exactly the muscle Exercises 1–4 exist to build, so you have an independent way to check.
7. "A missing row is obviously a data error, I'll just forward-fill it." Sometimes it is an error; sometimes it is a genuine market holiday, a trading halt, or, most dangerously, the exact day a company delisted. Forward-filling blindly can silently manufacture a fake, flat "still trading" price for a company that had, in fact, already stopped trading, quietly reintroducing the survivorship problem you thought you'd fixed, in a different disguise. The expert checks why a row is missing before deciding how (or whether) to fill it.
8. "I built the pipeline once; it's correct forever." A pipeline that was correct against last year's universe can silently break the day a ticker gets reused, a vendor changes its adjustment convention without an announcement, or a new type of corporate action (a spin-off, a rights issue) shows up that your events schema never anticipated. The expert treats a data pipeline as a piece of software with an ongoing maintenance burden, worth the same test-and-monitor discipline CS2.06 teaches for any other production code, not a one-time script that, once it runs, is done.

The thread through all eight: the expert never trusts a number just because it ran without an error, every check above is a specific, nameable question ("adjusted for what? knowable when? survived by construction or by omission?") asked of data that looks, at a glance, entirely fine. That is precisely why the failure mode is dangerous: it never announces itself. You have to go looking, every time.

Worked examples

A note on the data: every price, dividend, and fundamentals figure below is explicitly synthetic (illustrative, internally consistent (R3) on fictional companies (NILAM, BRPEAK, CRSTNE, KUSHAGRA, SARASWATI, and their counterparts)) never a claim about any real company's actual reported results. Every number below was verified by actually running the corresponding Python function, not by hand-arithmetic alone, and matches this module's code pack exactly.

Worked example 1: A phantom 50% crash, and the one-line fix (India, ₹)

An analyst pulls NILAM's raw NSE-style closes for a short window: [1200.00, 1180.00, 1220.00, 610.00, 605.00, 615.00], and a 2-for-1 split occurred so that the fourth value (index 3) onward is on the new basis. Their return function reports a −50% day. Diagnose and fix.

Solution. Step 1, check whether the move is economically plausible before checking anything else. A single-day −50% move for an ordinary company is essentially never a real trading outcome, it is either a data error or a mechanical corporate action, and §5's 15% flag threshold would have caught this immediately ((610−1220)/1220 = −0.50, comfortably past the threshold). Step 2: name the mechanism. A 2-for-1 split doubles share count and halves price, with zero change in total shareholder value, 100 shares × ₹1,200 = ₹120,000 before, 200 shares × ₹600 = ₹120,000 after (this series' actual post-split level, ₹610, differs slightly from the theoretical exact-half ₹600 only because the market also moved a little on the day itself, the split accounts for the overwhelming majority of the jump, not all of it, which is itself worth noticing). Step 3: apply the fix. adjust_for_split([1200.00, 1180.00, 1220.00, 610.00, 605.00, 615.00], 3, 2.0) divides every pre-split value by 2: [600.00, 590.00, 610.00, 610.00, 605.00, 615.00]. Step 4, recompute the boundary return on the adjusted series. (610.00 − 610.00) / 610.00 = 0.00, correctly flat, because the adjusted series lands exactly at the post-split series's own starting level. Read it. The fix took one function call; the danger was entirely in not knowing to apply it. A model consuming the raw −50% figure would have treated a bookkeeping event as a catastrophic loss with 100% of the same downstream confidence it gives a real one.

Worked example 2: The total-return gap a dividend leaves behind (US, $)

CRSTNE pays a $2.25 dividend; raw closes around the ex-date are [153.00, 151.00, 150.00, 147.75, 148.50], ex-dividend at index 3. An analyst reports the price-only return over this window and is asked to also report the total return an investor actually experienced.

Solution. Step 1, the naive (price-only) return. (148.50 − 153.00) / 153.00 ≈ −2.94%, using the raw closes exactly as they printed. Step 2, the adjustment. Prior close before the ex-date is closes[2] = 150.00; factor = 1 − 2.25/150.00 = 0.985. adjust_for_dividend([153.00, 151.00, 150.00, 147.75, 148.50], 3, 2.25) gives [150.705, 148.735, 147.75, 147.75, 148.50], the pre-ex-date closes rescaled down by 1.5%, the ex-date close and after left untouched (they already reflect the payout). Step 3, the total return. (148.50 − 150.705) / 150.705 ≈ −1.46%, a materially smaller loss than the price-only −2.94% suggested, because the investor also received $2.25 in cash that the price-only figure never counted. Step 4: generalize the lesson. Over a single dividend the gap is a percentage point or two; over a decade of quarterly dividends compounding, the gap between a price-return index and a total-return index for the same stock is routinely dramatic, this is exactly why a serious long-horizon comparison (an index vs. a strategy, a stock vs. a benchmark) must specify, and match, which of the two it's using on both sides of the comparison. Read it. "The stock was flat over ten years" and "the stock returned zero to a holder" are two different claims, and only the second one is usually what anyone actually means to say.

Worked example 3: Composing two events correctly, traced fully (US, $)

BRPEAK has both a 3-for-1 split (index 4) and a $1.20 dividend (index 6) in an eight-day illustrative window: raw closes [300.00, 306.00, 294.00, 303.00, 99.00, 101.00, 99.80, 100.50].

Solution. Step 1, compute each event's own LOCAL factor, from raw data, independent of the other event. Split: 1/3 = 0.33333.... Dividend: prior close is the raw closes[5] = 101.00 (this value is already on the post-split basis, since index 5 > the split's index 4, and that's fine, because by the time this dividend was paid, the stock genuinely was trading on the post-split basis; using the raw, contemporaneous price at the time of payment is exactly correct, whatever basis that happens to be on): factor = (101.00 − 1.20)/101.00 = 0.98812.... Step 2: walk backward, accumulating. Indices 6–7 (on or after the dividend, the last event): factor 1.0, unchanged: 99.80, 100.50. The moment we pass index 6, the cumulative factor becomes 0.98812.... Index 4–5 (after the split, before the dividend): factor 0.98812...: 99.00 × 0.98812 ≈ 97.824, 101.00 × 0.98812 ≈ 99.8 (this lands almost exactly on the ex-dividend close of 99.80 because the dividend factor was computed from this exact value, a useful internal-consistency check to notice). The moment we pass index 4, the cumulative factor multiplies again by the split's 0.33333..., becoming ≈ 0.32937. Indices 0–3 (before both events) carry the full product: 300.00 × 0.32937 ≈ 98.81, 306.00 × 0.32937 ≈ 100.79, 294.00 × 0.32937 ≈ 96.84, 303.00 × 0.32937 ≈ 99.80. Step 3, sanity-check against Exercise 1's own single-split intuition. Notice index 3 (≈99.80) lands almost exactly at the same level as index 4 (99.00, pre-dividend-adjustment), because between those two indices only the split had happened yet, nothing else, so the adjustment there is doing exactly the same single job Exercise 1 already validated. Read it. The order in which you're handed events doesn't matter, and the order in which they occurred doesn't require special-casing, because each event's own factor is always computed from the raw prices around it, composing many events is just "multiply together whichever local factors apply to each date," which is exactly what makes the algorithm correct without ever needing a special case for "what if there are three events instead of two."

Worked example 4: The point-in-time trap, and the number it silently changes (India, ₹)

NILAM's (illustrative) revenue filings: FY2022 ₹3,600, filed 2022-05-18; FY2023 ₹4,200, filed 2023-05-20; FY2024 ₹4,800, filed 2024-05-19. A junior analyst is asked, "what is NILAM's latest known revenue, as of 2023-08-01?" and writes max(records, key=lambda r: r["fiscal_year"]).

Solution. Step 1, run the naive shortcut and see what it returns. max(...) by fiscal year, with no date filter at all, simply returns whichever row has the largest fiscal_year value in the entire table (FY2024, revenue ₹4,800) regardless of when that row was filed. Step 2, check whether that figure could possibly have been known on the query date. FY2024 was filed 2024-05-19, over nine months after the query date of 2023-08-01. The naive answer is not merely imprecise; it describes a fact that did not yet exist in the world on the date being asked about. Step 3, apply the correct point-in-time filter first. filter_point_in_time(records, "2023-08-01") keeps only rows with filed_date <= "2023-08-01" (FY2022 and FY2023, dropping FY2024 entirely) then latest_as_of takes the largest fiscal year among those, correctly landing on FY2023, ₹4,200. Step 4: quantify the damage. ₹4,800 vs. ₹4,200 is a 14.3% overstatement of the correct point-in-time figure, entirely from skipping one date comparison. If this revenue feeds a valuation multiple or a factor score inside a backtest, that backtest has just been handed information nine months before it existed. Read it. The bug is not in the arithmetic, max() did exactly what it was told. The bug is in what it was told to maximize over: the entire table, instead of the slice that was actually knowable on the date in question. This is precisely the class of error QD1.03's no_lookahead harness exists to catch automatically, once your backtests are built at scale, but no automated harness helps if the data itself, as this module teaches, was never filtered correctly in the first place.

Worked example 5: Survivorship bias, computed exactly (India ₹ and US $)

A five-year-old research note claims "our universe of Indian small-caps returned +105% on average." An analyst is asked to check this against the full historical record, including names no longer listed.

Solution. Step 1: reproduce the claimed number. The three currently-listed survivors returned +180% (NEELKANTH), +95% (SARASWATI), and +40% (HIMAVAN). Their average: (1.80+0.95+0.40)/3 = 1.05, +105%, exactly matching the note's claim. Step 2, ask the question the note didn't: were there only ever three companies in this universe? No, a full historical membership record shows two more names, KUSHAGRA and AMBALIKA, that were part of the original universe and were subsequently delisted at a total loss (−100% each). Step 3, recompute the TRUE average, including them. (1.80+0.95+0.40−1.00−1.00)/5 = 0.23: +23%. Step 4: quantify the bias. 1.05 − 0.23 = 0.82, the claimed number overstates the true historical average by 82 percentage points. Step 5, confirm the same order of magnitude in a different market. A companion US universe (CLRRDG +220%, STNHVN +60%, WLBROK +30% surviving; OKMULEE and PENWICK wiped out at −100% each) gives a naive average of (2.20+0.60+0.30)/3 ≈ 103.3% against a true average of (2.20+0.60+0.30−1.00−1.00)/5 = 22.0%, a gap of 81.3 percentage points, essentially the same magnitude despite entirely different underlying numbers. Read it. An 80-plus-point gap from a universe of only five names is not a contrived worst case, it is close to the mechanical default whenever a meaningful share of a small universe fails completely, because a −100% outcome subtracts a full percentage point of average return per name for every point a comparable +100% winner adds, while a survivors-only view sees only the winners. Any "our universe returned X% historically" claim that does not explicitly say how delisted names were handled should be treated as unverified until you can answer that question yourself.

Worked example 6: The full pipeline, and the return that only exists once you build it (US, $)

A universe of three US names over a shared timeline: BRPEAK (listed throughout, 3-for-1 split partway through), PENWICK (delisted partway through), CLRRDG (IPO'd partway through). Query: what was each currently-eligible constituent's return from 2022-01-01 to 2023-06-01?

Solution. Step 1, establish membership first, before touching a single price. PENWICK's exit date is 2022-06-01, strictly before the query's as_of of 2023-06-01, so PENWICK is excluded from this query's answer entirely (it is still counted in a survivorship_bias_gap-style historical average, per Worked Example 5, it is simply not part of this specific forward-looking snapshot, because by 2023-06-01 it genuinely wasn't investable any more). CLRRDG's entry date (2022-01-01) exactly matches this query's since, so it qualifies. BRPEAK was listed throughout. Step 2, for each surviving-and-eligible name, back-adjust using only events known by as_of. BRPEAK's 3-for-1 split (known well before 2023-06-01) is applied; its adjusted return from since to as_of comes to +6.52%. CLRRDG has no corporate actions in this window; its raw return from 140 to 150 is +7.14%. Step 3, state what the panel actually returned. [{"ticker": "BRPEAK", "return_since": 0.065217}, {"ticker": "CLRRDG", "return_since": 0.071429}], exactly two rows, PENWICK correctly absent, computed from a function that never needed a single special case for "oh, and also check if anyone delisted" or "oh, and also check if the split matters here", those checks are simply built into build_pit_panel's membership and adjustment logic, running the same way for every query. Read it. This is the entire module in one function call: the query date determines who's even asked about (survivorship), each answer is computed only from what was knowable by that date (point-in-time), and each answer is stated on a consistent, current-share-basis footing regardless of how many splits or dividends sit in the company's history (adjustment). Get any one of the three wrong, and this same function call returns a number that looks exactly as clean and exactly as confident, and is wrong.

Practice set

Work every problem fully (computing your own answer) before reading its solution. Every problem below uses numbers distinct from the module's own running examples, so you cannot simply recall a worked answer instead of computing one. All companies are illustrative and fictional; all data is explicitly synthetic (R3).

P1 (guided). Split arithmetic, sanity check. A company executes a 4-for-1 split. Before the split, a holder has 50 shares at ₹800 each. State the exact share count and per-share price immediately after, and confirm total value is unchanged.

Solution. 4-for-1 means every 1 share becomes 4: 50 × 4 = 200 shares. Total value before: 50 × 800 = ₹40,000. That value must be unchanged, so the new per-share price is 40,000 / 200 = ₹200. Confirm: 200 × 200 = ₹40,000, exactly unchanged, as it must be, since a split is purely an accounting re-slicing of the same total value.

P2 (guided). Split-adjust a short series. Raw closes [240.00, 245.00, 60.00, 61.50] ($) reflect a 4-for-1 split taking effect at index 2. Compute the split-adjusted series and the boundary-day return, both raw and adjusted.

Solution. Divide indices 0–1 by 4: [60.00, 61.25, 60.00, 61.50]. Raw boundary return: (60.00 − 245.00)/245.00 ≈ −75.5% (a phantom crash: 4-for-1 splits produce even more dramatic phantom moves than 2-for-1 or 3-for-1). Adjusted boundary return: (60.00 − 61.25)/61.25 ≈ −2.04%, the real, modest move.

P3. Dividend factor by hand. A stock's close the day before its ex-dividend date is ₹500; the dividend is ₹10. State the adjustment factor and the adjusted value of a ₹480 close from three days earlier.

Solution. Factor = 1 − 10/500 = 0.98. Adjusted: 480 × 0.98 = ₹470.40.

P4. Price-return vs. total-return, stated precisely. In your own words, state the one-sentence difference between a price-return series and a total-return series for the same stock, and say which one a real investor's account balance actually tracks.

Solution. A price-return series reflects only the change in share price (split-adjusted but dividend-blind); a total-return series additionally adds back every dividend paid, as if immediately reinvested (or at minimum, credited). A real investor's account balance tracks total return, the dividend cash was genuinely received (or reinvested), so any comparison claiming to describe "what an investor actually earned" that uses price-return numbers is understating the truth.

P5. Compose two splits. A stock undergoes a 2-for-1 split at index 2 and a further 3-for-2 split (ratio 1.5) at index 5, over an 8-point raw series [80, 82, 79, 40, 41, 27, 27.5, 28]. What is the adjustment factor applied to index 0 (before both splits), and to index 3 (after the first split, before the second)?

Solution. Working backward: index 5 onward has factor 1.0 until we pass index 5, where the running factor picks up 1/1.5 = 0.6667. Index 3–4 (after the first split, before the second) carry factor 0.6667. The moment we pass index 2, the running factor also picks up 1/2 = 0.5, compounding: 0.6667 × 0.5 = 0.3333. So index 0 (before both splits) carries factor 0.3333 (the product of both splits' factors), and index 3 carries factor 0.6667 (only the second split's factor, since it comes after the first split already happened).

P6. Diagnose a suspicious jump. A data-quality scanner with a 15% threshold flags day 12 of a series with a −34% single-day move, but a check of the company's corporate-actions feed shows no split or dividend on that date. What are the two most likely explanations, and which would you check first?

Solution. Two most likely: (a) a genuine large move, an earnings shock, a regulatory action, a merger collapse, or similar real news, in which case the flag is a false positive and the number is correct as-is; (b) a data error, a bad tick, a currency/unit mixup (e.g., a figure reported in the wrong scale for one day), or a corporate action that the feed's own actions table simply failed to record (a false negative in the actions feed, even though the price-jump scanner correctly caught its symptom). Check first whichever is cheaper to rule out, typically, checking a second independent price source or a news search for that date and ticker is faster than manually re-deriving whether a real corporate action occurred.

P7. Point-in-time filter, from scratch. Records: {FY2020, rev=200, filed="2020-05-01"}, {FY2021, rev=230, filed="2021-05-15"}, {FY2022, rev=250, filed="2022-05-10"}. As of "2021-01-01", which records are point-in-time known, and what is the correct "latest known" answer?

Solution. Only FY2020 (filed="2020-05-01" <= "2021-01-01") is known; FY2021's filing (2021-05-15) is still over four months in the future relative to the query date, and FY2022 is further still. The correct latest-known answer is FY2020, revenue 200, even though, in hindsight, "2021" as a calendar year has already begun, FY2021's actual results were not filed yet.

P8. Quantify a point-in-time leak. Using P7's records, what would the WRONG (naive, max by fiscal year with no date filter) answer be as of "2021-01-01", and by how many rupees/dollars does it overstate the correct figure?

Solution. The naive shortcut ignores filed_date entirely and returns whichever row has the largest fiscal_year in the whole table (FY2022, revenue 250) regardless of query date. This overstates the correct point-in-time answer (FY2020, revenue 200) by 250 − 200 = 50, a 25% overstatement, and does so by using a figure that would not be filed for another 16 months.

P9. Survivorship gap, from scratch. A universe has four constituents: two survivors at +60% and +20%, and two delisted at −100% and −80% (a partial-salvage outcome, not a total wipeout). Compute the naive (survivors-only) average, the true (full-universe) average, and the gap.

Solution. Naive: (0.60+0.20)/2 = 0.40 (+40%). True: (0.60+0.20−1.00−0.80)/4 = (−1.00)/4 = −0.25 (−25%). Gap: 0.40 − (−0.25) = 0.65 (65 percentage points, and notice the true average here is not just smaller but actually negative) the naive survivors-only view doesn't just overstate the return, in this case it hides that the honest historical answer was a net loss.

P10. Why "−100%" and not simply "excluded". Explain why a delisted company's realized return must be included in a survivorship-controlled average as an actual (often very negative) number, rather than simply being dropped from the average entirely.

Solution. Dropping a delisted name from the average entirely (rather than counting its real, realized return) still produces a survivors-biased number; it just does so more subtly, by silently shrinking the denominator instead of silently shrinking the numerator. An investor who actually held that name lived through its real outcome (frequently near-total loss), and any average claiming to describe "what happened to the universe" must include that lived outcome, at its real value, or the average is describing a universe that never existed.

P11. Membership timing. A company's entry_date is "2021-06-01" and it has no exit_date (still listed). For a build_pit_panel query with since="2021-01-01", is this company included? What if since="2021-09-01" instead?

Solution. With since="2021-01-01": the company had not yet entered by that date ("2021-01-01" < "2021-06-01"), so it is excluded, there is no valid since-date price to compute a return from, because it wasn't listed yet. With since="2021-09-01": the company had already entered ("2021-06-01" <= "2021-09-01"), and with no exit date it remains eligible, so it is included (assuming it also has valid price data on both query dates).

P12. Exit-date boundary. A company's exit_date is exactly "2022-06-01". For a query with as_of="2022-06-01" (exactly matching the exit date), is the company included or excluded? Justify from the membership rule exit_ is not None and as_of >= exit_.

Solution. Excluded. The rule excludes whenever as_of >= exit_, and "2022-06-01" >= "2022-06-01" is true (equal counts as excluded). The convention here treats the exit date itself as the first date the company is no longer investable, consistent with this module's split/dividend convention where an event's own index is already "on the new basis," i.e., the boundary date belongs to the new regime, not the old one.

P13. The strict-PIT adjustment nuance. A stock has a split at a date AFTER your as_of query date. Under this module's strict convention (Section 8), is that split applied when computing the adjusted price as of that as_of date? What is the argument for the opposite (looser) convention some vendors use?

Solution. No: this module's strict convention only applies events with event["index"] <= as_of's index, so a split dated after as_of is excluded from that query's adjustment, consistent with never letting any input reach past the query date. The argument for the looser, "adjust using full history regardless of query date" convention some vendors use: a split ratio is a purely mechanical, non-informative fact once it has happened, so using the fully-current-adjusted number is simply more convenient for comparing prices across periods and arguably isn't really "the future leaking in" the way a fundamentals number would be.

P14. Currency-scale trap. A US vendor reports a company's revenue as 4,500 with a footnote "figures in $ thousands"; an India vendor reports another company's revenue as 450 with a footnote "figures in ₹ crore." Convert each to its full plain-currency value.

Solution. US: 4,500 thousand dollars = $4,500,000 ($4.5 million). India: 450 crore = 450 × 10,000,000 = ₹4,500,000,000 (₹450 crore = ₹4.5 billion). Note how easy it would be to compare these two footnoted figures directly and wrongly conclude they're of similar magnitude, they differ by roughly three orders of magnitude once correctly converted to a common plain-currency basis.

P15. Ticker-reuse hazard. Explain, in your own words, why joining two datasets purely on ticker symbol (with no stable internal company ID) creates a specific, real risk over a multi-decade history, and name this program's own bundled dataset that avoids the problem.

Solution. An exchange can, after enough time has passed since a company's delisting, reassign that now-vacant ticker symbol to a completely unrelated new company. A join keyed purely on the ticker string, with no other identity check, will silently splice the old company's history and the new company's history together as if they were one continuous entity, a slow-motion version of the exact same "invisible wrong number" problem this whole module warns about. This program's bundled finance-db.sqlite (built for CS2.03) avoids the problem by keying every table on a stable internal company_id, with the ticker treated as just one more attribute that can change without breaking the join.

P16. Choosing a vendor, applied. You are building a serious multi-year backtest of a value-and-momentum strategy on US small-caps, on a limited individual budget, and survivorship-bias-free history is non-negotiable for you. Name one appropriate paid source and explain, in one sentence tied to this module's own material, why it fits.

Solution. Norgate Data (or, if academic access is available, CRSP via WRDS), both are specifically built and marketed around the exact guarantee this module spends Section 7 demonstrating the danger of skipping: delisted/failed companies are retained in the historical record rather than silently vanishing the way a "current constituents" list would drop them, which is precisely the property a small-cap value-and-momentum backtest cannot afford to be missing.

Applied mini-project

Build and stress-test a small point-in-time, survivorship-controlled panel.

This project is your own hand-built rehearsal of exactly what Exercise 8 and Worked Example 6 do in the code pack, the app never grades this project automatically; you self-check it against the rubric below, and the discipline it builds is what QD1.02's factor work and QD1.03's backtesting engine will assume you already have.

Do this, in order:

1. Design a small universe. Invent (clearly fictional, never a real company) 4–5 companies in one market of your choice (India or US), spanning at least one that is still listed throughout your chosen window, at least one that is delisted partway through, and at least one that lists partway through. Give each a raw price series over a shared set of at least 6 dates, and give at least one company at least one corporate action (a split or a dividend) partway through its series.
2. Compute, by hand, what a naive pipeline would report. Pick a since and as_of date pair. For every company, compute the raw, unadjusted return over that window, and separately list which companies a naive "current constituents" view would even include (i.e., every company still listed today, regardless of whether it was listed for your whole window).
3. Compute the correct, point-in-time, survivorship-controlled, adjusted answer. By hand or by adapting this module's code pack functions, determine the correct membership (who was genuinely listed for the entire since-to-as_of window) and the correct adjusted return for each eligible company.
4. Quantify the gap. For at least one company with a corporate action in its window, state the raw vs. adjusted return and the size of the gap. Separately, state which compan(ies) the naive view included that the correct view excludes (or vice versa), and why.
5. Write the one-paragraph verdict. In your own words, explain which of the three disciplines (adjustment, point-in-time, survivorship) mattered most for your specific universe design, and what a strategy backtested on the naive version of your data would have wrongly concluded.

Deliverable and self-check. One short write-up containing: your universe (companies, dates, prices, at least one corporate action), the naive figures, the correct figures, the quantified gap(s), and your one-paragraph verdict. Score yourself: 2 points each for (a) the universe genuinely includes a still-listed, a delisted, and a late-entrant company; (b) at least one corporate action is present and its adjustment is computed correctly; (c) the naive-vs-correct membership difference is explicitly identified; (d) the adjusted-vs-raw return gap is correctly quantified with real numbers, not just described qualitatively; (e) the verdict correctly names which discipline mattered most for your specific design and ties it back to a concrete number from your own work. Pass at 8/10 with no zero on any item. Keep this write-up, it is the exact shape of reasoning QD1.02's factor-construction work and QD1.03's no_lookahead harness will assume you can already do without being walked through it again.

Lab: open this module's code pack in the app's Code tab and clear all eight exercises' hidden tests, that is the machine-graded version of the same discipline this project rehearses by hand.

Reading & resources

Tags: ✓ free · $ paid · ▲ beginner · ▼ advanced. Everything below is a real, named anchor to verify vendor claims against and to learn from, never a substitute for the vendor's own current documentation, which is the only authoritative source for pricing, coverage, and methodology specifics.

Perishable specifics, as of 2026-07; verify current. Vendor names, product names, pricing tiers, and even a vendor's continued existence are exactly the kind of fact that changes without warning (a platform gets rebranded. Refinitiv Eikon became LSEG Workspace after LSEG's acquisition of Refinitiv; a free tier shrinks; a company is acquired). Every vendor named below is treated as a spec to re-verify on the vendor's own current page, never as a permanent fact. The disciplines this module teaches, adjustment, point-in-time filtering, survivorship control, are durable regardless of which vendor's logo happens to be on the data next year.

Core mechanism (adjustment, point-in-time, survivorship):

• This module's own code pack (QD1.01), the fastest way to re-derive every formula above against hidden data you haven't seen. ✓ ▲
• CRSP (Center for Research in Security Prices), via a university/WRDS subscription where available, the academic gold standard for survivorship-bias-free, point-in-time US equity data; reading its own methodology documentation is the single best primary source for how a rigorous vendor actually implements what this module teaches by hand. $ ▼
• Norgate Data's own documentation on its survivorship-bias-free equities data, a clear, practitioner-oriented (rather than academic) explanation of the same guarantee, aimed at individual quant developers. $ ▲–▼

United States: official / free:

• SEC EDGAR (sec.gov/edgar, and the data.sec.gov XBRL companyfacts API), the primary filing repository itself; the single best free source for US fundamentals with real, checkable filing dates. This program's own tools/fetch_edgar.py talks to exactly this API. ✓ ▲–▼
• FRED (fred.stlouisfed.org), the standard free source for US macroeconomic series. ✓ ▲
• Stooq (stooq.com), free historical EOD price CSVs in a simple format; the real-world analogue this program's synthetic us-eod dataset stands in for. ✓ ▲

United States: paid:

• Bloomberg (Terminal / Data License) and LSEG Workspace (formerly Refinitiv Eikon), the two dominant full-service institutional platforms, global coverage, real-time and historical. $ ▼
• Polygon.io and Databento: developer-first, API-based, priced for an individual quant or small fund rather than an institutional desk. $ ▲–▼
• Tiingo: an affordable freemium/subscription API with clean adjusted EOD prices and fundamentals, a genuinely accessible entry point. $ ▲
• Nasdaq Data Link (formerly Quandl), a marketplace spanning both free and paid datasets from many contributing providers. ✓/$ ▲

India: official / free:

• NSE and BSE own websites, the daily bhavcopy official EOD dump, and each exchange's own filings section. ✓ ▲
• AMFI (amfiindia.com): daily mutual-fund NAV data. ✓ ▲
• RBI's DBIE (Database on Indian Economy), the free, official source for Indian macro/monetary data, the direct India analogue of FRED. ✓ ▲
• Screener.in: a widely-used free (with a paid tier) source for Indian company fundamentals and screening, especially popular with the retail/prosumer research community. ✓ ▲
• data.gov.in and the MCA (Ministry of Corporate Affairs) filings portal, broader Indian public-data and company-filings sources. ✓ ▲–▼

India: paid:

• CMIE (Prowess for corporate financials, Economic Outlook for macro), long-standing, widely used in Indian academic and institutional research; the closest India analogue to a Compustat/FRED pairing. $ ▼
• Capitaline and ACE Equity (Accord Fintech), established Indian financial-database vendors covering listed-company fundamentals and prices. $ ▲–▼
• TrueData and Global Data Feed (GDFL): developer-accessible real-time/historical India market-data vendors serving the algorithmic-trading community. $ ▲
• Zerodha Kite Connect: a broker API (not a pure data vendor), the most common practical way an individual Indian quant developer gets live and historical NSE/BSE data programmatically today. $ ▲

Do this, not just read. Pick one free source from each market above (SEC EDGAR for the US, NSE's bhavcopy for India) and actually look at one real, current filing or one real, current daily price file. Note what a real filing's own filing date looks like versus its fiscal-period-end date, the gap between them is the exact point-in-time discipline Section 6 teaches, now seen on a document that actually exists, not just this module's synthetic example.

Teach it back & journal

Feynman prompt. Explain to a colleague who has only ever used a "clean" pre-adjusted price file why raw vendor data cannot be trusted as-is, using exactly three sentences: one about splits/dividends (a mechanical price change that isn't an economic one), one about point-in-time discipline (a fact isn't usable before it was knowable), and one about survivorship (today's "current list" silently deletes history's failures). Then, without looking back at this module, state the exact numeric magnitude of the survivorship-bias gap this module demonstrated (roughly how many percentage points, from how few companies), if you can produce that number from memory and explain why it's that large rather than a rounding error, you have installed the module's real lesson, not just its vocabulary.

Journal prompt. Think of one number you have used, cited, or trusted recently (in this program, at work, or in the news) that came from a "current," "as of today," or "latest" data source with no stated point-in-time or survivorship caveat. Write three sentences: (1) what discipline (adjustment, PIT, or survivorship) that number was most exposed to, and why; (2) what you would have to check, concretely, to know whether it was actually correct; (3) whether you would still trust it at face value after this module, or would now go check the underlying source first. Then write one further paragraph: this module's payoff is not "here is a list of bugs to remember" but a standing reflex (ask what a number was adjusted for, what it could have known, and who it silently stopped counting) before you build anything on top of it. Which of the three disciplines is most likely to bite YOU specifically, given the kind of data work you actually do, and what will you do this week to make checking for it automatic?

This module's flashcards and mastery quiz are wired into the app: see the node's Quiz and Reviews.