Portfolio Optimization in Code

Branch: Quant Developer (quant-dev, Quant-Code region) · Node QD1.04 · ~20 focused hours · Gate: Quiz ≥ 85% + code-pack ≥ 85% · Quant Optimizer badge

AA1.01 handed you a spreadsheet-Solver's confident answer: "24.51% equity, 70.60% debt", and then, in its own final act, broke that answer on purpose: nudge one expected-return assumption by about a percentage point, well inside any honest estimate of that number's own uncertainty, and the "optimal" weight on one asset swings by ten-plus points and flips its sign, while the optimizer's own scorecard (its Sharpe ratio) barely moves at all. That demonstration was run in a scratch Python script, by hand-verified, and reported to you. This module is where you stop reading someone else's verified script and start owning the script itself. Every closed form AA1.01 wrote out on paper, every reverse-optimized equilibrium return AA1.02 blended a view into, every risk-parity iteration AA1.02 ran to convergence, you will implement, run, and break, in your own code, against your own hidden tests, until the fragility is something you have personally reproduced rather than something you were told about.

Here is the whole module in one sentence you will spend twenty hours earning the right to believe: an optimizer is a very fast, very literal machine for turning your assumptions into weights, and the only way to know whether to trust a specific set of weights is to have the code that can re-run the same optimization under a slightly different, equally defensible assumption. Mean-variance optimization done honestly is not "call a solver once." It is: build the covariance matrix and prove it is a valid input before you touch it; solve the unconstrained case in closed form; add the constraints a real mandate actually has (long-only, a single-name cap, a sector or liquidity ceiling) and watch the elegant algebra become a numerical program; then run the two stress tests that separate a professional build from a fragile one, perturb an input by a defensible amount and watch what happens to the weights (QD1.04-e05), and re-estimate the covariance matrix itself from a short, noisy real sample and watch what happens there (QD1.04-e08). Two entirely different failure modes, both genuinely called "the optimizer blew up," and this module is where you learn to tell them apart because you built the code that produces each one.

The promise for the hours ahead: by the end, you will have a small, tested library of functions, covariance construction and validation, GMV, unconstrained and constrained tangency portfolios, a single-view Black-Litterman blend, an N-asset risk-parity solver, and a shrinkage-covariance estimator, that reproduces, to the decimal, every number AA1.01 and AA1.02 verified by hand. You will also have run, yourself, the two numerical demonstrations of exactly why an unconstrained optimizer fed noisy real-world inputs is not a reliable black box, and the two fixes (Black-Litterman, risk parity) and one hardening technique (shrinkage) that a working quant desk actually reaches for instead of simply trusting a bare scipy.optimize call.

Every exercise in this module lives in its code pack, code-packs/QD1.04.json (module id QD1.04, eight exercises QD1.04-e01 through QD1.04-e08), reachable from this node's Code tab. Every one ships a structured fallback as its definition of record, so the pack is 100% completable offline with zero runtime, live execution in the browser is a bonus, never a requirement to pass the gate.

A binding runtime note, stated once and then never re-litigated. This program's live code runner is Pyodide (CPython-in-WASM); it ships numpy, pandas, scipy, and scikit-learn, but it does not ship cvxpy (no pure-Python convex-optimization DSL of that shape is available in-browser). Every quadratic or convex program in this module is therefore solved one of two ways: a closed form wherever one exists (GMV, unconstrained tangency, the single-view Black-Litterman posterior, pure linear algebra, no solver needed), or scipy.optimize.minimize (method "SLSQP") for anything carrying inequality or box constraints. A real quant desk with a large constraint set (hundreds of names, sector and turnover budgets, transaction-cost penalties) very often does reach for cvxpy or a dedicated QP solver (OSQP, quadprog), those are named here as production tools, exactly the way this program names XGBoost/LightGBM or PyTorch elsewhere: real, worth knowing, and not required to build genuine understanding of the underlying optimization.

Learning objectives

By the end you can:

1. Implement mean-variance optimization end to end in code: build an n-asset covariance matrix from volatilities and a correlation matrix, validate it is positive semi-definite before trusting anything downstream of it, and compute the global minimum-variance (GMV) and unconstrained tangency portfolios from closed-form linear algebra: reproducing AA1.01's own hand-verified numbers to the decimal.
2. Add real constraints via scipy.optimize.minimize: turn a mandate's actual rules (long-only, a single-asset cap, a sector or liquidity ceiling, a hard exclusion) into bounds and constraints on a numerical Sharpe-maximization, and quantify the Sharpe ratio a specific constraint costs relative to the unconstrained optimum.
3. Prove, numerically and in your own code, that unconstrained MVO is an "estimation-error maximizer" (Best & Grauer 1991; Michaud 1989): perturb one expected-return assumption by an amount smaller than any honest confidence interval around it, and reproduce a ten-plus-point weight swing and a sign flip, while the portfolio's own Sharpe ratio moves almost nothing.
4. Implement Black-Litterman in code: reverse-optimize the market-implied equilibrium return vector from benchmark weights and a risk-aversion coefficient, blend in a single stated view via the closed-form posterior, verify the no-view identity that catches a broken build, and contrast the resulting portfolio against one built from noisy trailing-mean history on the identical asset menu.
5. Implement risk parity in code: compute marginal and percent contribution to risk for any weight vector, implement the two-asset closed form, and implement the N-asset iterative solver that a naive inverse-volatility shortcut cannot replace once three or more assets carry genuinely different pairwise correlations.
6. Implement and diagnose a second, independent way naive optimizers blow up: estimate a covariance matrix from a short real return history two different ways (the raw sample covariance and a Ledoit-Wolf shrinkage estimate) and show, numerically, that GMV weights built from the raw estimate swing far more across two different short windows of the same real menu than weights built from the shrunk estimate.
7. State precisely what this module owns versus what AA1.01/AA1.02 own: the theory, the derivations, and the by-hand verification of every formula below belong to AA1.01 and AA1.02 (Finance-Plus Ring); this module never re-derives them, it implements, validates, hardens, and stress-tests them in running code.
8. Know honestly what this program's runtime can and cannot run: name cvxpy as a real production tool this module's runtime cannot execute, and explain why scipy.optimize plus closed forms are the substitute here, not a downgrade in understanding.
9. (Productivity objective: R10 duality.) Use an AI coding assistant to accelerate drafting an optimization scaffold, the matrix algebra, the scipy.optimize boilerplate, for a task you can already solve and verify yourself, and catch the specific, silent ways AI-generated numerical code goes wrong: a transposed matrix, a missing constraint, a flipped objective sign, a covariance estimator applied with the wrong rowvar axis.

The duality, stated once (R10). As in every module in this program, this node carries two objectives at once. The understanding objective (items 1–8) is what the mastery gate rewards: you earn the pass by implementing each function yourself, clearing its hidden tests, and being able to explain in your own words why each demonstration comes out the way it does. The productivity objective (item 9) is the payoff you keep afterward: knowing how to point a copilot at optimization code without being quietly misled by it. A tool can never buy you a pass on the gate; QD1.04-e04 and QD1.04-e08's choose-impl structured fallbacks are built from exactly the bugs a fluent-but-wrong AI-generated answer tends to contain.

Prerequisites & connections

Builds on. AA1.01 (Mean-Variance Optimization, the Efficient Frontier, SAA vs TAA. Finance-Plus Ring, Asset Allocation & Performance branch) is this module's direct parent and the theory this module never re-derives: the n-asset covariance matrix, the Markowitz problem, the GMV and tangency closed forms, the constrained-frontier discipline, and, most importantly, the input-sensitivity ("estimation-error maximizer") failure this module reproduces numerically in QD1.04-e05. AA1.02 (Black-Litterman, Risk Budgeting, Liability-Relative & Goals-Based Allocation) supplies the second half of this module's spine: the reverse-optimized equilibrium return, the single-view posterior closed form, marginal/component contribution to risk, and the N-asset risk-parity iteration, all implemented here in QD1.04-e06/e07. Read both fully before this module if you have not already, this module implements their mathematics, it does not re-teach it, and every worked example below cites the exact section of AA1.01 or AA1.02 it is reproducing in code. CS1.05 (Data Structures & Algorithms) is assumed for ordinary Python fluency and comfort reading numpy/scipy code; DA1.01 (NumPy & Vectorized Computation) is recommended before this node (not a hard prerequisite: QD1.01 is this branch's day-one entry) for anyone who has not yet built real comfort with array broadcasting and linear algebra in numpy. QD1.01 (Market & Fundamental Data Pipelines) is this branch's own prerequisite chain step immediately before this node and supplies the point-in-time, survivorship-controlled price panels a real optimization would run against in production; QD1.04-e08 uses a small, honest slice of that same discipline against this program's bundled us-eod/nse-eod datasets. You do not need the statistics/ML theory this program keeps in G1's QM branch (C-3 binding boundary), nothing here re-derives a distributional assumption or a hypothesis test; every number below is either a closed-form matrix computation or a numerical optimization over inputs you are handed.

Feeds forward. QD1.03 (Building a Backtesting Engine), already authored in this branch, says so explicitly in its own text: its position sizing is "deliberately trivial (a scalar in {−1, 0, +1}) so that nothing in the measured gaps can be attributed to a clever optimiser," because (in its own words) "QD1.04 owns portfolio construction." Every function you write here is exactly what a more realistic backtest would plug into that scalar-sizing slot: a real strategy sizes each name by a constrained tangency weight, a risk-parity weight, or a Black-Litterman-informed tilt, not a flat ±1. QD1.02 (Signal & Factor Construction) supplies the kind of cross-sectional view a real Black-Litterman blend would use as its Q, a factor score is a defensible, evidence-based view in exactly BL's sense, sized with a stated confidence rather than asserted as fact. QD2.02 (Risk Systems in Code. VaR, ES, Stress & Scenario) and QD2.03 (Performance & Attribution Analytics in Code) both take a portfolio's weights as a given input; the weights this module teaches you to build, constrain, and stress-test are exactly what those two modules measure the risk and performance of. QD2.05, the branch's capstone, assumes you can size a strategy's positions defensibly, this module is where that skill lives. Soft companion: QD1.01's vendor/data-pipeline discipline and AA1.03's attribution machinery both assume a policy portfolio was built competently somewhere upstream; this module, alongside AA1.01/AA1.02, is that upstream.

Core teaching content

1. Why this module writes code instead of re-deriving anything

AA1.01 and AA1.02 already did the derivations: the covariance matrix from volatilities and correlations, the GMV and tangency closed forms, the reverse-optimized equilibrium, the single-view Black-Litterman posterior, and the risk-parity iteration are all written out, hand-verified, and cross-checked in those two nodes. Re-deriving any of it here would violate this program's own R1 rule (reference, don't duplicate) and would waste your time on formulas you already own. What this module adds is the thing a spreadsheet or a single verified script cannot give you: a tested, reusable implementation you can re-run under a changed assumption in seconds, which is the only real defense against the fragility AA1.01 spent an entire section proving. Every one of this module's eight exercises is, in a precise sense, "the same math AA1.01/AA1.02 already checked by hand, now checked by a machine, against hidden tests, on inputs you have never seen."

2. The machine, assembled in numpy

Recall the n-asset machine from AA1.01 §2: collect volatilities in a vector σ, correlations in a matrix ρ, and build Σ = diag(σ) · ρ · diag(σ), so Σ_ij = σ_i σ_j ρ_ij. In code this is a single nested comprehension or a numpy broadcast. QD1.04-e01's build_covariance(sigma, corr) does exactly this, and its hidden tests reuse AA1.01's own Bharat Family Office (India, four-asset: Equity/Debt/Gold/REIT) and Northfield Endowment (US, three-asset: Equity/Bonds/Alts) menus, so your implementation is graded against the exact matrix AA1.01 built by hand: Σ_bharat(%²) = [[361.00, 11.40, -14.25, 177.65], [11.40, 36.00, 13.50, 35.70], [-14.25, 13.50, 225.00, 12.75], [177.65, 35.70, 12.75, 289.00]].

Before a single frontier number is computed from any covariance matrix, run the check almost nobody runs by hand and everybody should run in code: is the underlying correlation matrix even valid? A set of pairwise correlations assembled from separate judgment calls is not guaranteed to be internally consistent, some combination can imply a negative variance somewhere. The test, exactly as AA1.01 §3 states it, is that every eigenvalue of the correlation matrix must be non-negative (positive semi-definite): numpy.linalg.eigvalsh(corr).min() >= -tol. QD1.04-e01's is_psd_correlation implements this one-line check, and its hidden tests include a deliberately broken correlation matrix (three assets pairwise correlated at 0.9, 0.9, and −0.9) whose smallest eigenvalue comes out −0.8, unambiguously invalid, and your function must say so.

4. Real constraints: from a closed form to a numerical program

No real mandate lets an institution run the unconstrained tangency portfolio above, regulation, mandate language, and plain common sense forbid shorting a domestic sleeve to fund a bigger position elsewhere, and illiquid buckets routinely carry a policy cap regardless of what the unconstrained math wants. The instant a bound like 0 ≤ w_i ≤ cap enters the problem, the elegant closed form of §3 is gone, and you are solving a numerical quadratic program. QD1.04-e04's constrained_tangency_weights(Sigma, mu, rf, bounds) does this with scipy.optimize.minimize(neg_sharpe, w0, method="SLSQP", bounds=bounds, constraints=cons), where cons enforces sum(w) == 1 and bounds is a list of (lo, hi) tuples, long-only, a single-asset cap, or a hard exclusion via (0, 0), all expressible the same way. Run it on Bharat's menu with a long-only bound plus a 50% single-asset cap and you reproduce AA1.01 Worked Example 3 Step 3 exactly: Equity 29.37%, Debt 50.00% (binding), Gold 20.63%, REIT 0.00%, Sharpe 0.5535: down 4.2% relative to the unconstrained 0.578, because the cap binds exactly on the asset (Debt) doing the most Sharpe-maximizing work. Run it on Northfield's menu with Alts capped at a 20% illiquidity ceiling and you reproduce Equity 28.36%, Bonds 51.64%, Alts 20.00%, Sharpe falling from 0.317 to 0.3132, a much smaller relative cost, because Alts there was doing comparatively less of the unconstrained portfolio's diversification work. A constrained frontier can only match or lie inside the unconstrained one, a constraint never improves the best achievable Sharpe ratio, it can only cost you some of it, and the size of that cost is a genuine, quantified diagnostic for how expensive a specific policy rule actually is.

5. Numerically proving the estimation-error maximizer

Here is this module's first center of gravity. AA1.01 §9 named unconstrained MVO an "estimation-error maximizer" (Best & Grauer 1991; Michaud 1989): it does not merely tolerate noise in your return assumptions, it actively amplifies it, because the optimizer's job is to lean as hard as the math allows into whatever asset the inputs say is best. QD1.04-e05's tangency_weights_sensitivity(Sigma, mu, rf, asset_index, delta) reproduces this in code: it is QD1.04-e03's exact formula, except mu[asset_index] is bumped by delta before solving. Run it on Bharat's menu, nudging REIT's assumed return up in small, realistic steps: at the 9.00% baseline REIT sits at −7.37%; nudge to 10.00% (+100bp, a change smaller than many desks' year-to-year capital-market-assumption revisions) and REIT moves to −1.20%; nudge to 10.20% (+120bp) and REIT's weight flips sign to +0.07%, the same optimizer that told you to short REIT by 7.4% now tells you to hold it long, on a change well inside any honest confidence interval around that single number. Watch the column a bare tool never shows you: the portfolio's own Sharpe ratio across this entire sweep moves from 0.578 to 0.571 to exactly 0.5710, nearly flat. The optimizer is close to indifferent between shorting REIT by 7% and owning it at +5%, and nothing in its output tells you your particular answer sits on a knife's edge rather than a stable plateau.

The same fragility, run the other direction, in the other currency: cut Northfield Endowment's Bonds assumption by a mere 80 basis points (4.8% → 4.0%) and the tangency portfolio's Bonds position flips from a substantial long (+44.71%) to an outright short (−27.32%), with Equity and Alts both surging to absorb it (48.17% and 79.15% respectively), one 80bp revision to one input, on a menu that started out looking perfectly ordinary.

6. Fixing the input problem: Black-Litterman in code

Black-Litterman (Black & Litterman, 1992) repairs exactly the fragility §5 just proved, by refusing to start from a noisy trailing-mean return vector at all. It starts instead from the equilibrium return the market's own observed weights already imply, reverse-optimized via Π = δ·Σ·w_mkt (AA1.02 §4.2), and blends in a stated view only where you actually hold one, sized to your actual confidence. QD1.04-e06's black_litterman_posterior(Sigma, w_mkt, delta, tau, P, Q) implements both steps. A units warning that matters here and nowhere else in this module: Black-Litterman's δ (typically 2.5–3.5) is calibrated assuming Σ is in decimal-return units (sigma=0.20 for a 20% vol, not sigma=20), every other function in this module happily works in either percent or decimal units because the ratios cancel out, but δ does not, so build Σ in decimals specifically for this function.

Run it on Ananta Family Office's menu (India, three-asset: Equity/Bonds/Gold at 55%/35%/10%, δ=3.0, τ=0.05) and you reverse-optimize Π = [6.768%, 0.614%, 0.951%], exactly AA1.02 Worked Example 1's numbers. Blend in a single view, equity beats bonds by 8 points (P = [1, -1, 0], Q = 8%), and the posterior comes out [7.637%, 0.560%, 0.954%], a tilt of [+0.869pp, -0.054pp, +0.003pp]. Gold's tilt is essentially zero because the view never named it: Σ⁻¹(posterior − Π) comes out [+0.00224, -0.00224, 0.0], the view's effect is confined exactly to the two named assets, verified to machine precision, never spilling into gold. Before trusting any of this, run the identity check AA1.02 §4.2 insists on: with zero views, Σ⁻¹Π must equal δ·w_mkt exactly, by construction ([1.65, 1.05, 0.30] on Ananta's menu), if a fresh implementation doesn't reproduce this, the bug is in Σ or δ, not in the blending step.

Now for the payoff that directly answers §5. Feed Bharat's (sorry, Ananta's) same three-asset menu to QD1.04-e03's tangency_weights function twice: once with a noisy trailing 3-year sample of average excess returns (equity 15%, bonds 3%, gold −1%, an ordinary bad stretch for gold), and once with this section's Black-Litterman posterior (call tangency_weights with rf=0, since both vectors here are already excess returns, not total returns). The naive vector recommends Equity 36.0%, Bond 76.6%, Gold −12.7%: shorting gold by 12.7% and abandoning the strategic allocation entirely, precisely the corner-solution instability §5 warned about, produced by one unlucky trailing window. The Black-Litterman vector recommends Equity 62.5%, Bond 27.5%, Gold 10.0%, a sensible tilt toward equity, funded from bonds, while leaving gold at exactly 10.0%, unchanged to the decimal from the original benchmark. Same optimizer, same three assets, same code, the only thing that changed is which return vector fed it.

{
  "skill": "Standing up a Black-Litterman scaffold quickly, without letting the tool own whether it is correct",
  "use": "Have an AI coding assistant draft the reverse-optimization and single-view posterior blend in Python, so your own time goes to choosing Sigma/delta/tau/the view itself, not to re-deriving matrix syntax you already understand from AA1.02.",
  "tools": ["A coding-focused chat assistant or IDE copilot", "This module's own hidden tests, run locally, as ground truth -- never the assistant's own claim of correctness"],
  "prompt": "Write black_litterman_posterior(Sigma, w_mkt, delta, tau, P, Q) in Python/numpy: reverse-optimize Pi = delta * Sigma @ w_mkt, then blend a single view using Omega = tau * P @ Sigma @ P.T via posterior = Pi + 0.5 * (Sigma @ P.T) / (P @ Sigma @ P.T) * (Q - P @ Pi). Include an assertion that, with a view whose Q exactly equals P @ Pi, the posterior equals Pi unchanged (zero surprise, zero tilt).",
  "verify": "Never accept the assistant's matrix algebra unchecked. Recompute the no-view identity yourself (Sigma^-1 @ Pi must equal delta * w_mkt exactly, Guardrail #3 Recompute) and cross-check the Ananta Family Office posterior against this module's own verified numbers ([7.637%, 0.560%, 0.954%]) before trusting a single figure downstream. A silently swapped Sigma @ P.T for P.T @ Sigma, or Omega built without the transpose, still returns a plausible-looking number -- AA1.02's own warning that this branch carries the corpus's highest AI-over-trust risk applies with full force to the CODE, not only to the by-hand arithmetic.",
  "diy": "You must already be able to compute Pi = delta * Sigma @ w and the single-view posterior by hand for a 2-3 asset case (AA1.02 Sec 4.2-4.3) before delegating the general case to a tool -- otherwise you cannot tell a correct scaffold from a subtly broken one.",
  "market": "US"
}

7. Abandoning the return forecast altogether: risk parity in code

Black-Litterman still asks you to hold a view. Risk parity (AA1.02 §4.5–4.6) sidesteps the return-forecasting problem entirely: instead of choosing weights to hit a return target, choose weights that give every asset an equal percent contribution to portfolio risk. For exactly two assets, the closed form is elegant: w_1/w_2 = σ_2/σ_1 (the weight ratio is the inverse of the volatility ratio, independent of correlation). QD1.04-e07's risk_parity_weights starts from the naive inverse-volatility weights (exact for two assets) and, for three or more, runs the short iterative correction AA1.02 §4.6 specifies: compute each asset's percent contribution to risk, scale its weight by the square root of (target contribution ÷ actual contribution), renormalize, repeat. On Fieldstone University Endowment's Equity/Bond/Commodities menu (US), the naive inverse-volatility shortcut gives [19.5%, 58.5%, 22.0%] with risk contributions [37.1%, 30.0%, 32.9%], nowhere near the equal-thirds the shortcut is supposed to deliver, because the third asset's distinct pairwise correlations no longer cancel out of the two-asset algebra. The iterative solve converges to [17.8%, 60.4%, 21.7%], where every asset's contribution lands on exactly 33.3%. The identical exercise on a fresh India menu (Equity/Debt/Gold, reusing AA1.01's own Bharat vols and correlations, minus REIT) converges to [19.6%, 56.1%, 24.3%], again exactly equal-thirds, bonds/debt end up with the largest weight in both currencies, precisely because low-volatility, risk-diluting assets are what risk parity systematically favors over a cap-weighted book.

9. A closing word on data honesty in this module's own datasets

Every real-dataset exercise in this module (QD1.04-e08) draws on us-eod and nse-eod, both labeled kind: "synthetic" in app/public/datasets/MANIFEST.json with a seed and a provenance note: real-source fetches (Stooq-style for US, NSE/BSE-style for India) were attempted and blocked by this environment's network proxy, so both ship as seeded, honestly-labeled synthetic look-alikes rather than a silently-substituted real dataset. This is exactly the R3 discipline AA1.01/AA1.02 apply to their own illustrative capital-market-assumption tables (every number there is stated as explicitly synthetic, chosen to teach the mechanics honestly): a number's trustworthiness comes from knowing precisely what kind of number it is, real-and-cited or synthetic-and-seeded, never from assuming the more official-sounding label. QD1.04-e08's provenance chip in the Code tab shows exactly this label live, and the instability numbers you reproduce there are real properties of the shipped, seeded data, reproducible by anyone re-running the same seed, exactly as a real-money desk's own risk report should be.

10. The whole machine on one page

Eight sections of code have been eight separate objects so far. They are one object, and it is worth assembling before the worked examples, because a learner who can state the program in one line will never again ask a solver a question it was not built to answer.

10.1 The program, stated once

Every optimiser in this module solves the same shape of problem. Choose a weight vector w to make one scalar as large or as small as possible, subject to conditions w must satisfy.

maximise    (w'μ − Rf) / sqrt(w'Σw)          the tangency problem
minimise    w'Σw                              the GMV problem
subject to  1'w = 1                           fully invested (an equality)
            lo_i <= w_i <= hi_i               bounds (inequalities)
            other linear conditions           sector caps, turnover budgets

Three parts, and each one is a different kind of thing. The objective is the scalar being pushed; changing it changes the answer's meaning, not merely its value. The equality says the weights describe a whole portfolio; drop it and the solver will happily hand you a bounded vector summing to 0.83, which is the bug §4's exercise plants deliberately. The inequalities are the mandate: they never improve the objective and they always restrict the feasible set, which is the structural reason a constrained frontier lies inside the unconstrained one rather than beside it.

Two facts follow from the shape alone, before any data. The GMV objective contains no μ, so no revision to any expected return can move a GMV weight by a basis point. And the tangency objective contains μ in the numerator and Σ in the denominator, so it is homogeneous in neither: doubling every expected excess return leaves the tangency weights unchanged, while doubling every volatility does not. If those two sentences are not obvious, read the two formulas again rather than the next section, because everything below is a consequence of them.

10.2 Why the covariance matrix is the hard input

The intuition most learners bring is that μ is the hard input and Σ is bookkeeping. It is exactly backwards on the count that matters. An n-asset problem needs n expected returns and n(n+1)/2 distinct entries in Σ. At n = 4 that is 4 against 10. At n = 30 it is 30 against 465. At n = 100 it is 100 against 5,050. The covariance matrix is where the parameter count lives, and every one of those entries is estimated from the same finite history.

That count collides with a hard requirement. Σ must be positive semi-definite, meaning w'Σw >= 0 for every w, because that quantity is a variance and a negative variance is not a small error but an impossible object. A sample covariance matrix estimated from T observations on n assets has at most T − 1 non-zero eigenvalues, so with fewer observations than assets it is singular by construction, no matter how clean the data. The optimiser needs Σ⁻¹, which does not exist. Even above that threshold, at T merely comparable to n, the smallest eigenvalues are estimated with enormous relative error and the inverse blows them up: inversion divides by the eigenvalues, so the noisiest direction in your estimate becomes the loudest voice in your answer. That inversion step is the mechanism behind the phrase "estimation-error maximizer", and it is why §5's sweep and §8's window comparison find the same fragility from two different directions.

10.3 The four numerical conditions that produce nonsense

Four checks separate an answer from an artefact, and all four are cheap.

1. Non-PSD input. Correlations assembled from separate judgment calls need not be jointly possible. Test with numpy.linalg.eigvalsh(corr).min() >= -tol; §2's deliberately broken matrix returns −0.8. Bharat's real one returns eigenvalues 0.390, 0.797, 1.107, 1.707, all positive.
2. Ill-conditioning. Compute numpy.linalg.cond(Sigma). Bharat's is 17.1069, which is comfortable. A condition number in the thousands means the inverse is amplifying small input errors by roughly that factor, and a solved portfolio from such a matrix is a report on your estimation noise.
3. Too few observations. With T <= n the sample matrix is singular and the inverse is undefined; with T between n and roughly 10n it exists and is badly conditioned. §8's 22-day windows on five assets sit squarely in the second regime, which is why two adjacent windows disagreed by 25.8 percentage points on one weight.
4. A converged solver that answered the wrong question. scipy.optimize.minimize reports success on a sign-flipped objective, a missing sum-to-one constraint, and a numpy.cov call without rowvar=False. Convergence is not correctness. The only defence is a case whose answer you already know, which is what every hidden test in this module is.

10.4 The three fixes, compared on the same problem

Constraints, shrinkage and resampling all reduce how much the answer moves. They pay for it differently, and choosing between them is a judgement about which cost you can defend to whoever signs the mandate.

The rows are not alternatives to each other. A production stack usually runs all three, and the order matters: shrink Σ first, because everything downstream inverts it; resample second, because resampling a badly conditioned matrix simply reproduces the same bad conditioning ten thousand times; apply constraints last, because they are the mandate and the mandate is not negotiable by a statistician.

One question outranks all three and no technique answers it. If a 100 basis point revision to one expected return flips a weight's sign while the portfolio's own Sharpe ratio moves by 0.007, the honest report says the optimiser is indifferent across that range and the decision must be made on grounds the optimiser cannot see. Reporting the stability sweep alongside the weights is the deliverable. A weight vector with no sweep attached is a number pretending to be an answer.

Common mistakes & how experts think differently

1. Treating a single scipy.optimize call's output as ground truth rather than a point estimate that needs a stress test. QD1.04-e05 exists because a fluent, converged solver call looks identical whether its inputs are stable or on a knife's edge. Amateurs run the optimizer once and write down the answer; experts re-run it after perturbing one input by a realistic amount before writing anything down.

1. Building Black-Litterman's Σ in the wrong units. δ (typically 2.5–3.5) is calibrated to a decimal-return-units Σ; feeding it a percent-scale Σ (sigma=20 instead of 0.20) makes the reverse-optimized Π off by a factor of roughly 10,000 while returning no error at all, the code runs, the number is simply wrong. Always run the no-view identity check (§6) on a fresh Black-Litterman build before trusting anything past it.

1. Skipping the PSD check because "the correlations all looked reasonable individually." §2 and QD1.04-e01's hidden tests show a matrix that is pairwise plausible (0.9, 0.9, −0.9) and jointly impossible (a −0.8 eigenvalue). This is a thirty-second numpy.linalg.eigvalsh call; skipping it means every downstream number is garbage dressed as precision.

1. Extending the two-asset risk-parity shortcut (w ∝ 1/σ) to three or more assets. It is exact for two assets and visibly wrong for three or more (§7's naive [19.5%, 58.5%, 22.0%] gives contributions [37.1%, 30.0%, 32.9%], not equal thirds) once pairwise correlations stop canceling out of the algebra. The N-asset case needs the iterative solve, not the shortcut, and QD1.04-e07's structured fallback drills exactly this reorder-the-steps discipline.

1. Confusing "GMV doesn't depend on μ" with "GMV is immune to estimation noise." The formula never touches μ: true, and AA1.01 proves it. But QD1.04-e08 proves the covariance matrix GMV does depend on can itself be badly estimated from a short real sample, producing weight swings just as dramatic as anything §5 shows for tangency. "Formula-stable" and "estimate-stable" are different claims; conflating them is one of this module's most common misreadings.

1. Reaching for more shrinkage (or more views, or a tighter τ) as a free universal fix. §8's own honest result (full shrinkage collapsing GMV to an exactly equal-weighted portfolio regardless of the underlying data) shows shrinkage trades variance for bias, not eliminates variance for free. The expert's question is always "how much bias did I just introduce to buy this much stability," never "did the number stop moving."

1. Passing bounds to scipy.optimize.minimize without also passing the sum-to-1 equality constraint. QD1.04-e04's own choose-impl distractors include exactly this bug: SLSQP will happily return a bounded but non-normalized weight vector with no error, and a learner who only checks "are the bounds respected" (never "does it sum to 1") will ship it.

1. Flipping the sign of the optimization objective by accident. scipy.optimize.minimize always minimizes; maximizing Sharpe means minimizing its negative. Double-negate by mistake (minimize -sharpe(w) where sharpe itself already returns a negative) and the solver converges cleanly, reports success, and hands you the single worst investable portfolio on the menu, with no error message anywhere in sight, because nothing about a converged, successful optimization call tells you whether you asked it the right question.

1. Believing "AI wrote working optimizer code, so it must be correct." A neg_sharpe function with a subtly wrong sign, a numpy.cov call missing rowvar=False, or a GMV formula that inverts Σ instead of using Σ⁻¹ (this module's own tested wrong-option families, QD1.04-e04/e08) all run without error and return a plausible-looking weight vector that sums to one. The only defense is the same one AA1.02 names for the by-hand mathematics: run it against a case whose answer you already know (this module's own hidden tests exist for exactly this reason), never trust convergence or a lack of an exception as proof of correctness.

1. Treating this module's job as "re-derive AA1.01/AA1.02's math." It is not. Every theoretical claim in this module cites the AA1.01/AA1.02 section it implements; if you find yourself deriving a formula from scratch rather than looking it up and coding it, you are doing the wrong module's work, go verify the derivation there, then come back and implement it here.

Worked examples

Worked example 1: Building and validating a covariance matrix in code (India, ₹)

Given. Bharat Family Office's four-asset menu from AA1.01 §3: σ = (19%, 6%, 15%, 17%) for (Equity, Debt, Gold, REIT), with AA1.01's own correlation matrix.

Solution. Step 1: implement and run build_covariance. Sigma[i][j] = sigma[i] * sigma[j] * corr[i][j] for every pair, giving Σ(%²) = [[361.00, 11.40, -14.25, 177.65], [11.40, 36.00, 13.50, 35.70], [-14.25, 13.50, 225.00, 12.75], [177.65, 35.70, 12.75, 289.00]]: matching AA1.01 Worked Example 1's hand-built matrix to the decimal. Step 2: implement and run is_psd_correlation. numpy.linalg.eigvalsh on the underlying correlation matrix returns [0.390, 0.797, 1.107, 1.707] (rounded), every eigenvalue strictly positive, so the function returns True, matching AA1.01 §3's own check. Step 3, prove the check actually catches a bad input. Feed is_psd_correlation a deliberately inconsistent 3×3 matrix: pairwise correlations 0.9, 0.9, and −0.9 among three assets, and its smallest eigenvalue comes out −0.8, so the function correctly returns False. Read it. A function that only ever returns True on the inputs you expected it to is not tested; QD1.04-e01's hidden tests exist specifically to prove your is_psd_correlation rejects a genuinely bad matrix, not merely accepts a good one.

Worked example 2: GMV's first-order condition, verified by code (India, ₹)

Given. Σ from Worked Example 1; μ = (12.0%, 7.0%, 7.5%, 9.0%).

Solution. Step 1. gmv_weights(Sigma) returns [9.32%, 86.94%, 9.73%, −5.99%]: AA1.01 Worked Example 2's numbers exactly. Step 2. Compute Sigma @ w in the same script: [31.54, 31.54, 31.54, 31.54] (%², to two decimals) in every entry, the first-order condition (every asset's marginal contribution to variance is identical at the true minimum) holds to machine precision, confirmed by code rather than merely asserted by AA1.01's prose. Step 3, the fact worth testing directly. Re-run gmv_weights on the identical Σ after arbitrarily changing μ to (20.0, 7.0, 7.5, 9.0) (Equity's assumed return raised from 12% to 20%): the returned weights are bit-for-bit identical, because the function's signature does not even accept μ as an argument. Read it. This is the cleanest possible code-level proof that not every MVO output is fragile, only the μ-dependent ones are (§5), and QD1.04-e08 is where you discover that even this "stable" landmark is only stable in its formula, not necessarily in its estimate.

Worked example 3: Constrained tangency: pricing a real cap in code (India, ₹)

Given. Σ and μ as above; Rf = 4.5%; a mandate imposing long-only plus a 50% single-asset cap.

Solution. Step 1, the unconstrained baseline. tangency_weights(Sigma, mu, 4.5) returns [24.51%, 70.60%, 12.26%, −7.37%], Sharpe 0.578 (AA1.01 Worked Example 3 Step 1). Step 2: solve the constrained version. constrained_tangency_weights(Sigma, mu, 4.5, [(0, 0.5)] * 4) calls scipy.optimize.minimize with method="SLSQP", the sum-to-1 equality constraint, and the caller's own bounds, returning [29.37%, 50.00%, 20.63%, 0.00%], Sharpe 0.5535: matching AA1.01 Worked Example 3 Step 3 to the decimal. Step 3, verify the cap actually binds, not merely coincides. Check w[1] <= 0.5 + 1e-6 in code (QD1.04-e04's own hidden test does exactly this), the 50.00% is not a coincidence of the unconstrained solution happening to land there; the unconstrained Debt weight was 70.60%, and the constraint is what pushed it down. Step 4: quantify the price. (0.578 - 0.5535) / 0.578 ≈ 4.2%, the exact relative Sharpe cost AA1.01 names, now something you computed yourself rather than read. Read it. A constraint's Sharpe cost, computed this way for any real mandate rule, is a genuinely useful diagnostic for which policy constraints are expensive and which are nearly free, and it only exists because you have code that can re-solve the unconstrained and constrained cases on demand.

Worked example 4: The estimation-error maximizer, proven by a sensitivity sweep (US, $)

Given. Northfield Endowment's three-asset menu (Equity 8.5%/17%, Bonds 4.8%/6%, Alts 8.0%/14%, correlations Equity-Bonds −0.10, Equity-Alts 0.55, Bonds-Alts 0.05), Rf = 4.2%.

Solution. Step 1, the baseline. tangency_weights_sensitivity(Sigma, mu, 4.2, 1, 0.0) (asset index 1 = Bonds, delta 0) reproduces the unperturbed tangency portfolio: Equity 24.57%, Bonds 44.71%, Alts 30.72%. Step 2: apply a modest, realistic revision. Call the same function with delta=-0.80, cutting Bonds' assumed return by 80 basis points, from 4.8% to 4.0%. The function returns Equity 48.17%, Bonds −27.32%, Alts 79.15%. Step 3: name what just happened. Bonds flipped from a substantial long position (+44.71%) to an outright short (−27.32%) (a 72-point swing from an 80-basis-point input revision smaller than many desks' routine year-to-year capital-market-assumption updates) while Equity and Alts both surged to absorb the freed-up capital. Read it. This is the same code path as Worked Example 3's constrained solve, run unconstrained and swept across one input instead of solved once, the exact same function (tangency_weights, generalized by tangency_weights_sensitivity's one-line perturbation) that gave you a defensible-looking answer in Worked Example 2/3 is capable of handing you a completely different, equally "optimal"-looking answer from a change well within ordinary estimation noise. That capability, not a bug, is what "estimation-error maximizer" names.

Worked example 5: Black-Litterman versus naive history, the same code, two input vectors (India, ₹)

Given. Ananta Family Office's three-asset menu (Equity/Bonds/Gold at 55%/35%/10%, σ = 20%/6%/14%, δ = 3.0, τ = 0.05); a stated view that equity beats bonds by 8 points.

Solution. Step 1: reverse-optimize the equilibrium. black_litterman_posterior's first internal step, Pi = delta * Sigma @ w_mkt, returns [6.768%, 0.614%, 0.951%], verify the no-view identity Sigma^-1 @ Pi == delta * w_mkt ([1.65, 1.05, 0.30]) before trusting anything further. Step 2: blend the view. With P = [1, -1, 0] and Q = 8%, the posterior comes out [7.637%, 0.560%, 0.954%], a modest tilt, because 8% is only slightly more bullish than the 6.15-point equilibrium spread already implies. Step 3, the payoff. Feed both the naive trailing-3-year mean vector (Equity 15%, Bonds 3%, Gold −1%, an ordinary bad stretch for gold) and the module's Black-Litterman posterior to the identical tangency_weights function, with rf=0 since both vectors are already excess returns:

The naive vector recommends shorting gold and abandoning the strategic allocation entirely, one unlucky trailing window, run through the same code as Worked Example 2/3, producing a corner solution no committee would approve. The Black-Litterman vector leaves gold at exactly 10.0%, unchanged to the decimal, because the view never named it (Sigma^-1 @ (posterior - Pi) comes out [+0.00224, -0.00224, 0.0]: gold's raw-weight shift is zero). Read it. This is Black-Litterman's entire case, reproduced with one function call each: the same optimizer, the same three assets, and the only thing that changed between a portfolio no one would sign off on and a defensible, explainable tilt is which return vector fed it.

Worked example 6: Risk parity and covariance-estimation instability, side by side (US, $)

Given. Fieldstone University Endowment's Equity/Bond/Commodities menu (σ = 18%/6%/16%, correlations Equity-Bond 0.10, Equity-Commodities 0.20, Bond-Commodities −0.05); separately, five real US tickers from us-eod over two different 22-trading-day windows.

Solution. Step 1: risk parity. The naive inverse-volatility shortcut gives [19.5%, 58.5%, 22.0%]; compute each asset's percent contribution to risk from these weights and you get [37.1%, 30.0%, 32.9%], visibly unequal, because Commodities' distinct pairwise correlations no longer cancel out of the two-asset algebra. risk_parity_weights iterates from this starting point and converges to [17.8%, 60.4%, 21.7%], where every asset's contribution lands on exactly 33.3% (verified directly in code, not merely asserted). Step 2, covariance-estimation instability, a completely different demonstration on completely different data. Pull five real us-eod tickers' daily returns; take trading days 0–21 as window 1 and days 22–43 as window 2. small_sample_gmv(window, shrink=False) returns GMV weights that differ by 10.3 percentage points at their most-different entry between the two windows, real sampling noise in a genuinely short sample. small_sample_gmv(window, shrink=True) (Ledoit-Wolf) returns weights that are identical to four decimal places between the same two windows. Read it. Two entirely separate lessons live in one worked example on purpose: risk parity abandons the return forecast to fix §5's problem; shrinkage re-estimates the covariance matrix itself to fix a different problem, and a professional build checks both, because "the optimizer blew up" can mean either one, and the fix for one does nothing for the other.

Practice set

Work each problem fully before reading its solution. Where a computation appears, numpy/scipy are assumed available (this program's runtime); numeric answers are to about ±1% unless stated otherwise.

P1 (guided). Build a 2×2 Σ by hand, then check it in code. σ = (25%, 10%), ρ = 0.20. Compute Σ, then state what build_covariance([25, 10], [[1, 0.2], [0.2, 1]]) must return.

Solution. Σ_11 = 25² = 625, Σ_22 = 10² = 100, Σ_12 = Σ_21 = 25 × 10 × 0.20 = 50. The function must return [[625, 50], [50, 100]] (all in %²).

P2 (guided). GMV for the same 2-asset menu. Using w_gmv = Σ⁻¹1/(1′Σ⁻¹1), compute the GMV weights for P1's Σ, and verify by hand that the two-asset GMV weight formula collapses to w_1 = (σ_2² - ρσ_1σ_2)/(σ_1² + σ_2² - 2ρσ_1σ_2).

Solution. Plugging in: numerator = 100 - 0.20×25×10 = 100 - 50 = 50; denominator = 625 + 100 - 2×50 = 625 ; w_1 = 50/625 = 8.0%, w_2 = 92.0%. The lower-volatility asset dominates, exactly as AA1.01 §4's Bharat example showed for Debt.

P3. PSD check by determinant, for a 2×2 case. For a 2×2 correlation matrix [[1, ρ], [ρ, 1]], state the condition on ρ for the matrix to be PSD, and check ρ = 1.2 (a typo'd correlation greater than 1).

Solution. Eigenvalues of [[1, ρ], [ρ, 1]] are 1 + ρ and 1 - ρ; both non-negative requires -1 <= ρ <= 1. At ρ = 1.2, 1 - ρ = -0.2 < 0, not PSD, and indeed ρ = 1.2 is not even a valid correlation coefficient in the first place (correlations are bounded in [-1, 1] by definition): is_psd_correlation must catch this.

P4. Unconstrained tangency for a 2-asset case. Σ = [[0.04, 0.0012], [0.0012, 0.0036]] (decimal units), μ = (10%, 5%), Rf = 2%. Compute the tangency weights.

Solution. excess = [0.08, 0.03]. The determinant is 0.04 × 0.0036 − 0.0012² = 0.00014256, so the standard 2×2 inverse formula gives Σ⁻¹ = [[0.0036, −0.0012], [−0.0012, 0.04]] / 0.00014256 = [[25.2525, −8.4175], [−8.4175, 280.5836]]; raw = Σ⁻¹ @ excess = [1.7677, 7.7441]; normalizing, w = [18.58%, 81.42%]. Watch the off-diagonal: it is roughly −8.4, not roughly −0.008. Dividing the off-diagonal cofactor by 1,000 too many (a units slip, decimals against percents, in the determinant) leaves the diagonal almost untouched and still lands you near [19.7%, 80.3%], which is 1.1 points from the right answer and outside this module's own weight tolerance. (If you implemented tangency_weights already, running it directly is faster and exact, this by-hand route is the audit, not the primary method.)

P5. Quantify a constraint's Sharpe cost. Bharat's unconstrained Sharpe is 0.578; long-only-only gives 0.571; long-only+50%-cap gives 0.5535. Compute both relative costs.

Solution. Long-only: (0.578 - 0.571)/0.578 ≈ 1.2%. Long-only+cap: (0.578 - 0.5535)/0.578 ≈ 4.2%. The cap costs roughly 3.5x more than the long-only constraint alone, because it binds on Debt, the single asset doing the most Sharpe-maximizing work in the unconstrained solution.

P6. Why does the Sharpe ratio barely move across QD1.04-e05's REIT sweep? REIT's tangency weight moves from −7.37% to −1.20% to +0.07% to +5.24% as its assumed return rises from 9.00% to 11.00%, while Sharpe moves only from 0.578 to 0.5710 to 0.5710 to 0.5743. Explain in mechanism terms.

Solution. Near the unconstrained optimum, the objective surface (the Sharpe ratio as a function of weights) is comparatively flat, many genuinely different weight vectors are almost equally "optimal" in the sense of achieving nearly the same Sharpe ratio, even though their compositions differ enormously. This is exactly AA1.01's "invisible-probability" lesson transplanted to optimization: a near-flat objective near the optimum means the specific answer you got can sit on a knife's edge (small input change, big weight change) while looking, by its own scorecard, just as good as its neighbors.

P7. Black-Litterman units check. A learner builds Σ with sigma = [20, 6] (percent-scale, not decimal) and δ = 3.0, then wonders why Π comes out as [734.4, 64.8] instead of [7.34, 0.65]. Diagnose.

Solution. Π = δ · Σ · w; with percent-scale σ, Σ entries are on the order of hundreds (Σ_11 = 20×20×1 = 400) instead of a decimal-scale 0.04, so Π comes out roughly 10,000x too large (the mismatch compounds because Σ enters as σ_i × σ_j, a squared effect). δ = 3.0 is calibrated for a decimal-scale Σ; the fix is to build Σ from sigma = [0.20, 0.06], not [20, 6], for this function specifically (Common Mistakes item 2).

P8. Black-Litterman single-view tilt, by hand. 60/40 Equity/Bond menu, Σ (decimal) = [[0.04, 0.0012],[0.0012,0.0036]], δ=3.0, w_mkt=[0.6,0.4], P=[1,-1], Q=10%. Compute Π and the tilt.

Solution. Σw = [0.04×0.6+0.0012×0.4, 0.0012×0.6+0.0036×0.4] = [0.02448, 0.00216]; Π = 3×[0.02448,0.00216] = [0.07344, 0.00648] (7.344%, 0.648%). PΣ = [0.0388, -0.0024], PΣP' = 0.0412, PΠ = 6.696%, surprise = 10% - 6.696% = 3.304pp; tilt = 0.5 × [0.0388,-0.0024]/0.0412 × 0.03304 ≈ [+1.556pp, -0.096pp]; posterior ≈ [8.90%, 0.55%].

P9. Risk parity, 2-asset closed form. σ = (27%, 9%), zero correlation. Compute the risk-parity weights.

Solution. w_1/w_2 = σ_2/σ_1 = 9/27 = 1/3; normalizing, w_1 = 25%, w_2 = 75%. (This is the same ratio structure as AA1.02's own Fieldstone 60/40 example, σ=18%/6% giving [25%,75%], the pattern is identical whenever one volatility is exactly 3x the other.)

P10. Why does the naive inverse-volatility shortcut fail at three assets but not two? Explain the algebraic reason, not just "it's wrong."

Solution. For exactly two assets, setting the two components' contributions equal and simplifying causes every correlation term to cancel out algebraically, leaving the clean ratio w_1/w_2 = σ_2/σ_1 regardless of ρ. With three or more assets, each asset's risk contribution depends on its covariance with every other asset, and those cross-terms no longer cancel in general, so an allocation that equalizes contribution for one pair can leave a third asset's contribution systematically off, which is exactly why Worked Example 6's naive shortcut gives [37.1%, 30.0%, 32.9%] instead of equal thirds.

P11. Shrinkage is a bias-variance tradeoff, not a free lunch: argue both sides. State one honest cost of using Ledoit-Wolf shrinkage even though it stabilized GMV in Worked Example 6.

Solution. Variance side (the win): a raw sample covariance from a short window is noisy, and that noise propagates directly into wildly different GMV weights; shrinkage reduces this noise by pulling the estimate toward a more structured target. Bias side (the honest cost): pulling every estimate toward a common structured target can systematically under-represent real, persistent differences between assets, in the extreme (as Worked Example 6's US case showed), heavy shrinkage collapsed GMV to an exactly equal-weighted portfolio regardless of the underlying data, which is only correct if the assets genuinely are that similar. A professional treats the shrunk estimate as a answer to compare against the raw one, not the answer by default.

P12. Dataset provenance reasoning. This module's us-eod and nse-eod datasets are both labeled kind: "synthetic" in the manifest. Does this make QD1.04-e08's instability demonstration less valid? Explain.

Solution. No. The demonstration's claim is about a mechanism, how sensitive a raw-sample-covariance GMV estimate is to which short window of data it sees, versus a shrinkage estimate, and that mechanism is a property of the estimator, reproducible on any sufficiently short, sufficiently noisy real-shaped return series, synthetic or not. What would be invalid is presenting a number computed from this seeded synthetic data as if it were a specific real company's actual historical return, which this module never does; every ticker in us-eod/nse-eod is explicitly fictional (per MANIFEST.json's own provenance note), and the lesson is about the optimizer's behavior, not about any real market history.

P13. Why does adding one bound turn a closed-form problem into a numerical one? tangency_weights (QD1.04-e03) is four lines of linear algebra with no loop. constrained_tangency_weights (QD1.04-e04) needs scipy.optimize.minimize. What changed?

Solution. The unconstrained tangency problem's only constraint is a single linear equality (sum(w) = 1), and maximizing a ratio of quadratic forms subject to one linear equality has a known closed-form solution via the Lagrangian, which is exactly the Σ⁻¹(μ − Rf) formula. The moment you add an inequality constraint (w_i >= 0, or w_i <= cap), the solution may sit exactly on that boundary (as Debt did at 50.00% in Worked Example 3) rather than at the unconstrained interior optimum, and which constraints will bind is not known in advance, this is precisely the class of problem (a quadratic or convex program with inequality constraints) that requires an iterative numerical solver rather than a single algebraic formula.

P14. Name the production tool this module's runtime cannot execute, and why the substitute is not a downgrade.

Solution. cvxpy, a convex-optimization modeling DSL, is not available in this program's Pyodide runtime. The substitute here, closed forms wherever they exist, scipy.optimize.minimize (SLSQP) otherwise, is not a conceptual downgrade: every optimization problem this module poses (a handful of assets, a handful of constraints) is exactly the class of problem SLSQP solves reliably, and the underlying mathematics (Lagrangians, KKT conditions, quadratic objectives) is identical to what cvxpy would set up under the hood. cvxpy's real advantage is at production scale (hundreds of names, many simultaneous linear constraints, specialized fast solvers) which is a software-engineering advantage (a cleaner problem-specification DSL, faster large-scale solvers), not evidence that this module's smaller-scale, hand-auditable approach teaches the wrong thing.

P15. Spot the bug: a choose-impl reasoning drill. A submitted constrained_tangency_weights passes every visible test but fails hidden tests that use a tighter per-asset cap. Its objective function is def neg_sharpe(w): er = w @ m - rf; sig = np.sqrt(w @ S @ w); return -er / sig, called via minimize(lambda w: -neg_sharpe(w), w0, bounds=bounds, constraints=cons). Find the bug.

Solution. neg_sharpe already returns the negative Sharpe ratio; wrapping it in lambda w: -neg_sharpe(w) negates it a second time, so minimize is actually minimizing the positive Sharpe ratio, i.e., finding the worst, not the best, investable portfolio on the menu. It will still converge, still report success, still return weights that sum to 1 and respect the bounds, nothing about the mechanics signals the error, which is exactly why QD1.04-e04's own choose-impl structured fallback drills this precise bug as one of its four options (Common Mistakes item 8).

P16. Why does R6 (India + US hidden tests) apply to QD1.04-e08 but not to QD1.04-e01 through e07?

Solution. R6 (this program's rule that any market-data-touching exercise must carry hidden tests over both an India and a US dataset) is keyed off whether an exercise actually references one of the program's shipped dataset ids (us-eod, nse-eod, and similarly paired ids). QD1.04-e01 through e07 use explicitly synthetic, illustrative capital-market-assumption tables (Bharat, Northfield, Ananta, Fieldstone) that are not tied to any bundled dataset file, they are India- or US-flavored by the CMA menu's own currency and market convention, but mechanically they do not reference a dataset id, so R6's specific gate does not apply to them (though each still carries at least one India-flavored and one US-flavored hidden test, by this module's own choice, for pedagogical parity). QD1.04-e08 is the one exercise that reads real bytes from us-eod/nse-eod, so it is the one exercise the validator's R6 check actually enforces.

P17. Read a resampling run, then say what it did and did not buy. §8's ten Michaud resamples of Bharat's menu return REIT weights ranging from −34.14% to +13.82%, with a standard deviation across the ten draws of 16.48 percentage points, and an average of −8.11% against the point estimate's −7.37%. The resampled average scores a Sharpe ratio of 0.5712 on the true inputs; the point estimate scores 0.5780 and the ten individual draws average 0.5280. State (a) what the technique bought, with the number, (b) what it did not fix, and (c) the one change to the procedure that would make the ten draws disagree less.

Solution. (a) It bought stability against sampling noise, and the honest way to price that is against the realistic alternative rather than against the truth. Optimising once on one 60-month sample earns an expected 0.5280; averaging ten of them earns 0.5712, a gain of 8.2%. Measured instead against a point estimate that already knew μ and Σ exactly, resampling costs 0.7% (0.5780 to 0.5712), which is the comparison a critic will reach for and the one that is unavailable in practice, because nobody has the true inputs. (b) It did not fix the direction of the call: the REIT short survives at −8.11%, barely changed from −7.37%, so a learner hoping resampling would dissolve an uncomfortable position is asking it to correct a bias with a variance tool. It also cannot repair a Σ estimated from too few observations, since every draw inherits the same shortage, and it leaves the resampled portfolio off the efficient frontier, defensible on stability grounds and on nothing else. (c) Increase T. The draws disagree because 60 observations is thin for four assets with ten distinct covariance entries; drawing 240 observations instead shrinks the sampling error in each μ̂ by a factor of two and narrows the weight distribution accordingly. Note what that implies, and say it out loud when you report a resampling run: the technique's output is a statement about how much evidence you own, so the honest way to report it is the spread across draws next to the average, never the average alone.

Applied mini-project

Build and stress-test a small real policy portfolio, then write the honest memo.

This project assembles every function this module's code pack ships (build_covariance, is_psd_correlation, gmv_weights, tangency_weights, constrained_tangency_weights, tangency_weights_sensitivity, black_litterman_posterior, risk_parity_weights, small_sample_gmv) into one realistic decision, using code-packs/QD1.04.json (module id QD1.04) as your toolkit. It is a self-scored worksheet: the code pack's own exercises are what the mastery gate grades; this project is where you assemble them into a single, honest mini-decision memo.

Do this, in order:

1. Choose a menu. Pick 4–6 assets from either us-eod or nse-eod (or a mix, if you build a cross-market menu, note the honesty caveat about combining currencies without adjustment, AA1.01 Common Mistakes item 9). State your seed/snapshot date (R3).
2. Build and validate Σ. Estimate volatilities and a correlation matrix from your chosen menu's real return history (any window ≥60 trading days). Run your is_psd_correlation check and report the result, if it fails, say what you did about it (a common real fix: shrink or nudge the correlation matrix toward the nearest valid one, and note that you did so).
3. Solve four portfolios. Report the weights, expected return, volatility, and Sharpe ratio for: (a) the unconstrained tangency portfolio; (b) a long-only tangency portfolio; (c) a long-only tangency portfolio with a realistic single-asset cap you choose and justify (state why that cap, not a generic one); (d) the risk-parity portfolio for the same menu.
4. Run the sensitivity stress test. Pick one asset and one plausible ±50–150bp revision to its assumed return; report how much the unconstrained tangency weight for that asset moves, and whether any weight's sign flips.
5. Write the memo. In 200–350 words: which of your four portfolios (from step 3) would you actually recommend, and why, grounded in the Sharpe-cost-of-constraint arithmetic (step 3) and the sensitivity result (step 4), not in generic language. State explicitly whether your recommendation would survive the input revision from step 4, and if not, say so.

Deliverable and self-check. One worksheet: your menu and seed/snapshot date; the PSD check result and what you did if it failed; all four portfolios' weights/return/vol/Sharpe; the sensitivity result (asset, delta, weight movement, sign flip or not); the 200–350 word memo. Score yourself 2 points each for: (a) the PSD check genuinely ran against your own estimated correlation matrix, not a toy example; (b) all four portfolios are reported with correct, internally consistent Sharpe arithmetic; (c) the sensitivity test uses a defensible, stated delta and correctly reports whether a sign flip occurred; (d) the memo's recommendation is directly justified by your own numbers from steps 3–4, not generic claims; (e) the memo explicitly states whether the recommendation survives the stress test, including admitting if it does not. Pass at 8/10 with no zero on any item.

Lab: open the QD1.04 code pack in this node's Code tab (code-packs/QD1.04.json) and clear all eight exercises (QD1.04-e01 through QD1.04-e08), each ships a structured fallback, so the lab stays completable even fully offline, with zero runtime.

Reading & resources

Everything below is an anchor to verify claims against, never a substitute for hand-tracing AA1.01/AA1.02's derivations or running this module's own code pack. Free unless marked; level tags reflect the ramp from intuition to mechanism.

The direct theoretical sources (read these first, if you have not already):

• AA1.01 (this program, Finance-Plus Ring), the full mean-variance derivation, the GMV/tangency closed forms, and the input-sensitivity demonstration this module implements in QD1.04-e01 through e05. [Free ✓ within this program] [Beginner–Intermediate ▲]
• AA1.02 (this program, Finance-Plus Ring). Black-Litterman, risk budgeting/parity, and the sanity-range table this module's own numbers were checked against. [Free ✓ within this program] [Intermediate ▲]
• Harry Markowitz, "Portfolio Selection" (Journal of Finance, 1952), the founding paper; short, and worth reading once in the original. [Free ✓ (widely available)] [Intermediate ▲]
• Fischer Black and Robert Litterman, "Global Portfolio Optimization" (Financial Analysts Journal, 1992), the original Black-Litterman paper. [Paid $ (journal access) / summarized freely in many secondary sources] [Advanced ▼]

The estimation-error and robustness literature:

• Philippe Jorion, "Bayesian and CAPM Estimators of the Means: Implications for Portfolio Selection" and related work; Michael J. Best and Robert R. Grauer, "On the Sensitivity of Mean-Variance-Efficient Portfolios to Changes in Asset Means" (1991), the paper that names the "estimation-error maximizer" critique QD1.04-e05 reproduces numerically. [Paid $ (journal access)] [Advanced ▼]
• Richard O. Michaud, "The Markowitz Optimization Enigma: Is 'Optimized' Optimal?" (1989), the resampling-based robustness response named in §8. [Paid $ (journal access) / widely summarized] [Advanced ▼]
• Olivier Ledoit and Michael Wolf, "Honey, I Shrunk the Sample Covariance Matrix" (2004) and related papers, the shrinkage estimator QD1.04-e08 uses via sklearn.covariance.LedoitWolf. [Free ✓ (widely available preprints)] [Advanced ▼]
• Richard Grinold and Ronald Kahn, Active Portfolio Management, the institutional-quant standard reference tying signal construction (QD1.02) to portfolio construction (this module). [Paid $] [Advanced ▼]

Tools and documentation (this module's actual runtime):

• scipy.optimize.minimize official documentation, especially the SLSQP method, the solver every constrained exercise in this module uses. [Free ✓] [Intermediate ▲]
• numpy.linalg documentation (inv, eigvalsh), the linear-algebra primitives every closed-form exercise in this module uses. [Free ✓] [Beginner ▲]
• sklearn.covariance.LedoitWolf documentation, the shrinkage estimator QD1.04-e08 uses. [Free ✓] [Intermediate ▲]
• The PyPortfolioOpt package's own documentation (also named in AA1.02's Modern Data Analysis callout), a tested, open-source implementation of mean-variance, Black-Litterman, and risk-parity, worth reading after you have built your own, to see a production-grade version of the identical mathematics. [Free ✓] [Intermediate–Advanced ▲▼]
• cvxpy's own documentation, worth reading precisely because this module's runtime cannot execute it; understanding what a dedicated convex-optimization DSL adds at production scale (§ opening runtime note) is part of this module's honesty about its own limits. [Free ✓] [Advanced ▼]

Do this, not just read: on your own machine (not this program's in-browser runner), install cvxpy and re-solve QD1.04-e04's constrained tangency problem with it instead of scipy.optimize. Confirm you get the same answer (within numerical tolerance) two structurally different ways, the exact discipline AA1.01/AA1.02 model throughout: never trust a single computational route until you have checked it against an independent one.

Teach it back & journal

Feynman prompt. Explain to a colleague who has only ever clicked "solve" in a spreadsheet Solver why the sentence "the optimizer said 24.51% equity" should never be written down without also answering "and what does it say if I nudge one assumption by a percentage point?" Use QD1.04-e05's exact result (REIT's tangency weight flipping sign on a 120-basis-point revision while the Sharpe ratio barely moves) to make the point concrete, then explain why the SAME caution applies to a completely different part of the pipeline: even the GMV portfolio, whose formula never touches an expected-return assumption at all, can still swing wildly if the covariance matrix feeding it was estimated from too short a real sample (QD1.04-e08). If your colleague can, unprompted, state that "the optimizer being confident" and "the optimizer being right" are two different claims, and explain why both are threatened the moment inputs come from real, finite, noisy data, you have installed the mechanism this module exists to teach.

Journal prompt. Over one working exercise (your own portfolio, a class project, or the mini-project above), build a portfolio with this module's code, then deliberately run two stress tests on it: (1) nudge one expected-return assumption by 50–150 basis points and record whether any weight's sign flips; (2) if you have real return data, re-estimate the covariance matrix from two different short windows and record how much the GMV weights move. Write a paragraph on which of the two failure modes was larger for your specific menu, and why you think that is (a concentrated, small menu versus a diversified one; a short window versus a longer one). Close with one sentence on this: an optimizer's own reported success: "converged," "feasible," "Sharpe = 0.58", carries zero information about whether the specific answer it handed you would survive a defensible change to one input, and the only way to know is to have run the two stress tests yourself, on your own code, rather than trusting a single call's output.

This module's flashcards and mastery quiz are wired into the app: see the node's Quiz and Reviews.