Returns, Statistical Measures and the Discipline of Inference

Finance-Plus Ring · Branch QM (Quantitative Methods), Node QM1.04 · ~18 focused hours · Mastery gate ≥ 85%

A fund returned +40%, then −30%, then +40%, then −30%. Its factsheet prints an average annual return of 5.0%, and that number is correctly computed. Ten lakh rupees left alone through those four years is worth ₹9,60,400 at the end.

Both statements are true at the same time. The 5.0% is an arithmetic mean, and an arithmetic mean of returns answers a question about a single future draw, not about four years that have already happened. The number describing what happened to the money is the geometric mean, −1.005% a year. The six-point gap between them is nobody's error. It is a property of volatility, and it is calculable in advance from the variance alone.

That gap is the first of a dozen places where a return statistic answers a question its reader did not ask. The second half of the work is worse, because it involves counting. A screen of 400 candidate factors hands you a winner with a t-statistic of 2.19 and a two-sided p-value of 0.028, clearing every conventional bar. Deflated honestly for the 400 attempts that produced it, that winner sits below what the best of 400 coin flips would have shown, and you would need eighteen and a half years of monthly data before a Sharpe ratio that size could mean anything.

Two skills sit inside those two paragraphs and neither is hard. Compute the right statistic for the question in front of you, and then account honestly for how many questions you asked before you liked the answer. The first is arithmetic. The second is a habit, and the habit is rarer.

This Ring is additive institutional breadth. It gives you the statistical grammar in which returns are stated, compared and defended, alongside the programme's own view that a result is evidence about a process rather than a substitute for one. That view survives intact. What follows adds the tools to interrogate a number somebody else computed, which is most of what an analyst actually does with statistics.

Despite the node number, the first eleven sections come before the probability and distributions work in study order. Take them first, then read the probability branch, then come back for sections 12 to 18, which lean on regression. The order of the branch is not the order of the shelf.

By the end you will have computed a holding period return four ways and priced the gap between arithmetic and geometric means. You will have run a bootstrap and a jackknife against one Sharpe ratio and got the same bias estimate twice, filled an ANOVA table and checked its F two ways, and deflated a backtest for the number of trials that produced it.

Budget roughly six hours on the core sections, three on the worked examples, four on the practice set, three on the mini-project, and two on the cards and the mastery check.

Learning objectives

You can:

1. Compute a holding period return from a price change and its income, chain HPRs across sub-periods, and say why chaining is multiplicative rather than additive.
2. Choose among the arithmetic, geometric, weighted and harmonic means by naming the question each one answers, and quantify the arithmetic-minus-geometric gap from the variance alone.
3. Compute a money-weighted return as an internal rate of return on dated cash flows, compute the Modified Dietz approximation to it, and state the conditions under which the approximation breaks.
4. Annualise a return of any period length correctly, show why prorating understates a sub-year return and overstates a multi-year one, refuse to annualise a period shorter than a year, and convert between simple and continuously compounded returns.
5. Separate the five subtractions that stand between a headline return and money a person keeps: fees, taxes, inflation, leverage and the gross-versus-net basis, and price each one.
6. Compute the full descriptive set by hand on a return series, from the median and the quartiles through the sample standard deviation, downside deviation and coefficient of variation to sample skewness and excess kurtosis, and show numerically how leptokurtosis makes a normal-based VaR understate its own promised loss frequency.
7. Run a bootstrap and a jackknife, build a percentile interval that a t-interval cannot honestly give, and name and detect the five sampling biases with a specific fix for each.
8. Fill an ANOVA table for a regression, compute F two independent ways, encode categories as dummies without falling into the dummy trap, and build a prediction interval that widens correctly away from the sample mean.
9. Fit and read a logistic regression, from the log-odds link and the odds ratio to the likelihood-ratio test and why McFadden's pseudo-R-squared must never be read on an ordinary R-squared scale, and compute Spearman rank correlation by hand with ties.
10. Correct a screen for the number of trials that produced it (Bonferroni, Holm, Sidak, Benjamini-Hochberg), deflate the winner's Sharpe ratio against the expected maximum from N attempts, derive the minimum backtest length that Sharpe implies, and (productivity objective: R10) reproduce all of it in Python while keeping the verification no tool performs.

The duality, stated once (R10). The gated skill is the arithmetic: a skewness computed from a column of twenty numbers, a percentile interval read off a sorted vector, a BH threshold ladder checked by eye. The productivity payoff is doing all of it in fifteen lines over a thousand series. The risk sits in between. A statistic is a single number with no internal evidence of its own correctness, so a wrong sample standard deviation and a right one look identical on the page, and the only defence is a second route to the same figure. Every worked example below carries one.

Prerequisites & connections

Builds on. M0.03 supplies the arithmetic of percentages and compounding, used here without re-teaching. M3.01 supplies present value and the internal-rate-of-return solve; the money-weighted return below is an XIRR and nothing more. QM1.01 supplies probability, the normal and lognormal distributions, the Central Limit Theorem and the confidence-interval logic that the resampling section extends; take the last seven sections only after it. QM1.02 supplies the regression build, the four diagnostics and the omitted-variable formula.

Feeds forward. QM1.05 takes the volatility models and the penalised regressions that the model-selection discipline here is a prerequisite for. QD1.02 builds factors and consumes the rank correlation taught here as its rank information coefficient. QD1.03 builds the backtest engine and consumes the multiple-testing and Sharpe-deflation work wholesale; its cross-link table already points here. QD2.03 implements the deflated Sharpe ratio in code against the theory derived here. AA1.03 takes the return definitions into performance measurement proper. DA1.05 is where all of it becomes vectorised.

Several neighbouring nodes own material that is deliberately not re-taught here (R1). QM1.01 owns probability, distributions and the construction of a confidence interval from a sampling distribution; the resampling section assumes all of it and adds only the alternatives. QM1.02 owns hypothesis testing, the regression build, heteroskedasticity, serial correlation, multicollinearity and misspecification; the ANOVA, dummy, interval and logistic sections read the output of a regression and never re-derive the fit. AA1.03 owns the time-weighted-versus-money-weighted fork, its worked example, and the whole risk-adjusted measure family (Sharpe, Treynor, Information Ratio, M-squared, Jensen); what follows borrows the money-weighted definition only, to make the Modified Dietz approximation error visible, and treats the Sharpe ratio purely as an estimator with a sampling distribution. M3.06 owns harmonic-mean aggregation of valuation multiples; the harmonic mean appears here for a different job, averaging purchase prices. QD1.03 owns the backtest engine, QD2.02 owns production VaR systems, and QD1.02 owns factor construction. Cross-link, never restate.

Core teaching content

1. What a holding period return actually measures

Buy one share of a synthetic Indian large-cap at ₹1,840, hold it a year, collect ₹34 of dividend, sell at ₹1,975. The holding period return is the whole gain over what you put up:

HPR = (P₁ − P₀ + income) / P₀ = (1975 − 1840 + 34) / 1840 = 0.0918478 = 9.184783%

Split it and the two halves are worth naming separately. The price return is 135/1840 = 7.336957% and the income return is 34/1840 = 1.847826%. They sum to the total because both sit over the same denominator.

The denominator is the whole idea. A return is a gain expressed per unit of capital committed, which is what lets you compare a ₹1,840 share with a ₹4 crore flat. Get the denominator wrong and every statistic downstream inherits the error.

Now chain across sub-periods. Four quarterly HPRs of +4.20%, −3.10%, +6.55%, +1.80% compound to

(1.0420)(0.9690)(1.0655)(1.0180) − 1 = 0.0951982 = 9.519822%

not the 9.45% you get by adding them. The 6.98 basis points of difference is compounding, and it grows with both the number of periods and their volatility. Adding returns is only ever an approximation. It gets worse exactly when the numbers get interesting.

One warning about the income term. A dividend received on 12 August and reinvested is worth more at year end than the same dividend held in cash, so a total return index and a price index diverge, and a net total return index diverges from a gross one by the withholding tax. The performance-measurement work later in the ring turns that into a benchmark-validity question. Here it is enough to insist that whenever you write an income number into a numerator, you know which of those three things it is.

3. Whose return: the investor's cash flows and the manager's period

Once money moves in and out, one price series produces two defensible returns. A time-weighted return cuts the period at every external flow and chain-links the pieces, so it describes what happened to one rupee left alone. A money-weighted return solves for the single rate that makes the whole dated cash-flow series net to zero, so it describes what the investor earned on the capital actually at risk.

Σ CFₜ / (1 + r)^t = 0

The performance-measurement node owns that fork and its worked example. The standards require a time-weighted return for composite reporting, on the argument that a manager should not be scored on flows the client controlled. A money-weighted return is required instead where the firm controls the timing and the strategy is closed-end, which is why private-market results arrive as a since-inception internal rate of return. What belongs here is narrower. The estimator problem inside the money-weighted branch is a statistics question, and it has a price.

Before daily valuation was cheap, solving an IRR on every account every month was impractical, so the industry used a linear approximation called Modified Dietz:

R = (EMV − BMV − CF) / (BMV + Σ wᵢCFᵢ), with wᵢ = (D − dᵢ)/D

The denominator is average invested capital, weighting each flow by the fraction of the period it was actually invested. The numerator is the gain with every external flow stripped out. Worked example 2 runs both on a US separately managed account and finds Modified Dietz 3.29 basis points below the true dated IRR on a year containing a contribution worth 150% of the opening value. On a gentler flow pattern the same comparison closes to 0.20 basis points.

Read the direction of that error rather than its size. Modified Dietz assumes the money earns at a constant simple rate across the period, so it misses the compounding the true IRR captures, and the miss grows with flow size, flow timing and within-period volatility. Three basis points is not a reason to abandon the formula. Three basis points you cannot bound in advance is a reason to know when they appear.

4. Annualising, and the difference between compounding and prorating

A 126-day holding period return of 9.10% annualises to

(1.0910)^(365/126) − 1 = 28.697970%

Prorating it instead, 9.10 × 365/126, gives 26.361111%, which is 2.336859 points too low. The direction surprises people. For a period shorter than a year, prorating understates, because it throws away the compounding of the stub periods. For a period longer than a year, prorating overstates: a three-year cumulative return of 46.9% annualises to 13.677348% and prorates to 15.633333%, an overstatement of 1.955986 points. The sign flips at exactly one year.

The same conversion runs on any base period. A quarterly 3.27% is (1.0327)⁴ − 1 = 13.735675% compounded and 13.08% prorated. A monthly 1.06% is (1.0106)¹² − 1 = 13.488414% compounded and 12.72% prorated.

For volatility the exponent is different, because variance rather than return is what adds across independent periods. A quarterly standard deviation of 6.1% annualises as 6.1 × √4 = 12.20%, and a monthly 3.4% as 3.4 × √12 = 11.778%. Those two are not two routes to one number. They are different statistics computed from different sampling frequencies, and a series that is not independent across periods breaks the square-root rule in whichever direction its autocorrelation points.

There is one annualisation you must simply refuse. A fund up 9% in four months does not have a 29.5% annual return; it has a 9% four-month return. Extrapolating it assumes the remaining eight months repeat the first four, and short flattering windows are exactly what a marketing process selects for. The performance standards make the prohibition explicit for periods under a year, and the reason is statistical rather than legal.

5. Continuously compounded returns, and why they add

Take the log of one plus a simple return and you get the continuously compounded return, r_cc = ln(1 + HPR). The four quarterly holding period returns from earlier become

ln(1.0420) = +0.04114194, ln(0.9690) = −0.03149067, ln(1.0655) = +0.06344417, ln(1.0180) = +0.01783992

and they sum to 0.09093537. Exponentiate that sum, subtract one, and you recover 0.09519822: exactly the chained simple return computed earlier, to ten decimals. Log returns add where simple returns multiply, which is the whole reason quantitative work is done in logs.

Two more properties earn their keep. Log returns are symmetric in a way simple returns are not: a +30% continuous move is e^0.30 − 1 = +34.986% simple and a −30% continuous move is e^−0.30 − 1 = −25.918% simple, so a round trip in logs is a round trip in money while a +55% followed by a −55% in simple terms is not. A log return also has no floor at −100%, so the normal distribution can model it without predicting impossible prices. A price whose log return is normal is itself lognormal, which is the model the probability branch builds and the option-pricing work later consumes.

The conversion is not free. A log return of −0.7985 is a simple return of −55%, and averaging log returns then exponentiating the average gives you the geometric mean, not the arithmetic one. Whenever you switch representation, switch back before you report, and label which one is on the page.

11. Five ways a sample lies to you

Every statistic above assumes the sample in front of you was drawn fairly from the population you care about. Five specific ways that assumption fails, each with a financial instance and a fix.

• Survivorship bias. The sample contains only what is still there. Start with 100 funds; ten years later 78 survive with a mean annualised return of 11.4%, and 22 closed with 3.1% up to closure. The survivors-only mean overstates the full cohort's 9.574% by 1.826 points a year, and the terminal-wealth overstatement compounds to 17.97% over the decade. Fix: a database that keeps dead entities with their final returns, and an explicit statement of the attrition rate, here 22%.
• Data snooping. The sample is fine and the search is the problem. Test 400 unrelated candidates at a 5% significance level and you expect 20 false positives by construction; the probability of getting through with none is 0.95⁴⁰⁰, about one in eight hundred million. Fix: the multiple-testing corrections below, and before any of them an honest count of how many things you tried. That count is an estimator input, and the only person who can supply it is you.
• Sample selection bias. Observations are missing for a reason related to what you are measuring. Unlisted companies do not report, illiquid bonds do not print prices, and hedge funds report to commercial databases voluntarily and stop when the results turn. Self-selection is the same defect wearing a different coat. Fix: characterise what is absent before modelling what is present, and prefer a source with mandatory reporting where one exists.
• Look-ahead bias. The backtest uses information the trader could not have had. Annual results for a 31 March year end are published with a lag, so a number available on 15 May is used from 1 April, buying 12.33% of a year of free foresight on every rebalance. Fix: point-in-time data keyed on the filing date rather than the period date, and a rule that a signal may only reference facts whose publication timestamp precedes the trade.
• Time-period bias. The window is too short, or was chosen after seeing the result. Drop the two worst of the twenty quarters above and the mean rises from 2.67% to 4.98%, the standard deviation falls from 9.32 to 6.28, and the Sharpe ratio goes from 0.1136 to 0.5362. Fix: a pre-registered window, a rolling-window sensitivity, and the result reported on every window you looked at.

Nobody has to be dishonest for the last of those to happen. A start date chosen because the data gets clean there does it automatically, and so does a ten-year window that begins after the last crisis.

12. The ANOVA table, and the two routes to F

A regression printout contains an analysis-of-variance table, and it is the fastest honest summary of whether the model explains anything. It rests on one decomposition:

SST = RSS + SSE

Total sum of squares is the variation in y about its own mean, regression sum of squares is the part the fitted values explain, and error sum of squares is what is left. The degrees of freedom split the same way: n − 1 = k + (n − k − 1).

Take a synthetic US fund regressed on three factors over 60 months, with SST = 1842.60, RSS = 1204.85, SSE = 637.75. Then R² = 1204.85/1842.60 = 0.653886, and the mean squares are MSR = 1204.85/3 = 401.616667 and MSE = 637.75/56 = 11.388393, giving

F = MSR/MSE = 35.265438

against a 5% critical value of 2.769431 on (3, 56) degrees of freedom. The second route to the same number uses R-squared alone:

F = [R²/k] / [(1 − R²)/(n − k − 1)] = (0.653886/3) / (0.346114/56) = 35.265438

The two agree to machine precision because they are algebraically the same expression. That is your check: an ANOVA table whose F does not reproduce from its own R-squared has an arithmetic error in it, and the error is usually a degrees-of-freedom count.

Two derived numbers earn their place. The standard error of the regression, √MSE = 3.374669, is in the units of y and is the natural scale for judging whether a residual is large. Adjusted R-squared, 1 − (1 − R²)(n − 1)/(n − k − 1) = 0.635344, charges rent for each regressor. Add a fourth factor that cuts SSE from 637.75 to 632.10 and R-squared rises to 0.656952 while adjusted R-squared falls to 0.632003. The partial F for that fourth regressor is 0.491615, whose square root, 0.701153, is its t-statistic, well under the 2.004 critical value. R-squared can never fall when a regressor is added, which is exactly why it cannot be used to choose a model.

13. Dummy variables, and the trap that makes a regression unsolvable

A categorical variable enters a regression as a dummy: one indicator taking the value 1 when the category applies and 0 otherwise. For a binary attribute, one dummy is enough. For k categories, you need k − 1 dummies, and the category you leave out becomes the baseline the others are measured against. Four rating buckets need three dummies.

Including all k, alongside an intercept, is the dummy variable trap. The k dummies sum to a column of ones, which is exactly the intercept, so the design matrix is singular and the coefficients are not identified. Statistical software either drops one silently or returns an error, and the silent drop is the more dangerous outcome because the printed baseline is then whichever category the sort order happened to put first.

Two placements do different work. An intercept dummy shifts the line up or down, holding the slope. A slope dummy, formed by multiplying the dummy by a continuous regressor, changes the slope itself. Both together let two groups have entirely separate lines fitted in one regression, which is more efficient than two separate fits and lets you test the difference directly.

Take a credit-spread regression on 25 synthetic US issuers with an investment-grade dummy D and its interaction:

spread = 46.0 + 38.5x − 57.0D − 12.0Dx

For high-yield issuers (D = 0) the line is 46.0 + 38.5x. For investment-grade issuers (D = 1) it is −11.0 + 26.5x. At two turns of leverage the two groups price at 123.0 and 42.0 basis points, a gap of 81.0; at four turns, 200.0 and 95.0, a gap of 105.0. The gap widens with leverage because the slope differs, which is a statement the intercept dummy alone could not have made.

Read the coefficients for what they are, though. The fitted lines cross at x = −4.75, outside any possible data, so the negative intercept for investment-grade issuers is an extrapolation artefact rather than a claim that a zero-leverage investment-grade issuer would price 11 basis points through the curve. A coefficient is a statement about the sample's range and nothing else.

14. Prediction intervals: the forecast is wider than the fitted line

A regression produces two different intervals at any point x₀ and confusing them is a routine professional error.

The confidence interval for the conditional mean asks: where is the average y for issuers at this x? Its standard error is

s × √(1/n + (x₀ − x̄)²/Sxx)

The prediction interval asks: where will this next issuer's y fall? It carries the same uncertainty about the line plus the irreducible scatter of individual points around it:

s × √(1 + 1/n + (x₀ − x̄)²/Sxx)

That extra 1 dominates. On the credit-spread regression, n = 25, s = 18.40, x̄ = 3.20, Sxx = 42.50, and t(0.025, 23) = 2.068658. At the sample mean the fitted value is 169.20 basis points, the mean interval runs [161.59, 176.81] and the prediction interval runs [130.38, 208.02], 5.099 times wider. Quoting the first when a client asked the second is the difference between a claim about the average issuer and a claim about their issuer.

Both intervals widen as x₀ moves away from x̄, because the fitted line pivots about the sample means and a small slope error becomes a large level error far from the pivot. At x₀ = 5.60, 2.4 turns above the mean, the leverage term rises from 0.0400 to 0.175529, the mean interval widens from ±7.61 to ±15.95, and the prediction interval from ±38.82 to ±41.27. The two widen at very different rates, and the ratio between them falls from 5.099 to 2.588.

Read that ratio for what it says about extrapolation. The mean interval more than doubled while the prediction interval grew by 6%, because far from the sample mean, uncertainty about the line finally becomes comparable to the residual scatter that always dominated it. Go far enough and the model's own uncertainty is the whole story, which is the arithmetic reason a regression must not be trusted beyond the range of its data.

18. The sampling distribution of a Sharpe ratio, and how long a backtest must be

A Sharpe ratio is an estimate, so it has a standard error, and that standard error is larger than almost anyone assumes. Under independent, identically distributed normal returns,

Var(SR̂) ≈ (1 + SR̂²/2) / T

For a Sharpe of 0.85 estimated over 36 periods, Var = (1 + 0.36125)/36 = 0.037813, a standard error of 0.194454 and a 95% interval of [0.469, 1.231]. Three years of monthly data cannot distinguish a mediocre manager from an excellent one. Quadruple the sample to 144 periods and the interval tightens to [0.659, 1.041], still wide enough to matter.

Returns are neither normal nor independent. The correction for the first is worth carrying:

Var(SR̂) ≈ (1 + SR̂²/2 − γ₃SR̂ + (γ₄ − 3)SR̂²/4) / T

with γ₃ the skewness and γ₄ the non-excess kurtosis. Negative skew and fat tails both inflate the variance, which is the correct behaviour: a series that occasionally loses a lot deserves less confidence in its mean. On the twenty-quarter series the correction raises the standard error by 5.51%.

The same denominator appears in the probabilistic Sharpe ratio, written √(1 − γ₃SR̂ + ((γ₄ − 1)/4)SR̂²). The two are algebraically identical, since (γ₄ − 1)/4 = 1/2 + (γ₄ − 3)/4, and checking that identity once is the cleanest way to be sure you have the kurtosis convention right. Pass an excess kurtosis where the formula wants a non-excess one and the statistic is silently wrong on every non-normal series, which is all of them.

Now combine the estimator's variance with the counting problem above. If N independent strategies are tried and none has any true edge, the expected maximum of the N estimated Sharpe ratios is approximately

E[max SR̂] ≈ σ_SR × [(1 − γ)Φ⁻¹(1 − 1/N) + γΦ⁻¹(1 − 1/(Ne))]

where γ is the Euler-Mascheroni constant, 0.5772157, and σ_SR is the standard deviation of a single trial's estimate, √(1/T) under the null. At N = 400 and T = 120, the bracket is 2.984802, σ_SR = 0.091287, and the expected best-of-400 monthly Sharpe from pure luck is 0.272474, or 0.943877 annualised. The winner's actual 0.692820 sits below that bar.

Substituting that luck bar for zero inside the probabilistic Sharpe formula gives the deflated Sharpe ratio. On the screen's winner it is 0.2280, against a naive probabilistic Sharpe of 0.980155 computed against a benchmark of zero. The deflation costs 0.752 of probability, and it is the entire difference between a finding and a coincidence.

Invert the same expression and it tells you how much history you would need. Setting the luck bar equal to the observed annualised Sharpe gives a minimum backtest length:

MinBTL ≈ bracket² / SR̂²_annual = 8.909040 / 0.480000 = 18.560500 years

Eighteen and a half years. You have ten. The requirement scales with the number of trials in a way worth memorising: at N = 10 the luck bar is an annualised 0.4979 and MinBTL is 5.17 years; at N = 100, 0.8002 and 13.34 years; at N = 1000, 1.0294 and 22.07 years. Every additional variant you try lengthens the history you would need to justify the winner, which is the precise sense in which searching harder makes evidence weaker.

{
  "skill": "Resampling a statistic that has no usable closed-form sampling distribution",
  "use": "Have an assistant write a seeded bootstrap and a jackknife for the statistic you care about (a Sharpe ratio, a drawdown, a rank correlation), each as a separate function, so the two can be run against the same data and their bias estimates compared before either interval is quoted.",
  "tools": ["Chat assistants with code execution - Claude, ChatGPT, Gemini", "Python: NumPy (default_rng, integers) and SciPy for the analytic comparison", "A notebook or spreadsheet for the side-by-side of percentile and delta-method intervals"],
  "prompt": "Here is a vector of period returns and a stated risk-free rate for the same period length. Write Python that computes (a) the point-estimate Sharpe ratio with ddof stated explicitly, (b) a bootstrap percentile 95% interval from B resamples drawn with a seed I pass in, (c) the leave-one-out jackknife bias estimate and standard error, and (d) the delta-method interval from Var(SR) = (1 + SR^2/2)/T. Print the seed and B alongside every interval. Then run the whole thing at three different seeds and report how far the interval endpoints move.",
  "verify": "Run it at three seeds and confirm the endpoints agree to the precision you intend to print; if the third decimal moves, do not print a third decimal. Check that the bootstrap and jackknife bias estimates agree in sign and rough magnitude - a disagreement means one of them has a bug, and on the module's twenty-quarter series they agree at +0.0404 and +0.0435. Recompute the point estimate by hand from the sum and the sum of squares. Confirm the ddof used in the resample matches the ddof used in the point estimate, because a mismatch produces a statistic whose interval is centred on a different number than the one you reported. If the data are serially dependent, an ordinary bootstrap is invalid and the code must say so rather than run.",
  "diy": "You must be able to build a percentile interval by hand from a sorted vector, and to state which of the two intervals is wider and why, before any of this is worth running. The gate rewards knowing that a symmetric interval on a right-skewed statistic reports the wrong shape of uncertainty, which no library will tell you.",
  "market": "IN"
}

{
  "skill": "Correcting a factor screen for the number of trials that produced its winner",
  "use": "Have an assistant implement Bonferroni, Holm and Benjamini-Hochberg over one p-value vector, plus the expected-maximum-Sharpe and deflated-Sharpe calculations, then have it report how many candidates survive under each so the difference is visible rather than argued about.",
  "tools": ["Chat assistants with code execution", "Python: NumPy and scipy.stats (norm.ppf, norm.cdf) for the quantiles", "Your own trial log - the count of variants actually run, which no tool can supply"],
  "prompt": "Here is a sorted vector of p-values from N candidate strategies, the number of periods T, and the winner's per-period Sharpe ratio, skewness and non-excess kurtosis. Compute (a) the Bonferroni and Sidak thresholds and the count rejected, (b) the Holm step-down count, (c) the Benjamini-Hochberg count at a false discovery rate I specify, printing the largest passing rank and its threshold, (d) the expected maximum Sharpe over N trials using the Euler-Mascheroni form, and (e) the deflated Sharpe ratio of the winner against that bar. State every quantity as per-period, never annualised, and say so in the output.",
  "verify": "Check the Benjamini-Hochberg step by hand on the twelve smallest p-values: the printed largest passing rank must match the last row where p is at or below (i/N)q. Confirm nothing was annualised before entering the deflated Sharpe formula, because annualising the inputs inflates the numerator by about the square root of the periods per year and corrupts the whole statistic silently. Confirm the kurtosis passed is non-excess, by checking that a normal series returns 3 and not 0. Above all, verify N yourself: the model will use whatever number you type, and the honest count includes every variant you discarded, every parameter you tuned, and everything a colleague tried before handing you the survivor. That count is an estimator input, and supplying it truthfully is the only part of this calculation a tool cannot do.",
  "diy": "You must be able to derive why the Bonferroni-clearing Sharpe at N = 400 over ten years of monthly data is an annualised 1.21, and to state what BH controls and what it does not (the expected proportion of false discoveries, not the realised one). Without that, you will run the code and misread the output.",
  "market": "US"
}

Common mistakes & how experts think differently

1. Reporting an arithmetic mean of returns as though it described the path. The +40/−30/+40/−30 series averages +5.00% and destroyed 3.96% of the capital. The expert's reflex on seeing any average return is to ask which mean it is, and to reach for the variance, because the gap between arithmetic and geometric is approximately half of it and can be estimated before the geometric mean is computed.

1. Averaging prices or multiples arithmetically. When a fixed sum is spent each period, the average cost is the harmonic mean of the prices, not the arithmetic mean; on the six-month purchase plan the two differ by 6.31%. The same logic governs aggregating a comp set of price-earnings multiples, and the expert reaches for harmonic or median by default and treats a large mean-versus-median divergence as a signal to go look at the set.

1. Prorating instead of annualising, in either direction. Multiplying a 126-day return by 365/126 understates it by 2.34 points; dividing a three-year cumulative by three overstates it by 1.96. Experts know the sign flips at one year, and they refuse to annualise anything shorter than a year at all, because the extrapolation says more about the window than the strategy.

1. Letting a library default choose the divisor. The standard array library's standard deviation divides by n; the standard dataframe library's divides by n − 1. On a twenty-observation series that moves the Sharpe ratio by 2.6%, enough to reorder a manager ranking, from a keyword nobody typed. Experts state the degrees of freedom explicitly in every computation and in every report footnote.

1. Believing a normal-based VaR because its arithmetic is exact. The two-regime model's 99% VaR is exactly right about a world with no fat tails and understates the real loss by 35.5%, with 4.35 breaches per 250 days instead of the promised 2.5. Experts compute the fourth moment first, compare the model's implied breach count to the realised one, and treat a breach-count mismatch as a model rejection rather than a run of bad luck.

1. Quoting a bootstrap interval without its seed, its B, or its resampling unit. The same computation at three seeds moves the upper endpoint from 0.7330 to 0.7726. Experts print the seed and the number of resamples beside the interval, round to the precision the Monte Carlo error supports, and never run an ordinary bootstrap on serially dependent data without saying that they have assumed independence.

1. Reading a t-statistic from a selected maximum as though it were a t-statistic, and then over-correcting. A winner at t = 2.19 chosen from 400 candidates is not significant, because 20 of 400 true nulls clear 5% by construction. The expert's first question about any backtest is how many variants were run, including the ones a colleague discarded. The second question is which error rate the decision needs: Bonferroni at N = 400 demands an annualised Sharpe of 1.21 and rejects nearly every genuine finding along with the noise, so a screen whose output is a shortlist wants false-discovery control instead.

1. Reading McFadden's pseudo-R-squared on an ordinary R-squared scale. A value of 0.137 on the credit panel is a respectable logistic fit, not a failure, because the statistic is a ratio of log-likelihoods with no variance interpretation. The expert compares pseudo-R-squared only across models on the same data with the same dependent variable, and reports the likelihood-ratio test and the odds ratio instead when talking to anyone else.

1. Confusing a confidence interval for the conditional mean with a prediction interval. At the sample mean the second is 5.10 times wider than the first. Quoting the narrow one when someone asked what their issuer will pay is not a rounding error; it is answering a different question with a more comfortable number.

1. Deleting the two worst observations because they look like outliers. Dropping them from the twenty-quarter series lifts the Sharpe from 0.114 to 0.536. Experts investigate an outlier, report the result with and without it, and treat any exclusion as a disclosure item rather than a cleaning step.

Worked examples

Worked example 1: Four means, four answers, on one investment plan (India, ₹)

A learner runs a monthly purchase plan into a synthetic Indian index fund, investing ₹10,000 on the same date each month for six months. The fund's net asset value on those six dates is ₹250, ₹200, ₹320, ₹400, ₹250, ₹200. Separately, the same fund's four annual returns over an earlier stretch were +40%, −30%, +40%, −30%. Compute every mean that anyone might quote, and say which question each one answers.

Solution. Step 1, the purchase plan, unit by unit. A fixed rupee sum buys more units when the price is low: 10000/250 = 40.000, 10000/200 = 50.000, 10000/320 = 31.250, 10000/400 = 25.000, 10000/250 = 40.000, 10000/200 = 50.000, for 236.250 units against ₹60,000 invested.

Step 2, the realised average cost. 60,000 / 236.250 = ₹253.968254 per unit.

Step 3, the three means of the six prices.

The harmonic mean reproduces the realised average cost exactly, to the last decimal, because both are total cash over total units. The arithmetic mean overstates the cost by ₹16.031746 per unit, or 6.3125%, and the ordering harmonic ≤ geometric ≤ arithmetic holds as it must for positive numbers.

Step 4, the return series. Terminal wealth relative 1.40 × 0.70 × 1.40 × 0.70 = 0.960400, so the geometric mean is 0.9604^(1/4) − 1 = −1.005051% and the arithmetic mean is (40 − 30 + 40 − 30)/4 = +5.000000%.

Step 5, the check on the gap. The population variance of the four returns is 0.1225, so the approximation AM − σ²/2 = 5.000000 − 6.125000 = −1.125000% lands 11.99 basis points from the exact geometric mean, on a series with a 35% standard deviation.

Step 6, the money. Ten lakh rupees compounds to 10,00,000 × 0.9604 = ₹9,60,400. At the arithmetic mean it would be 10,00,000 × 1.05⁴ = ₹12,15,506.25. The 5.0% is not a false statement; it answers a question about next year's single draw.

Read it. The arithmetic mean is the right input to a forward-looking expected-return model and the wrong description of a path; the harmonic mean is the right average for a fixed-rupee purchase programme and the wrong one for a set of returns. What would break the example is a price of zero, which the harmonic mean cannot accommodate, or a negative price, which leaves the geometric mean undefined. Both are why these two means live on prices rather than on returns.

Worked example 2: Modified Dietz against a true dated IRR (US, $)

Cascadia SMA-114 (synthetic) is a US separately managed account. It opens the calendar year at $2,000,000, receives a contribution of $3,000,000 on day 50, pays out $1,000,000 on day 257, and closes the year at $4,600,000. No intra-period valuations are available. Compute the Modified Dietz return and the true dated internal rate of return, and quantify the approximation error.

Solution. Step 1, the gain. Strip the external flows: 4,600,000 − 2,000,000 − (3,000,000 − 1,000,000) = $600,000.

Step 2, the day weights. w = (365 − d)/365, so the contribution gets (365 − 50)/365 = 0.863014 and the withdrawal gets (365 − 257)/365 = 0.295890.

Step 3, average invested capital. 2,000,000 + 0.863014(3,000,000) + 0.295890(−1,000,000) = 2,000,000 + 2,589,041.10 − 295,890.41 = $4,293,150.68.

Step 4, Modified Dietz. 600,000 / 4,293,150.68 = 0.139757 = 13.975750%.

Step 5, the true IRR. Solve −2,000,000 − 3,000,000/(1+r)^(50/365) + 1,000,000/(1+r)^(257/365) + 4,600,000/(1+r) = 0. Bracket first: at r = 0 the sum is +600,000 and at r = 20% it is −213,144, so the root is between. Converging gives r = 14.008649%.

Step 6, the error, and a control. Modified Dietz sits 3.29 basis points below the true IRR on a year whose single contribution was 150% of the opening value. Rerun the same structure with a $200,000 contribution and a $100,000 withdrawal and a closing value of $2,450,000: Modified Dietz returns 16.332140% against a true 16.334135%, an error of 0.20 basis points.

Read it. The linear approximation is excellent when flows are small relative to the portfolio and poor when they are not, which is both the case the formula was invented for and the case it is most often misused in. The naive answer that ignores flows entirely, 600,000/2,000,000 = 30.000000%, is off by sixteen points and is still printed on client statements. What would break the example is a flow pattern producing multiple sign changes in the cash-flow vector, which can give the IRR more than one real root. Bracketing before solving is what catches that.

Worked example 3: The full descriptive set, by hand (India, ₹)

The Sahyadri Nifty-Linked Portfolio (synthetic) reports twenty quarterly returns, in percent:

+8.40, −3.10, +12.60, +2.90, −14.80, +6.20, +1.40, +9.70, −2.30, +4.80, +15.30, −6.90, +3.60, +0.70, −21.40, +11.20, +5.50, −1.80, +7.30, +14.10

Compute location, dispersion and shape, and state where each statistic misleads.

Solution. Step 1, sort and locate. The sum is 53.40, so the arithmetic mean is 53.40/20 = 2.670000%. The sorted middle pair is 3.60 and 4.80, so the median is 4.200000%. All twenty values are distinct, so there is no mode; grouped into ten-point buckets with edges at −25, −15, −5, +5, +15 and +25 the counts are 1, 2, 8, 8, 1 and two buckets tie for modal, which is the mode telling you it has nothing to say.

Step 2, growth. The terminal wealth relative is Π(1 + rᵢ/100) = 1.557099, so the geometric mean is 1.557099^(1/20) − 1 = 2.238816% a quarter, or 9.260516% annualised. Compounding the arithmetic mean instead gives 11.115398% annualised, overstating growth by 1.85 points a year.

Step 3, dispersion.

Step 4, quantiles. With linear interpolation, Q1 = −1.925000, Q2 = 4.200000, Q3 = 8.725000, IQR = 10.650000. The 1.5-IQR fences sit at −17.900000 and +24.700000, so the whiskers run to −14.80 and +15.30 and one point, −21.40, plots outside. The first and ninth deciles are −7.690000 and +12.750000.

Step 5, shape. The cubed deviations sum to −14,747.799 and the fourth powers to 501,086.04. With s = 9.321063, so s³ = 809.834535 and s⁴ = 7548.518506:

S = [20/(19 × 18)] × (−14,747.799)/809.834535 = −1.064964

K_E = [20 × 21/(19 × 18 × 17)] × 501,086.04/7548.518506 − 3(19²)/(18 × 17) = +1.256184

Step 6, verify by two independent routes. The deviations sum to 0.000000000000, which they must. Recomputing the same skewness and excess kurtosis through a standard statistics library's bias-corrected estimators returns −1.0649636953 and +1.2561843900, matching to ten decimals.

Read it. The mean sits 1.53 points below the median because the left tail is longer, which the negative skewness confirms. The coefficient of variation of 3.49 is arithmetically correct and practically useless, because a mean of 2.67 against a standard deviation of 9.32 makes the ratio unstable to a single observation. What would break every number here is a change of sample. The Jarque-Bera statistic on the population moments is 3.606070 against a 5% critical 5.991465, so at n = 20 this data cannot reject normality even though it visibly is not normal. Twenty observations is enough to compute a fourth moment and nowhere near enough to test one.

Worked example 4: A normal VaR that promises 1% and delivers 1.74% (India, ₹)

The same portfolio's daily behaviour is modelled as a stated two-regime mixture: on 90% of days the return is drawn from a normal with mean zero and standard deviation 0.70%, and on the other 10% from a normal with mean zero and standard deviation 2.60%. The position is ₹2,00,00,000. Compute the 99% one-day value at risk the normal assumption implies, then compute what that number actually delivers.

Solution. Step 1, the mixture's moments, in closed form. Variance is the probability-weighted average of the two variances: 0.90(0.0070²) + 0.10(0.0260²) = 0.00004410 + 0.00006760 = 0.00011170, so σ = 1.056882% a day, or 16.710775% annualised at 250 days. The fourth moment is 3[0.90(0.0070⁴) + 0.10(0.0260⁴)] = 1.4357550 × 10⁻⁷, so kurtosis is 1.4357550×10⁻⁷ / 0.00011170² = 11.507315 and excess kurtosis is 8.507315.

Step 2, the normal-based VaR. z₉₉ = 2.3263479, so the cut is 2.3263479 × 1.056882% = 2.458676% of value, or ₹4,91,735 on the position.

Step 3, what that cut actually catches. Under the mixture, the probability of a loss worse than 2.458676% is the weighted sum of two normal tails:

0.90 × Φ(−2.458676/0.70) + 0.10 × Φ(−2.458676/2.60) = 0.90 × Φ(−3.512394) + 0.10 × Φ(−0.945644) = 0.90 × 0.00022204 + 0.10 × 0.17216497 = 0.00019984 + 0.01721650 = 0.01741634

Step 4, in breach counts. Over 250 trading days the model promises 0.01 × 250 = 2.50 breaches and the mixture delivers 0.01741634 × 250 = 4.354084, a ratio of 1.741634 times. Almost the whole of the excess comes from the storm regime, which contributes 0.0172 of the 0.0174.

Step 5, the true VaR. Solving 0.90Φ(−x/0.0070) + 0.10Φ(−x/0.0260) = 0.01 gives x = 3.332163%, or ₹6,66,433. The normal number understates the genuine loss by ₹1,74,697, which is 35.53% of itself. At 99.9% the gap widens: 3.266012% normal against 6.048504% true, ₹6,53,202 against ₹12,09,701, an understatement of 85.20%.

Step 6, the independent route, and its failure. A Cornish-Fisher expansion adjusts the normal quantile using the same moments: z_CF = 2.3263479 + (2.3263479³ − 3 × 2.3263479)/24 × 8.507315 = 4.315254, giving 4.560715% or ₹9,12,143. That overshoots the true 3.332163% by more than the normal number undershoots it.

Read it. The fourth moment is not a curiosity. On this position it is worth ₹1.75 lakh a day at the 99% level and ₹5.56 lakh at the 99.9% level. The Cornish-Fisher failure is the honest part of the example: the correct response at an excess kurtosis of 8.5 is a historical or mixture-based quantile rather than any adjustment to a normal one. What would break the whole example is the assumption that the regime probabilities are known, when in practice they are estimated from the same short sample whose tail behaviour is in question.

Worked example 5: A bootstrap and a jackknife on one Sharpe ratio (India, ₹)

Take Worked example 3's twenty quarterly returns and a 91-day Treasury bill yielding 6.60% a year (perishable; check against the current auction cut-off). Compute the quarterly Sharpe ratio, then put an honest interval around it three ways.

Solution. Step 1, the excess series. Convert the risk-free rate geometrically: 1.066^0.25 − 1 = 1.610667% a quarter. Mean excess return is 2.670000 − 1.610667 = 1.059333%, and the standard deviation of the excess series is the same as that of the raw series, 9.321063%, because subtracting a constant does not move dispersion.

Step 2, the point estimate. SR = 1.059333/9.321063 = 0.113649 a quarter, or 0.227299 annualised at × √4. Using the population divisor instead gives 0.116602, a 2.6% relative difference from one keyword.

Step 3, the delta-method interval. Var(SR̂) = (1 + 0.113649²/2)/20 = 0.050323, so the standard error is 0.224328 and the 95% interval is 0.113649 ± 1.959964 × 0.224328 = [−0.326025, +0.553324]. Adding the skewness and kurtosis correction, with γ₃ = −0.983367 and γ₄ = 3.677675, raises the standard error 5.51% to 0.236686 and widens the interval to [−0.350247, +0.577545].

Step 4, the bootstrap. Draw 10,000 resamples of 20 observations with replacement, using a stated generator and seed (default_rng(20260907), index vectors from integers(0, 20, 20)). The 10,000 Sharpe ratios have a mean of 0.154090 and a standard deviation of 0.263929, and the 2.5th and 97.5th percentiles of the sorted vector, interpolated linearly as the quartiles were in Worked example 3, give [−0.286535, +0.762551]. Reading the bare 250th and 9,750th order statistics instead, with no interpolation, moves the lower endpoint to −0.286998, which is why the convention belongs beside the interval.

Step 5, the jackknife. Drop one observation at a time and recompute. The twenty leave-one-out Sharpe ratios average 0.115941, so the bias estimate is 19 × (0.115941 − 0.113649) = +0.043543 and the jackknife standard error is 0.266341.

Step 6, the cross-checks. The bootstrap's own bias estimate, 0.154090 − 0.113649 = +0.040440, agrees with the jackknife's +0.043543 to within four thousandths from two entirely different resampling schemes. Rerunning the bootstrap at seeds 11 and 424242 gives upper endpoints of 0.732954 and 0.772570 against the headline 0.762551.

Read it. The percentile interval is 1.19 times as wide as the delta-method one and visibly asymmetric, with a lower arm of 0.400 against an upper arm of 0.649, and 28.96% of the resampled Sharpe ratios are negative. Twenty quarters cannot tell a skilled manager from a lucky one, and the honest output here is an interval containing zero. What would break it is serial dependence, since the ordinary bootstrap assumes the twenty observations are exchangeable and a volatility-clustered series is not. The jackknife's influence reading is the sharpest warning: dropping the single −21.40% quarter moves the Sharpe from 0.114 to 0.306.

Worked example 6: Filling an ANOVA table and checking F both ways (US, $)

A printout for Willamette Core Equity (synthetic) reports a three-factor regression on 60 monthly excess returns, giving SST = 1842.60 and SSE = 637.75. Complete the analysis-of-variance table, test the model, and check the F statistic by an independent route.

Solution. Step 1, close the decomposition. RSS = SST − SSE = 1842.60 − 637.75 = 1204.85.

Step 2, degrees of freedom. Regression k = 3; error n − k − 1 = 60 − 3 − 1 = 56; total n − 1 = 59. The two parts sum to the total, which is the first check.

Step 3, the table.

Step 4, the verdict. The 5% critical value of F on (3, 56) is 2.769431 and the 1% value is 4.151941, so 35.265438 rejects the joint null overwhelmingly, at a p-value of about 6.19 × 10⁻¹³.

Step 5, the independent route. R² = 1204.85/1842.60 = 0.653886, and

F = (R²/k) / ((1 − R²)/(n − k − 1)) = (0.653886/3)/(0.346114/56) = 35.265438

identical to twelve significant figures, because the two expressions are the same algebra.

Step 6, the penalty. The standard error of the regression is √11.388393 = 3.374669 in the units of the dependent variable. Adjusted R-squared is 1 − 0.346114 × 59/56 = 0.635344. Now add a fourth factor that reduces SSE to 632.10: R-squared rises to 0.656952 while adjusted R-squared falls to 0.632003. The partial F for that fourth regressor is (637.75 − 632.10)/(632.10/55) = 0.491615, whose square root, 0.701153, is its t-statistic, far below the critical 2.004045.

Read it. An F this large says the three factors jointly explain something. It says nothing about which one, and nothing about whether the model is correctly specified. The fourth-regressor demonstration is the useful half: R-squared cannot fall when a variable is added, so a process that selects on R-squared keeps adding noise forever, and the adjusted version exists to make that stop. What would break the table is a missing intercept, which changes what SST means and leaves R-squared uninterpretable.

Worked example 7: A logistic regression on a borrower panel (India, ₹)

A synthetic Indian non-bank lender groups 380 corporate borrowers into six leverage buckets and records defaults over one year.

Fit a logistic regression of default on leverage, interpret the coefficient, and test it.

Solution. Step 1, the base rate. 65/380 = 17.105263%, which is what an intercept-only model predicts for everyone.

Step 2, why not ordinary least squares. Fitting a straight line to the 0/1 outcomes gives p̂ = −0.106436 + 0.087871x, which returns a probability of −0.106436 at zero leverage. A model that cannot stay inside [0, 1] is not modelling a probability.

Step 3, the fit. Maximising the log-likelihood gives b₀ = −4.006553 and b₁ = 0.670655.

Step 4, the verification a reader can run. At the maximum, two score equations must hold exactly. Fitted probabilities are 0.034360, 0.065056, 0.119772, 0.210165, 0.342251, 0.504344, so fitted default counts are 2.0616, 5.2045, 10.7795, 14.7115, 17.1126, 15.1303. Then Σ(dⱼ − nⱼp̂ⱼ) = 0.000000000000 and Σxⱼ(dⱼ − nⱼp̂ⱼ) = 0.000000000000. Both fire to twelve decimals, which is the independent check that the optimiser converged.

Step 5, the odds ratio. e^0.670655 = 1.955518: each additional turn of Debt/EBITDA multiplies the odds of default by 1.955518, and two turns multiply them by 3.824051. The coefficient's standard error is 0.106541, so z = 6.294811 and the 95% interval for the odds ratio is [1.586989, 2.409626], comfortably clear of 1.

Step 6, fit statistics, read correctly. LL_full = −150.121555 against LL_null = −173.869522, so the likelihood-ratio statistic is 2(−150.121555 + 173.869522) = 47.495934 on one degree of freedom against a critical 3.841459. McFadden's pseudo-R-squared is 1 − (−150.121555)/(−173.869522) = 0.136585.

Read it. Predicted default probabilities are 11.98% at three turns, 27.12% at 4.5 turns and 66.55% at seven turns: the S-shape the logit imposes, and the reason the model behaves at the extremes. A McFadden of 0.136585 is a respectable logistic fit and would be a poor ordinary R-squared, which is why the two must never be compared. The honest summary for a credit committee is the odds ratio with its interval. What would break the example is the grouping itself, since bucketing a continuous leverage measure at integer midpoints imposes the midpoint on every borrower in it.

Worked example 8: Spearman's rho with ties, and the shortcut that fails (India)

Ten synthetic Indian mid-caps carry a quality factor score and a subsequent one-quarter forward return.

Compute the rank correlation properly, compute it with the shortcut, and test it.

Solution. Step 1, assign ranks with the tie rule. Sorted ascending, the scores are 0.15, 0.30, 0.45, 0.45, 0.61, 0.61, 0.61, 0.74, 0.82, 0.98. Indus takes rank 1 and Fenugreek rank 2. The two names tied at 0.45 occupy raw positions 3 and 4, so each takes (3+4)/2 = 3.5. The three tied at 0.61 occupy positions 5, 6 and 7, so each takes (5+6+7)/3 = 6.0. Girnar, Ambar and Eastwind take 8, 9 and 10. The forward returns have no ties and rank 1 through 10 directly.

Step 2, the general definition. Spearman's rho is the Pearson correlation of the two rank vectors, [9, 6, 6, 3.5, 10, 2, 8, 3.5, 1, 6] against [9, 6, 8, 3, 10, 1, 7, 4, 2, 5]. That gives ρ = 0.947804.

Step 3, the shortcut. The rank differences squared sum to Σd² = 8.500000, so

1 − 6(8.500000)/(10 × 99) = 1 − 51/990 = 0.948485

Step 4, the discrepancy. The shortcut overstates by 0.000680. It is derived assuming every rank appears exactly once, and the two tie groups break that assumption. On a heavily bucketed factor with tie groups of ten or twenty names, the same error runs into the second decimal.

Step 5, the test. t = 0.947804 × √(8/(1 − 0.947804²)) = 8.407647 on 8 degrees of freedom, against a 5% critical value of 2.306004, for a two-tailed p-value of 0.0000305.

Step 6, the comparison. The Pearson correlation of the raw numbers is 0.932493, so the two measures differ by 0.015311. The rank version is higher here because the relationship is monotone but not quite linear, and because ranks are less sensitive to the size of Eastwind's +8.8.

Read it. A gap between rank and raw correlation is diagnostic rather than a nuisance. A large gap says either that the relationship bends, or that one or two observations are carrying the raw number. What would break this example is a sample size of ten pretending to be evidence: at n = 10 a single re-ranking of two adjacent names moves rho by several hundredths. The factor-construction node uses this statistic as a rank information coefficient, computed period by period and then averaged, precisely because one period's value proves nothing.

Worked example 9: A screen of 400 candidates, corrected and deflated (US, $)

A US quant team screens 400 candidate factors over 120 months on one universe. The winner shows a mean monthly excess return of 0.62% with a monthly standard deviation of 3.10%, a return skewness of −0.42 and a non-excess kurtosis of 5.10. Report the naive result, correct it three ways, and deflate it.

Solution. Step 1, the naive numbers. Monthly Sharpe 0.62/3.10 = 0.200000, annualising to 0.200000 × √12 = 0.692820. The t-statistic is 0.200000 × √120 = 2.190890, a two-sided p-value of 0.028460 under a normal approximation, or 0.030408 on 119 degrees of freedom. Published on its own, this clears every conventional bar.

Step 2, family-wise corrections. Bonferroni sets the threshold at 0.05/400 = 0.000125 and Sidak at 1 − 0.95^(1/400) = 0.000128225, only 2.58% looser. The winner's 0.028460 misses by more than two orders of magnitude. Inverting Bonferroni, the two-sided z required is 3.836107, so a candidate needs a monthly Sharpe of 3.836107/√120 = 0.350187, or an annualised 1.213084. The winner is 0.571 of that bar.

Step 3, the false-discovery alternative. Take the full sorted p-value vector, generated reproducibly as 380 uniform draws plus 20 genuine signals at z ~ N(2.60, 0.60) under default_rng(4041). Benjamini-Hochberg at q = 0.10 compares each p₍ᵢ₎ to (i/400)(0.10):

The largest passing rank is 5, so BH rejects five hypotheses, of which four are genuine and one is a null. Bonferroni and Holm each reject one. An uncorrected 5% screen rejects 33, of which only 18 are genuine.

Step 4, the expected best from luck alone. With γ = 0.5772157, Φ⁻¹(1 − 1/400) = 2.807034 and Φ⁻¹(1 − 1/(400e)) = 3.115008, the bracket is 0.4227843(2.807034) + 0.5772157(3.115008) = 2.984802. Under the null σ_SR = √(1/120) = 0.091287, so

E[max SR̂] = 2.984802 × 0.091287 = 0.272474 a month, or 0.943877 annualised

Step 5, the deflated Sharpe ratio. Use that luck bar in place of zero:

numerator = (0.200000 − 0.272474)√119 = −0.790596 denominator = √(1 − (−0.42)(0.200000) + ((5.10 − 1)/4)(0.200000²)) = √1.125000 = 1.060660 DSR = Φ(−0.745381) = 0.228021

against a naive probabilistic Sharpe of 0.980155 computed against a benchmark of zero.

Step 6, the minimum backtest length. MinBTL = 2.984802²/0.692820² = 8.909040/0.480000 = 18.560500 years, or 222.7 months against the 120 available.

Read it. The winner's annualised Sharpe of 0.6928 sits below the 0.9439 that the best of 400 pure coin flips would be expected to show, so the honest conclusion is that this screen produced no evidence at all. The deflation costs 0.752 of probability, which is the entire distance between a finding and a coincidence. Two things would change the answer. Correlation among the 400 candidates makes them behave like a smaller effective N, which lowers the bar. And the whole calculation collapses if the reported N is wrong, which matters because N is supplied by the person with the most to lose from reporting it honestly.

The examples rest on the figures collected here, printed at the precision they were computed to, so that a reader who reworks one and lands somewhere else knows immediately which of the two is wrong.

Practice set

Work each problem fully before reading its solution. A calculator is assumed. Numeric answers within ±2% score as correct, except where a reconciliation or an identity is demanded, in which case it must hold exactly.

P1 (guided). A holding period return and its two halves. You buy a share at ₹412, receive a ₹9.50 dividend, and sell at ₹447. Compute the total HPR and split it into price and income components.

Solution. HPR = (447 − 412 + 9.50)/412 = 44.50/412 = 10.800971%. Price return 35/412 = 8.495146%; income return 9.50/412 = 2.305825%. They sum to the total because both sit over the same denominator, and that is the check to run whenever a decomposition is offered to you.

P2 (guided). Arithmetic against geometric. Five annual returns are +22%, −18%, +31%, −9%, +14%. Compute both means and check the gap against the variance approximation.

Solution. Arithmetic (22 − 18 + 31 − 9 + 14)/5 = 8.000000%. The terminal wealth relative is 1.22 × 0.82 × 1.31 × 0.91 × 1.14 = 1.359538, so the geometric mean is 1.359538^0.2 − 1 = 6.335492%, a gap of 1.664508 points. The population variance in decimals is 0.034520, so the shortcut gives 0.080000 − 0.034520/2 = 0.062740 = 6.274000%, which lands 6.15 basis points from the exact answer.

P3. Harmonic mean on a purchase plan. You invest ₹9,000 on three consecutive months at unit prices of ₹180, ₹150 and ₹225. Compute the average cost per unit and confirm which mean it equals.

Solution. Units: 9000/180 = 50, 9000/150 = 60, 9000/225 = 40, for 150 units against ₹27,000, so the average cost is ₹180.000000. The harmonic mean is 3/(1/180 + 1/150 + 1/225) = 3/0.016667 = ₹180.000000, identical. The arithmetic mean of the three prices is ₹185.000000, overstating the cost by ₹5.

P4. Annualising both ways. (a) A position returns 4.85% over 73 days. Annualise it correctly and by prorating. (b) A fund is up 9% in its first four months. What is its annual return?

Solution. (a) 1.0485^(365/73) − 1 = 26.719128% compounded, against 4.85 × 365/73 = 24.250000% prorated, so prorating understates by 2.47 points for a sub-year window. (b) It does not have one. It has a 9% four-month return, and annualising it to 29.502900% assumes the next eight months repeat the first four.

P5. Coefficient of variation as a ranking tool. Fund A has a mean return of 11.2% and a standard deviation of 14.6%. Fund B has 6.8% and 7.4%. Which is riskier per unit of return, and when does this comparison stop working?

Solution. CV_A = 14.6/11.2 = 1.303571; CV_B = 7.4/6.8 = 1.088235. Fund B is more efficient on this measure despite the lower absolute return, and the raw standard deviations point the other way. The comparison fails when a mean approaches zero, where the ratio explodes, and is meaningless when a mean is negative, where the statistic changes sign without changing meaning.

P6. Skewness and excess kurtosis by hand. A series of eight returns is +2.0, −1.0, +4.0, −6.0, +8.0, +1.0, −2.0, +3.0. Compute the mean, sample standard deviation, sample skewness and sample excess kurtosis.

Solution. Mean 9.0/8 = 1.125000. Sample standard deviation 4.223658. Sample skewness −0.115892 and sample excess kurtosis +0.495616, using the bias-corrected estimators S = [n/((n−1)(n−2))]Σd³/s³ and K_E = [n(n+1)/((n−1)(n−2)(n−3))]Σd⁴/s⁴ − 3(n−1)²/((n−2)(n−3)). Both are near zero, which at n = 8 is what you would expect whether or not the underlying distribution is normal: the estimators have enormous standard errors at this sample size and are being reported for practice rather than for inference.

P7. A normal VaR on a rupee position. A ₹5,00,00,000 book has a daily return standard deviation of 1.42% and a mean assumed to be zero. Compute the 99% and 95% one-day VaR.

Solution. z₉₉ = 2.326348, so the cut is 3.303414% and the VaR is ₹16,51,707. z₉₅ = 1.644854, so the cut is 2.335692% and the VaR is ₹11,67,846. Both figures are exactly right about a normal world and understate a fat-tailed one, which is why the breach count over the next 250 days is the number that actually tests them.

P8. The Sharpe ratio's standard error. Compute the standard error and 95% interval for a Sharpe ratio of 0.85 over 36 periods, then over 144 periods, then for a Sharpe of 1.60 over 36.

Solution. At SR = 0.85, T = 36: Var = (1 + 0.36125)/36 = 0.037813, se 0.194454, interval [0.468876, 1.231124]. At T = 144: Var = 0.009453, se 0.097227, interval [0.659438, 1.040562]. At SR = 1.60, T = 36: Var = 0.063333, se 0.251661, interval [1.106753, 2.093247]. Two lessons: three years of monthly data cannot separate a mediocre manager from an excellent one, and the standard error grows with the Sharpe ratio itself.

P9. Three corrections on one p-value table. Eight strategies produce p-values 0.0004, 0.0031, 0.0079, 0.0180, 0.0270, 0.0420, 0.0510, 0.1300. Apply Bonferroni at α = 0.05, Holm at α = 0.05, and Benjamini-Hochberg at q = 0.10.

Solution. Bonferroni threshold 0.05/8 = 0.006250, so 2 rejections (0.0004 and 0.0031). Holm compares against 0.05/8 = 0.006250, 0.05/7 = 0.007143, 0.05/6 = 0.008333, 0.05/5 = 0.010000: the first three pass and 0.0180 fails, so the step-down stops at 3 rejections. BH thresholds are (i/8)(0.10) = 0.0125, 0.0250, 0.0375, 0.0500, 0.0625, 0.0750, 0.0875, 0.1000; the largest rank passing is 7 (0.0510 ≤ 0.0875), so BH rejects 7. Same data, three answers, because the three procedures control three different quantities.

P10. The expected maximum from N trials. Compute the expected maximum Sharpe ratio under the null for (a) N = 25 trials over T = 60 periods and (b) N = 250 trials over the same T, and annualise both at √12.

Solution. With γ = 0.577216: at N = 25, Φ⁻¹(1 − 0.04) = 1.750686 and Φ⁻¹(1 − 1/(25e)) = 2.177673, giving a bracket of 1.997150; σ_SR = √(1/60) = 0.129099, so E[max] = 0.257831 per period or 0.893153 annualised. At N = 250 the bracket is 2.837677 and E[max] = 0.366343 per period, or 1.269048 annualised. Ten times the search raises the luck bar by 42%.

P11. McFadden, read correctly. A logistic model has a full-model log-likelihood of −388.4 and a null log-likelihood of −455.9. Compute the pseudo-R-squared and the likelihood-ratio statistic for the three restrictions the model imposes, and say what the pseudo-R-squared does not mean.

Solution. 1 − (−388.4)/(−455.9) = 0.148059. LR = 2(455.9 − 388.4) = 135.0000 on 3 degrees of freedom, against a 5% critical value of 7.815, so the model is overwhelmingly better than the intercept alone. The 0.148 does not mean the model explains 14.8% of anything: it is a ratio of log-likelihoods with no variance interpretation, and a value in the 0.2 to 0.4 range represents an excellent fit on McFadden's own scale.

P12. Two intervals from one regression. A regression on 30 observations has s = 2.40, x̄ = 12.0, Sxx = 96.0. Compute the confidence interval half-width for the conditional mean and the prediction interval half-width at x₀ = 16.0.

Solution. t(0.025, 28) = 2.048407. The leverage term is 1/30 + (16 − 12)²/96 = 0.033333 + 0.166667 = 0.200000. se_mean = 2.40√0.200000 = 1.073313, half-width 2.198581. se_pred = 2.40√1.200000 = 2.629068, half-width 5.385402. The prediction interval is 2.45 times wider, and at the sample mean itself it would be wider still relative to the mean interval.

P13. An ANOVA table from two numbers. A regression has SST = 940.0, SSE = 376.0, n = 40, k = 2. Fill the table and test the model.

Solution. RSS = 564.0; R² = 0.600000; df 2, 37, 39; MSR = 282.000000; MSE = 10.162162; F = 27.750000 against a 5% critical F(2, 37) = 3.251924, so reject. Adjusted R-squared = 1 − 0.400000(39/37) = 0.578378. Check F the second way: (0.600000/2)/(0.400000/37) = 27.750000, identical.

P14. Spearman without ties. Two rankings of six names give x = 5, 2, 8, 1, 7, 3 and y = 4, 3, 6, 1, 5, 2. Compute the rank correlation and test it.

Solution. Ranks of x are 4, 2, 6, 1, 5, 3 and ranks of y are 4, 3, 6, 1, 5, 2. There are no ties, so the shortcut is valid: Σd² = 2.000000 and ρ = 1 − 6(2)/(6 × 35) = 0.942857, which matches the Pearson correlation of the rank vectors exactly. t = 0.942857√(4/(1 − 0.888980)) = 5.659453 on 4 degrees of freedom, against a critical 2.776, so reject independence.

P15. Real, after everything. A fund returns 15.30% gross with a 2.05% expense ratio, in an economy with 5.9% inflation. Compute the net return and the real net return, and the error made by subtracting inflation instead of dividing.

Solution. Net = 1.1530 × 0.9795 − 1 = 12.936350%, a drag of 2.363650 points, more than the 2.05% headline fee because the fee is levied on a grown base. Real = 1.129364/1.059 − 1 = 6.644334%. Subtracting gives 12.936350 − 5.9 = 7.036350%, an error of 39.20 basis points in the flattering direction, which grows with both rates.

Applied mini-project

Deflate your own signal set. Budget about three hours. You will take a set of candidate signals, compute the naive best result, correct it honestly for the number of trials you actually ran, and write a one-page verdict. The count of trials is the deliverable that matters, because that count is an estimator input and you are the only person who can supply it.

1. Assemble the candidate set (~25 min). Use the signal set you built in the factor-construction node if you have done the quant track. Otherwise build a panel from twelve to twenty freely sourced return series over a stated ten-year window, six Indian and six US (R6), and define at least eight candidate rules over them: momentum at three lookbacks, a value proxy at two definitions, a low-volatility rank, two combinations. Record the source and pull date of every series.

1. The honest trial count (~15 min). Before computing anything, write down in a dated file how many distinct variants you will test. When you finish, write down how many you actually tested, including every parameter you tuned, every rule you discarded after one look, and every variant a colleague supplied. The second number is almost always larger, and the gap is the finding.

1. Naive results (~30 min). For each candidate compute the per-period mean excess return, standard deviation, Sharpe ratio with a stated risk-free rate and ddof, and the t-statistic SR × √T. Convert each to a two-sided p-value, and report the best candidate exactly as an uncorrected process would.

1. Correct for multiplicity (~35 min). Apply Bonferroni and Holm at α = 0.05, and Benjamini-Hochberg at a stated false discovery rate. Print the threshold ladder beside the sorted p-values so a reader can check the crossing point by hand. State how many survive under each, and which procedure you would defend to an investment committee.

1. Deflate the winner (~35 min). Compute the expected maximum Sharpe ratio under the null for your honest N and your T. Compute the winner's deflated Sharpe against that bar, with the skewness and non-excess kurtosis of its own returns in the denominator. Compute the minimum backtest length your winner's Sharpe implies at your N and compare it to the history you have. Keep every quantity per-period; never annualise before entering these formulas.

1. An interval, not a point (~20 min). Bootstrap the winner's Sharpe with a stated seed and at least 5,000 resamples, and report the percentile interval beside the delta-method interval from (1 + SR²/2)/T. Say whether it contains zero. Run the jackknife too, and name the observation whose removal moves the answer most.

1. The verdict (~20 min). One page, at most 500 words: what the best candidate showed, what survives each correction, what the deflated Sharpe says, how long a history the claim would need, and what you would do next. Name every place a number rests on an assumption rather than a source, and state your honest N in the first sentence.

Score it on twenty points, passing at sixteen with no zero on any row.

Any row whose computation cannot be reproduced from the numbers you printed scores zero, whatever the prose around it says. A verdict that reports a surviving candidate without stating the N it survived against has not passed, however good the analysis underneath.

Reading & resources

Core (do these).

• CFA Institute, Quantitative Methods readings on rates and returns, statistical measures of asset returns, and sampling and estimation: the curriculum treatment of the return and descriptive-statistics work here, and the source of the sample skewness and excess kurtosis estimators used here. [Paid] [Beginner→Intermediate]
• Bailey, Borwein, López de Prado & Zhu (2014), "Pseudo-Mathematics and Financial Charlatanism," Notices of the AMS: the minimum backtest length and the expected-maximum-Sharpe result, in twelve readable pages, free from the AMS site. Read it once and the deflation arithmetic stops feeling arbitrary. [Free] [Intermediate]
• Bailey & López de Prado (2012), "The Sharpe Ratio Efficient Frontier," Journal of Risk: the probabilistic and deflated Sharpe ratios derived properly, including the skew and kurtosis correction whose two algebraic faces are reconciled above. [Free preprint] [Advanced]
• Benjamini & Hochberg (1995), "Controlling the False Discovery Rate," Journal of the Royal Statistical Society Series B: the original, and much shorter and clearer than its reputation. The step-up rule is on the second page. [Paid] [Intermediate]
• Efron & Tibshirani, An Introduction to the Bootstrap: the standard reference for resampling, including the percentile interval's limitations and the block bootstraps for dependent data. Chapters 6, 11 and 13 are the ones to read. [Paid] [Intermediate→Advanced]

Going deeper.

• Lo (2002), "The Statistics of Sharpe Ratios," Financial Analysts Journal: the (1 + SR²/2)/T result and what serial correlation does to the annualisation factor. [Free/Paid] [Intermediate]
• Harvey, Liu & Zhu (2016), "…and the Cross-Section of Expected Returns," Review of Financial Studies: the multiple-testing problem applied to the published factor literature, with a proposed t-statistic hurdle above 3.0. The most consequential application of multiple-testing correction in academic finance. [Paid] [Advanced]
• Mertens (2002), "Comments on the Correct Variance of Estimated Sharpe Ratios": the skewness and kurtosis correction in two pages. [Free] [Advanced]
• Hosmer, Lemeshow & Sturdivant, Applied Logistic Regression: the reference for logistic regression, including why McFadden's pseudo-R-squared behaves as it does and what the Hosmer-Lemeshow goodness-of-fit test adds. [Paid] [Intermediate→Advanced]
• Conover, Practical Nonparametric Statistics: the rank methods in full, with the tie corrections spelled out. [Paid] [Intermediate]

Data and tools (free).

• NSE and BSE historical data, and AMFI NAV history (nseindia.com, bseindia.com, amfiindia.com): Indian price and NAV series for the mini-project, from the exchanges and the industry body themselves. [Free]
• FRED (fred.stlouisfed.org): US risk-free rate series, including DGS3MO and TB3MS, and inflation series for the real-return arithmetic. [Free]
• RBI Database on the Indian Economy (rbi.org.in): Treasury bill cut-offs and CPI, the primary sources for every perishable rate quoted here. [Free]
• Python: NumPy, SciPy and pandas: scipy.stats.skew and scipy.stats.kurtosis take a bias argument that switches between the population and corrected estimators, and kurtosis defaults to the excess convention while several finance formulas want the non-excess one. Read both docstrings before using either. [Free]

Do this, not just read: take any fund factsheet showing a "3-year return" and a "5-year return", and try to reproduce one from the other using the monthly NAV history the same site publishes. You will discover a compounding convention, a dividend treatment or a period-end mismatch inside twenty minutes, and that twenty minutes teaches more about return arithmetic than any chapter.

Teach it back & journal

Feynman prompt. Explain to a colleague who trusts spreadsheets why the fund whose factsheet says "average annual return 5.0%" turned ten lakh rupees into ₹9,60,400. Use the actual arithmetic: the four returns, the product 0.9604, the fourth root, and the variance shortcut that predicts the six-point gap before you compute the geometric mean at all. Then, without changing your tone, take the same colleague through the second half. A screen of 400 candidates hands you a winner with a t-statistic of 2.19, and the expected best of 400 pure coin flips over the same ten years is an annualised Sharpe of 0.94 against the winner's 0.69. If your colleague can, unprompted, ask you "how many did you actually try, including the ones you threw away in the first hour?", you have taught the discipline rather than the formulas.

Journal prompt. Find one number you have personally reported, or one you were given and passed on, that came out of a search: a screen, a parameter sweep, a set of comparable companies you narrowed, a model you refit after the first version disappointed. Write down honestly how many variants existed before that number survived, including the ones you abandoned in the first ten minutes and the ones somebody else abandoned before handing you the survivor. Then compute what the expected best of that many trials would have looked like under no skill at all, using sigma_SR = sqrt(1/T) and the Euler-Mascheroni bracket, and write the two numbers side by side. Most people find the exercise uncomfortable the first time, which is the point: the trial count is an estimator input, it is never in the data, and you are the only person in the world who can supply it accurately. Finish with a paragraph on what you will change in how you keep records, because the count is only recoverable if you write it down as you go, and reconstructing it three weeks later is guesswork wearing a number's clothes.

This module's flashcards and mastery quiz are wired into the app: see the node's Quiz and Reviews.