Building a 7-Factor Equity Ranking Pipeline Over 1,353 NSE Stocks, Refreshed Weekly

The problem with most screeners

A stock screener is easy to build badly. Pull some fundamentals, normalize each metric to 0–100, average them with weights, sort descending. The result looks authoritative and is quietly dominated by data errors — because in a universe of thirteen hundred stocks, the extreme values that top your ranking are disproportionately the wrong ones.

The system described here started that way and had to be rebuilt around a different assumption: every input is suspect, and the ranking's job is to be robust to that, not to trust it. Four of the five hardest problems turned out to be data-quality problems rather than modelling ones.

Universe construction

The universe is every NSE equity-series stock with a market capitalization of at least ₹1,000 crore: 1,353 names. Building it from free sources means reconciling several that disagree.

• Screener.in lists roughly 1,530 comparable names. The 177-name gap decomposes into about 97 stocks where the price source returns a blank market cap, plus NSE non-EQ series instruments and BSE-only listings.
• That gap is documented rather than silently absorbed — an unexplained universe difference is the kind of thing that quietly invalidates every downstream comparison.
• yfinance handles the .NS ticker suffix for price and fundamentals; nselib reaches NSE and NSDL for delivery and ownership data. Screener.in is UI-only with no API and terms that forbid copying, so it is used for manual cross-checks and never scraped.
• The whole stack is keyless. No vendor account is required to reproduce a run, which was a hard design constraint.

Three data layers, then a rank

Rather than one monolithic fetch, the pipeline is four scripts that each write a CSV, so any stage can be re-run independently and the ranking step touches no network at all.

1. Fundamentals and price — valuation, quality, growth, analyst targets, moving averages, EV/EBITDA.
2. Ownership overlay — per-stock delivery percentage as a conviction signal, bulk and block deals by client name, and NSDL FPI flows as market-regime context.
3. Shareholding — promoter percentage, institutional percentage, and institution count.
4. Ranking — merges the three CSVs, normalizes, applies weights, writes a scored 0–100 table. Reads local files only, so it runs instantly and can be re-tuned without refetching anything.

That last property matters more than it sounds. Separating fetch from score means weight experiments cost seconds instead of an hour, which is the difference between tuning the model and never touching it again.

The normalization bug that mattered

The first version used min-max normalization: map each metric's range onto 0–1. In a clean dataset this is fine. In a real one it hands the entire scale to whichever row is most broken.

One stock in the universe reported a return on equity of 276% — an artifact of a distorted equity base, not a business fact. Under min-max, that single value compressed every genuine ROE in the universe into the bottom few percent of the scale, so the quality factor stopped discriminating between a 28% ROE and a 9% one. The model was, in effect, ranking on one bad cell.

• Switched to percentile rank: a stock's score on a metric is its position in the universe's distribution, so an impossible outlier occupies exactly one rank slot and distorts nothing.
• Added sanity guards: non-positive P/E and P/B map to neutral rather than to 'infinitely cheap', which is what a negative earnings figure looks like to a naive cheapness score.
• Score dispersion went from clustered-and-meaningless to a 28–66 spread across the universe — a distribution you can actually reason about.

The general lesson transfers to any scoring system over messy inputs: prefer rank-based transforms to range-based ones. Ranks are bounded by construction and cannot be hijacked by a single row.

Coverage guardrails: refusing to score

Free data sources have holes, and holes are not evenly distributed — they concentrate in small caps, which are also where a screener is most tempted to find hidden gems. A stock missing half its fundamentals will happily receive a score, and that score will be mostly noise wearing a number's clothes.

The fix has three parts. First, a targeted enrichment pass runs only on stocks below 75% weighted coverage or missing core quality and valuation evidence, filling blanks from NSE daily P/E, BSE company headers, and specific Yahoo statement endpoints. In the live pass this recovered 420 fields — 12 from NSE, 98 from BSE, 310 from Yahoo.

Second, final scores are shrunk toward neutral in proportion to weighted coverage, so a thinly-covered stock cannot reach an extreme rank on three data points. Third, and most important, stocks below 60% coverage or lacking core evidence are shown but left unranked. After enrichment, 29 of 1,353 stocks stayed unranked — a deliberate refusal rather than 29 fabricated scores.

That last rule is the one I would port to any ranking product. A system that always produces an answer teaches its users to trust answers it should not have produced.

Accounting artifacts are not business decline

A worked example of why a purely mechanical model needs human review. ITC's hotels demerger, effective January 2025, put a one-time exceptional gain in the year-ago base. The data source therefore reported earnings growth of −72.7%, which collapsed the growth group to a score of 7 and dropped the stock to rank 260 of 1,353.

Nothing about the business had deteriorated; the comparison base was inflated by an accounting event. Re-scoring with only the two distorted fields neutralized — keeping genuine revenue growth of −5% and forward EPS growth of +7.7% — lifted the growth group from 7 to 30.4 and the stock to rank 81, a move of 179 places into the top 6%.

The instructive part is the nuance: even corrected, growth landed in the 20th–30th percentile. A slow-growing mega-cap staple genuinely is below median on growth in a universe full of fast small caps. The model was right to dock it, just not to bottom-decile it. Corrections should remove the artifact, not argue for a conclusion.

Making it run itself

A screen you have to remember to run is a screen you stop running. The refresh is scheduled weekly through a macOS launchd agent that catches up missed jobs on next wake, and the interesting engineering is in what it does besides refetching.

• A diff engine compares the new ranking against the previous snapshot and emails only material changes: top-25 entrants and drop-outs, price moves beyond ±15%, rank moves beyond ±25 places, score jumps beyond ±5, drawdowns deepening by 10 percentage points or more, and delivery-trend flips between accumulation and distribution.
• Thresholds live in one block at the top of the file, because a digest that fires on everything is a digest you filter to trash.
• Snapshots are written to a history directory and a rank tracker diffs the current ranking against both the staged Git index and origin/main, so movement is attributable to a specific run.
• Newly ranked and newly unranked stocks get transition labels instead of fabricated numeric deltas — you cannot move 300 places if you did not have a previous place.
• Credentials for the mail sender come from a gitignored environment file; nothing authenticating is committed.

Cutting the refresh from 61 minutes to under 5

The original weekly run took 60.7 minutes across 4,059 stock-level network loops. Profiling it produced a clean breakdown, and the fix required no paid API.

Three of those `sleep(0.5)` loops alone accounted for 33.8 minutes of pure waiting before any network latency. The rewrite attacked structure rather than pacing:

• The delivery stage made 1,353 per-symbol history calls when NSE publishes one whole-market file per trading day. Fetching roughly 20 daily files replaces 1,353 calls with about 20, cached by trade date.
• Price history moved to batches of 100 symbols, cached locally, with later runs appending only a short overlap window; current price, 52-week levels, moving averages, and one-year change are then computed locally instead of pulled per symbol.
• Fundamentals are retained for 30 days — they update quarterly, so refetching them weekly was pure waste — and refreshed concurrently with six workers, with a flag to force a cold refresh.
• The separate shareholding pass was deleted entirely; those percentages were already present in the fundamentals response, and only institution count needed the second path.

Steady-state runtime is now 1–5 minutes. The orchestrator was also hardened to stop continuing past a failed stage, which previously risked publishing a ranking that silently mixed fresh and stale data — a worse outcome than failing loudly.

What the system surfaced

Findings from the full-universe run, included here as evidence the pipeline discriminates rather than as recommendations. Scores are universe-relative and only comparable within a single run.

• Top balanced names at the time of the run: GESHIP, EXPLEOSOL, VENUSREM, VSTIND, HINDZINC, THYROCARE, OBEROIRLTY, COFORGE, NMDC.
• Expanding the ranked universe from 40 to the full 180-name subset, and then to all 1,353, surfaced names no smaller screen reached — the breadth is the point, since a screen over the usual large-cap list mostly rediscovers the usual large caps.
• A useful validation: EXPLEOSOL and GARUDA independently topped a separately-provided 50-stock list, which is weak evidence but the right kind — agreement across two disjoint inputs.
• A value trap the model caught: a stock 56% off its high with a falling delivery trend scored low. Cheapness plus a deteriorating conviction signal is the pattern the smart-money layer exists to separate from a genuine dip.

The output is a self-contained interactive dashboard — vanilla JS and SVG, no CDN, works offline — with a valuation-versus-quality scatter, sector scan, sortable table, and a per-stock breakdown of all seven factors. It includes a drawdown scenario slider that estimates a buy price as current price times (1 − beta × drawdown), with beta clamped to a sane range and missing betas defaulted, because an unclamped beta produces absurd targets on exactly the illiquid names where beta is least reliable.

What I would build differently

• Quarter-over-quarter shareholding change is the biggest remaining gap. Free sources give a snapshot, and a snapshot cannot distinguish accumulation from a position that has been static for years.
• The FII/DII split needs a real feed; the free NSDL path is stale enough to be context-only.
• Value-trap detection would improve most from promoter pledge percentage, debt trend, and interest coverage — the three things that most reliably precede a cheap stock getting cheaper.
• Point-in-time storage. The current design overwrites the ranking each week and archives snapshots, which supports diffing but not honest backtesting of the factor model itself.