Two sources, one number: stock for 13 branches, reconciled before anyone acted on it.
Stock by branch and SKU for every store, every day — the number branch staff use to answer a customer standing in front of them.
- 99.4%
- agreement with the old pipeline across 2,620 SKU-branch pairs
- 13 of 13
- branches reached per run, up from 11
- 17 days
- of lost stock data backfilled in 82 seconds
Problem
The only way to see stock across branches was the vendor's retail app. My first pipeline drove that app inside an Android emulator: tap, dump the screen, parse, repeat — 7–20 seconds per SKU, 50–70 minutes per run, and it needed the emulator open, the app logged in and nobody touching the machine. Over about five weeks it succeeded on four days, and it failed silently, so 5–21 Aug 2026 has no stock data.
Constraints
- No engineering team, no budget: one office PC on Windows Task Scheduler.
- The vendor portal is shared with other pipelines through a single login session.
- Staff act on the number on their phones — a stale value that looks fresh is worse than none.
What I built
The vendor's partner portal exposes the same stock report through a JSON endpoint. I found it by watching the portal's own network calls; the work was everything around it. The new pipeline calls the endpoint with the existing session: about 3,000 rows for all 13 store codes, 17 seconds for the call and 25 seconds for the whole round, with fields the app never showed (SKU ids, days-of-stock, in-transit quantities) and any past date on request. The screen-automation path stayed as an automatic fallback, selected by exit code, and a file lock serializes the six scripts that share the portal session.
How I verified it
Both pipelines ran on the same day and every SKU-branch pair was compared: 2,603 of 2,620 matched exactly; the 17 that differed were 1–2 units lower on the portal, consistent with items sold during the hour-long sweep. Only then did the scheduled run switch engines.
What broke afterwards, and what changed
- Silent partial writes. An incomplete pull could overwrite good data and delete branches it had not pulled. Rule now: an incomplete pull writes no files; pushes are upserts followed by a sweep, never delete-then-insert.
- Looping day by day died from day two — the browser profile stayed locked. Multi-day pulls use one browser and a date range.
- Stale files looked like success. Scheduled runs now force a fresh pull instead of skipping days that already had a file.
- Freshness meant the wrong thing. The page showed when we pulled, not when the portal's data was updated. The label now says "yesterday's stock, 23:00" in words.
