Procurement data pipeline

what I built

Collector scripts that backfilled public tender data from Western countries, enriched UK records with end dates and buyers, and loaded everything into Postgres. An open-weights model on a European supercomputer tagged contract text.

Python · Postgres · cron · public procurement APIs · LUMI supercomputer · vLLM · Qwen 2.5

what happened

• 11,865,512 contracts from 41 countries in the database on 19 October 2026.
• An audit found that only UK data has real end dates. 0 of 10.39 million award rows from other countries had one.
• 3,437 UK IT contracts expire within 6 to 18 months, across 890 buyers.
• A parser fix raised UK end date coverage on a 500 row sample from 11.8 to 41.8 percent.
• The supercomputer test tagged 300 rows with 0 errors. The UK batch loaded 550,245 of 550,278 rows.

what broke

• Supercomputer access expired before about 3.4 million European rows were loaded, and that output was lost.
• A text edit wiped the server's job schedule once. It was restored.
• The agent killed its own SSH session 3 times.

decision

The data works, but tracking contract expiry is a commodity. The collectors are paused until there is a customer for the data.