Updates July 6

PGrid

• started work on catalog images for products. this would take images downloaded from retailers and pick representative images for each product in the database
  • set up worker task and added a cli to process catalog selection jobs
  • got ai to select images for each product
  • iterated on image selection prompts, image selection can use embedded codex sdk
• cli updates to control prod feature flags by agents
• worked with gpt5.5 to make some small improvements on seo to product pages. removed articles for now. updated sitemap generation. updated description generation for product pages
• did some performance updates to postgres after investigating usage with fable. increased cache size and overall memory available to postgres. optimised snapshot index + cleaned up redundant index
• upgraded my codex account to be pro 20x version. hoping to use codex for specs search to speed up getting laptops up and running
• continue work on laptops
  • updated laptop variant specs search agent mode to make use of learnings from laptop model agent mode
  • added production specs search worker runner support, including prod api_long worker wiring and category/type routing for queued specs search jobs
  • added supervised classification runner work for longer overnight prod runs, with preview planning, exponential backoff and stop-on-failure handling
  • tuned laptop model specs search dedupe after checking prod queue keys; tightened prompt rules around CPU/platform labels and added brand examples
  • cleaned up/restructured BAML prompts and fixed stale BAML lint/test issues after the restructure
• switched BAML calls used by classification and specs search over to embedded codex sdk. this makes it easier to use codex for more of the actual runs instead of only using it for agent mode

• added more cli controls for agents running classification/specs search. can confirm runs, reset stuck queues, clear/search queues and use llm to pick representative items for dedupe
• updated laptop model and variant specs search agents more. fixed model no-op outcomes, added gpu specs search agent mode, require cpu release dates and ignore reseller modified laptop variants
• added scheduled classification guard + queue cleanup. should help with longer supervised runs and cases where an llm failure leaves things in progress

• fixed the sitemap build/deploy issue. sitemap now generated from astro output
• started work on audit-only keepalive browser daemon lifecycle work for faster crawler audits and future reused-browser crawler development
• continued iterating and ingestion of reddit build a pc knowledge base wiki
• playing around with claude fable to design some better description pages for products making use of our knowledge base. tried to implement but didnt. not happy with implementation from gpt 5.6.
• did more work on browser debugging for crawlers
  • unified browser debug capture so it can save screenshots, timeline and crawler events for supported prod tasks and local audits
  • added cli commands to inspect browser debug sessions from workers. added dashboard/worker read routes as well
  • added more detailed events for ebay, including navigation, comparison/verification and final crawler outcome. also enabled distributed amazon/ebay browser debug testing