Updates August 31

PGrid

• launched laptop pages publicly
  • iterated a little bit on what gets shown in the main laptops table
    • added a nullable dedicated/integrated gpu kind to laptop specs search, then completed an evidence-backed production review of all 1,894 variants across 431 models. 1,812 variants changed through 2,490 gpu kind, gpu name, vram, screen type and weight updates.
  • curated an initial ten-laptop recommendation set from live Australian pricing and reviewed catalog images, covering mainstream, budget, gaming, mac, arm and 2-in-1 options
    • added laptop deal finding agent support
      • this included read only command to run the deal finding agent
• work in progress on updating all category pages to use new design
  • extended the two-column gpu detail design to cpu, ssd and ram pages, pulled shared detail pieces together, and enabled reviewed gpu and cpu image galleries. this is still on a separate work-in-progress branch
  • explored buyer-focused content for gpu pages around market position, longevity, buyer fit and upgrade value. not implemented yet
• started working on adding support for monitors on frontend
• developed a new advice-article system and pseo direction
  • built the shared style guide, article components, full-build refresh support and an article-authoring skill, and drafted eight data-backed buying guides. none published yet
• improved classification and production catalog data
  • gave the ssd classifier and reviewer known part-number aliases, while keeping unmapped sku-only listings out of specs search
  • started using the enriched data to replace the laptop table’s configuration count with weight, add panel types, and filter dedicated versus integrated graphics. this is staged work in progress
  • retired an Amazon ram url that had redirected from a 64 GB kit to a different 32 GB product, deleted its one incorrect snapshot and preserved the earlier 337-price history
• improved catalogue image review and production coverage
  • taught Luna cutout review to inspect black backgrounds and edge crops, reducing missed halos and clipped products
  • completed the 1,429-image monitor cutout campaign: repaired 714 images, accepted 1,425, and removed the four unusable records whose source images could not be safely repaired
• investigated crawler reliability
  • found Officeworks listing crawls were failing around 97% of the time, mostly from AWS WAF blocks and repeatedly reused proxy exits. also removed an overly broad error-banner check that rejected otherwise populated pages
  • simplified Officeworks readiness and pagination, switched Australian crawling to PureVPN and tested a slower five-second page turn. this didnt fix properly but once in a while laptops on officeworks works
  • traced six hour-long Newegg and B&H jobs to container memory limits killing Chromium, followed by browser cleanup hanging until the worker watchdog fired. the underlying memory growth is still being investigated
  • added worker process and cgroup telemetry for memory usage, pressure, task counts and oom events so the next incidents can be measured directly
  • added support for laptop crawling on harvey norman
  • fixed Mwave readiness when the first products have no prices
  • fixed duplicated Amazon search titles, normal pagination-limit shutdown and disabled JB Hi-Fi pagination handling
• improved proxy operations
  • added regional provider controls and advisory warnings to the dashboard and cli, and switched the Australian crawler fleet to PureVPN
  • launched a public access (with password) instance of proxy service for a friend to use
• investigated observability and worker memory limits. increased memory for a few workers