Updates September 1

PGrid

• new cli command to show proxy status
• updated proxy controls and feature flags to define provider for vpns as well and use that when refreshing connections
• updated proxy service to have independent uptime calcs, idempotent iptable rules for nordvpn and show version numbers
• added a new verification step for specs searches on ssd variants. this currently only runs manually but takes ssd variants that haven’t run through a verification step and runs specific important specs of that variant through perplexity to double check.
  • updated logging to write to files and attempted to get claude to review verification with a slash command. not sure how effective it is yet.
• had some issues getting PureVPN of proxy service working on Nomad deployments. had to update docker plugin permissions for nomad for all servers to allow NET_RAW.
• migrated the entire proxy service code to the main PGrid repo. Before it was in the Nomad repo and now in the PGrid repo it can use shared code so after the proxy service is deployed it can automatically connect to the correct region for each deployed service.
• more refactoring of the generic item pipeline and related files. There were some Australia-specific code that needed to get removed and all scrapers needed updates to return metadata saying items are AUD and from Aus region.
• from doing updates to the generic item pipeline I found that the eBay search scraper was returning no results and that the HTML structure for eBay search results seemed to have changed a bit. Did some debugging and fixed this
  • other minor clean up of eBay scraper done
• still have an issue with Amazon scraping where the proxy isn’t picked up as being from Australia by Amazon so all results are shown out of stock. Hack is to send a link by Discord when GPUs go out of stock in Amazon to delete recent snapshots
  • got Claude to do a workflow and test script which checks each Australian proxy in our system, takes a screenshot of a test search and determines if Amazon thinks the request is coming from Australia. Turned this into a prompt so Claude can check more frequently till I find an issue proxy.