Updates Nov 18

PGrid

• Updating price alert functionality to handle CPUs as well as GPUs. Due to the schema structure being different for CPU and its price history vs GPUs needed to do a decent refactor. Should work in the future for all different categories.
• Added Scorptec CPU scraping
• Ran CPU classification on the remaining ~200 cpu listings from different sources. A lot of the CPU listings are for CPUs that I don’t have details for in my database so may need to add more CPUs. But its a balance between having a useful number CPU models to choose as I don’t want to show every possible CPU model for sale.
• One server was getting captchas from amazon, added a log and timeout when theres captchas for amazon.
• Added email review page and email queue. Emails will get queued for review if theres a price decrease > 5%
• Added deployment for email review page which runs on nomad
• There were some deals on ebay, posted on ozbargain and facebook but not a lot of traction on both. But still best week for sales.
• Some progress on getting generic item search and price update pipeline running for ebay and amazon where it searches for classified items like CPUs and updates the price. So far only amazon searching for CPUs is implemented.
• Fixed bug with email price alerts where if someone unsubscribed and then resubscribed for the same product, it would keep sending them emails and incorrectly update their old unsubscribed record in the db.
• Fixed bug with ebay incorrect price detection where out of stock items would still get saved if an incorrect price was scraped. Source of random incorrect prices still not found.
• Fixed bug with nomad worker deployments where it would kill workers even if it was running a job. Now waits for jobs to complete before deploying new versions.
• Fixed bug with workers accruing zombie processes over time. Adding an init process (tini) which cleans this up.