Updates May 12

PGrid

• Bought pgrid.app domain as the international version of the website
• Amazon emailed about price tracking not being allowed which could result in amazon affiliate account being cancelled.
  • Replied to this giving points on how price history is important to a consumer and other larger websites have same data but unknown status
• new function to manually delete an incorrect gpu price snapshot and have it propagate to generic items and clickhouse. another new function to manually fail a gpu and delete corresponding item snapshots
• Issues with ebay scraper resulting in job queue getting out of control. added plot in grafana for this
• Reintroduced killing the process after ebay scraping issues but this time using a global variable so that job metric gets written and job gets marked as completed
• Updated postgres, clickhouse to include regions in item and snapshot data
• Updated queries for clickhouse and some queries for postgres that are used for price alerts to take regions as an argument and default to australia
• Refactoring db tasks of migrating gpu data to generic items to the shared folder so that it could be called by frontend build before doing frontend build
• Added a new page for dashboard to run clickhouse commands
• Started migrating frontend builds to use clickhouse queries for pages starting with the main page with gpu and cpu tables. This didn’t result in the build speed increase I expected as the price history queries are quite computationally expensive. Will need to implement caching