Updates Mar 10

PGrid

• Continuing work on removing GPU specific price snapshot tracking
• While converting gpu price snapshots to generic snapshots, faced very slow postgres transactions while having to write around 2m rows. Decided it would also be a good time to try offload time series data writing and queries to clickhouse. Starting that process
  • Got local clickhouse running
  • Initial schema creation for item snapshots
  • Basic migration and migration tracking tool for clickhouse
  • Basic replacement queries for cpu generic item data
• Due to migrating gpu price snapshots into generic snapshots, that increased the postgres db by nearly double since we have duplicate data now temporarily. Updated the postgres backup script running on a vps to use incremental backups introduced in pg17. Had some issues related to pg_hba.conf files, then updating the script to run incremental backups properly. Not using yet due to bugs while running