Updates February 23

PGrid

• New email alert functionality that uses ClickHouse, works across all categories and regions, and is configurable per category and source. Enabled for GPUs only for now.
• Fixed a bug with prod-to-local ClickHouse sync not rebuilding MV tables, which resulted in missing data locally.
• Work in progress to generate marketing static images. Happy with the current style.
• Removed the Mantine dependency and recreated the price alert button as part of frontend cleanup.
• Posted a slideshow on Instagram. This ended up getting 0 views.
• Issue with Google search result description: Google didn’t use the meta description for some searches. Updated the front page to hopefully address this in future updates.
• Silently launched the US version of PGrid. The frontend now generates a lot of pages every build; this has been reduced to every 10 minutes (from every 9 minutes). Investigating ways to make this more efficient.
• Started planning out a single Postgres table for all events related to item updates. This will allow dynamic updating of the frontend instead of rebuilding the entire frontend every x minutes. Should also help with ClickHouse syncing and price alerts being more efficient. This was after finding a bug with a ClickHouse materialised view not updating after an item was marked failed in Postgres.
• More work on the Best Buy scraper. Their bot detection seems strong; results in listings not showing or next page links going to error pages.
• Upgraded packages throughout the project, including Next.js. Haven’t updated packages in a while.
• Progress on DuckDB as cold storage for snapshot data, and as a method to fast full sync between Postgres and ClickHouse, and to help Postgres efficiency.
• New account on TikTok. Posted 3 things; 2 generated on TikTok following what other computer parts stores did that went viral. It was just asking “should I buy x GPU or y GPU”. Then posted the same RAM price history charts as I did on Instagram. All 3 posts got around 700 views. No new followers yet.
• Added a Gemini CLI bridge so I can use Gemini CLI from BAML to do classification.
• Fixing issues with SSD specs search where it isn’t as detailed with verification as other categories, and doesn’t use shared utils as much. While doing this I found all categories could be better written and share patterns better. Fixed a few things but not all. This needs to be more aligned so it’s easier and more reliable to create new categories.
• Work in progress on a line graph video generator script to create line graph videos.
• Added coupon codes to email alerts if the sale has a related coupon code.