Updates Jan 27
PGrid
- Started on a new frontend for ssd classification that just replicates a backend version I wrote but allows viewing outputs as its running and adds manual buttons between steps. This is basically a rewrite of how the classification happens and is taking a different approach.
- Current issue with classification is populating the database with accurate data. My current pipeline searches through all SSDs (just using LLMs to filter) and then I tried to get a new step where based on the resulting titles it would use LLMs to create different variants of an SSD. But this was not accurate. Did some different experiments on this.
- Added new gpu to database: 5070 ti
- Got ai to rewrite 50 series card descriptions. Trying to make it useful but still not require manual research. Using search based LLMs can include performance data. Manually generated and partially edited the descriptions for the 50 series cards.
- Noticed that there have been some ebay listings that have changed what they list from a gpu card to a laptop. Keeping the same URL this is pretty annoying to deal with as now the gpu page incorrectly lists laptops. Added an ai that goes through all cards for each gpu chip and finds incorrect listings.
- The review ai isn’t super accurate so had to do a frontend to manually delete items it picked
- For 5080 and 5090 launch, the classification took a while but ran successfully. I had to fix the scraper for PLE as it didn’t save out of stock status.
- Added new button and functionality to get alerts when stock becomes available
- Did some cleaning up of private docker registry. Previous attempts at deleting tags left stuff dangling and when building, all layers weren’t pushed resulting in missing layers when trying to do docker pull. Instead of debugging I just deleted the entire thing on the filesystem and rebuilt specific images.
- Updated all browsers to use a library called patchwright due to some issues with cloudflare blocking some sites. Had some issues with deployment as had to move browsers to not use headless.
- Added one server and removed one server from nomad cluster
- Implemented a discord alert bot that will notify when specific 50 series gpus are in stock for each store.