Updates Oct 7

Fujifilm PGrid

• Progress on amazon scraper for fujifilm. Will be done a slightly different and hopefully more efficient way compared to PGrid with the finding and matching amazon products to lenses and cameras in db happening in one step.
• Ai classification done alongside scraping. A 3 step ai classification seems to work well:
  • first step gets the ai to extract details about the lens like focal length and aperture based on the listing title
  • second step checks if extracted details match with a given lens
  • third step reviews what the second step did
  
• Work on schema to handle camera and lenses, classification of listings to those cameras and lenses and price snapshots of those listings.
• Moved away camera stuff from PGrid database
• Work in progress on amazon scraper to search from international stores. Goal is to have Aus, USA and Germany and/or UK to start with.

PGrid

• Saw on umami there was a referral from arstechnica.com, found this post on their forum. So cool to see stuff like this. Sad that old school forums like that are mostly gobbled up by reddit/fb.
• There was more issues with the price alert email logic. Triggered by the ebay scraper on the rare occasion getting the wrong coupon discount amount. Issue was that the safe guard percentage check was based on the user provided alert price instead of checking the previous card price. Fixed that.
• Added another price drop check function that runs before saving a price snapshot. Specifically for ebay, a second scrape job will be queued if there is a price drop greater than 10% and the first price snapshot wont be saved.
• Decent refactor of functionality to check lowest price for a given gpu chip. There was 2 different queries that did almost the same thing for alerting me on price changes and for the price alert emails. Updated both to use a shared query. The same query also gets used before inserting ebay price snapshots. Added an index to db to make the query a bit faster.
• Added a new function to send manual custom emails. Was used to send a manual email to some people who subscribed to a specific GPU when there was a suspiciously cheap price for a GPU on amazon but a comment said they actually shipped a different GPU.

Other

• Tiktok experiment posting Micallef Program sketches seemed to be very popular. Later in the week a few of the post got marked as “low quality” and didn’t get promoted to peoples feeds. Hopefully can recreate a tiny amount of that success with fujifilm pgrid content.