torielstrom/hire-tori

This project is a portfolio piece built as a job application. Rather than writing a resume for a developer marketing role focused on AI observability, the author built a real working feature and instrumented it with the observability tool PostHog to show what the product can do in practice. The feature itself solves a real problem from the author's own Pokemon trading card app. A card scanning tool could already tell which card was in a photo, but not its finish, meaning whether it is a normal card, a holo card with foil over the artwork, or a reverse holo card with foil everywhere except the artwork. Finish matters because it changes the card's price. The project builds a vision based AI classifier that looks at a card photo and decides its finish, using Claude's vision capability. Every time the classifier runs, that call is recorded as a trace inside PostHog's AI observability feature, capturing the image sent in, the model's reasoning, how many tokens were used, how long it took, and what it cost. Along with the classifier, the project includes an evaluation script that runs the classifier against a set of real, previously labeled card scans, scores its accuracy, sends the results to PostHog, and writes out a report showing a confusion matrix and every mistake the model made along with its reasoning. There is also a version of the classifier written to drop directly into the author's existing production scanning pipeline. The README explains that reverse holo cards are the hardest case to classify correctly, since the foil cue can be faint in a low light photo, and that this kind of failure is exactly what AI observability tools are meant to surface. The author is upfront that running a large language model on every scan would not be practical at full scale, and describes this project as the first step toward training a smaller, cheaper model using this one as a teacher and a fallback for uncertain cases.