Another project I worked on while at RC was building a set solver with nicolas hahn. His writeup is more detailed than I can be, but in short: we used a series of computer vision algorithms to locate set cards in a photo, then identify them and determine where the sets are.

Set is a popular card game that exercises spatial and visual recognition by requiring you to find 3 cards that fit a particular set of criterion. We wrote a computer algorithm that can find sets on a given board faster than a normal human.

The major portions were:

• Using rectangle detection with hough lines to locate potential rectangular shapes and then transforming them into proper rectangles
• Using feature detection and heuristics to identify which cards we were seeing and what color / fill they were
• Outlining all potential set matches we saw

The project was a lot of fun - nick setup some cool pipelines that we used to continually improve accuracy and we ended up with something like 90 or 95% accuracy rate across all the tested photos. I can’t remember how many hours I sat and fiddled with heuristics, but it was all worth it.