About Color-Fi. Color-Fi is filtering software to allow the creation of custom image filters based on the contents of the image. The main feature of Color-Fi is an analysis tool for picking out prominent colors from the image (weighted towards the brightest/most saturated colors) which then get used to generate a complementary color palette for use with filters.
Snow App Filter Demonstration with Loretta Stokes and me.
Before Color-Fi. I use the Snow app, which was once a fairly obvious clone of Snapchat, but with much stronger filter capabilities. Now, it only provides photo and video editing/filters. At the time, I was very interested in how it would do complex face detection (though this now pretty standard in most photo filter apps), picking out features of your face and adjusting them with contacts, eye-shape changing, adding makeup, and different backgrounds. I wondered if, in addition to putting cute cat ears or whatever, I could do ones that would match your outfit, eyes, or hair. As a first pass, I wanted to figure out what would be required to generate colors that match. Adobe Color has a similar feature, and so I wanted to reverse engineer some of the more simple color manipulations used by that app. I made this Color-fi project two years ago, since then Snow has also added color-reactive filters like what I describe. (Arguably, executed much better.)
An example output of Color-Fi before 2019.
How Color-Fi Works. Color-Fi allows the user to input two photos, a base photo, which will have a filter applied to it, and a graphic photo, which should be black and white. Color-Fi will do analysis on both photos. It does random sampling to pick up a number of pixels in the base photo, and picks the most saturated one. (There’s a way to calculate saturation mathematically, which I learned about during this project. This is how you can systematically boost saturation in photo-editors.) Then, taking that color as a base, it generates four additional colors which look nice with your chosen color. These new colors are the filter colors. Color-Fi uses flood fill to find the distinct regions in the graphic photo and apply a random filter color to each of the shaded regions. Then, it applies the graphic photo over top of the base image and tints the previously black portions of the image towards the filter colors and displays your new custom filter. With the randomness involved, you get different results even when you use the same base image and graphic image, but all combinations look quite nice. (Though, I’m sure you could somehow generate something that doesn’t look nice.) Prior to 2019, Color-Fi would select a random color of the five generated ones, and apply that to all of the shaded areas, but recently I wanted to use all five colors that were generated, so I made a small update to allow for this.
An example of Color-Fi after the 2019 flood fill update.