A New Level to Visual Searches
I am definitely not much of an artist as my drawings are at best simplistic, but typically, they are recognizable; at least for another human, but what about a machine? Perhaps not my scratches, but researchers at Brown University and the Technical University of Berlin have created a piece of software that can actually recognize sketches. The paper about the software is available online now, along with other materials.
This software was able to accomplish this quite difficult task through the use of machine learning algorithms and pre-existing computer vision datasets. To start off, the researchers only considered objects in 250 categories, so the software does not have a large vocabulary, but it achieved 56% accuracy after learning from some 20,000 sketches that were crowdsourced into those categories. While that accuracy rating may not seem very high, humans were only able to achieve a 73% accuracy rate when asked to identify sketches in the same database.
With more sketches to learn from, and more categories to fit things into, this software could eventually be used to improve visual search engines. Collecting the data for that could be a challenge though, but the researchers recognize that they may be able to make a smartphone game, to crowdsource new sketches and categories for them. It may be awhile, but eventually we may see a computer beating humans in a game of Pictionary.