Projects:Sketchy recognition

Nicolas Malevé, Michael Murtaugh

Sketchy recognition

Bread, Nose, Kangaroo or Teddy Bear?

A photograph from the collection of the Museum of Musical Instrument is processed by a contour detector algorithm. The algorithm draws the lines it found on the image sequentially. While it is tracing the contours, another algorithm, a sketch detector, tries to guess what is being drawn. Is it bread? A kangaroo? It is a teddy bear.

Sketchy Recognition (working title) is an attempt to provoke a dialogue with, and between, algorithms, visitors and museum collections.

Cast:

• Musical instruments: MIM collection, Brussels.
• Line detector: The Hough algorithm in the OpenCV toolbox, originally developed to analyse bubble chamber photographs.
• Sketch recognizer: an algorithm based on the research of Mathias Eitz, James Hays and Marc Alexa (2012), and the code and models by Jean-Baptiste Alayrac.
• Data: from the hands of the many volunteers who contributed to Google's Quick, Draw! Dataset.
• Special sauce, bugs and fixes: Michael and Nicolas

Sketches

Early sketch

Contours of the Musical Instruments database

Sketch recognition outcomes

(Re)sources

• Code for this project
• You were asked to draw an angel, Working notes from the Scandinavian Institute for Computational Vandalism (April 2017)
• Assisted drawing, Working notes from the Scandinavian Institute for Computational Vandalism (January 2016) + Assisted drawing: Exploring Augmented Creativity, original blogpost by Samim (December 2015)
• How Do Humans Sketch Objects? and C/C++ implementation, Mathias Eitz, James Hays and Marc Alexa (2012)
• Python/Jupyter https://github.com/ajwadjaved/Sketch-Recognizer
• sketch-recognizer, Jean-Baptist Alayrac's working Python code that we ended up using

Collection: Musical Instruments Museum (MIM)

Reconnaissance esquissé

[translation FR]

Schetsmatige herkenning

[translation NL]

Working sketches + notes (not in publication v1)