Created by Richard Vijgen, ‘Through Artificial Eyes’ is an interactive installation that lets the audience look at 558 episodes of VPRO Tegenlicht (Dutch Future Affairs Documentary series) through the eyes of a computer vision Neural Network. The installation employs an object detection algorithm that has been trained on Imagenet data across three categories; people, artefacts and natural objects. These categories are the “eyes” through which the algorithm observes the archive.
By choosing an eye, the user chooses a narrow perspective; The eye trained on people can only see people. With these eyes, the algorithm can identify between 30 and 60 different classes; men, women, children, but also politicians, economists and “bad people”. These classes follow the structure of the imagenet dataset and range from obvious and utilitarian to absurd and problematic. As such they highlight the inevitable cultural perspectives embedded in the process of computer vision; from data collection to annotation and categorization. After choosing an eye and browsing through its different classes the user can set the confidence threshold, a value that sets how “confident” the algorithm needs to be for a result to appear. By playing with this value, the audience can explore the limits of the algorithm; where does it draw the line between a man and a woman? By exposing these choices; the eye, the class and the confidence, the installation allows for a more intuitive relation to A.I. One that does not present A.I. as an outcome or an answer, but as a dialog between man and machine. Sometimes powerful and revealing, but also fallible and coloured.
After training the algorithm it has observed over 400 hours of video and detected more than 1 million objects. These objects are stored in a database as short clips of bounding boxes of ever changing scale and position. The installation presents these objects both in a linear way, where the bounding boxes are shown within the frame of the film as well as a non linear way, where the objects exist as autonomous entities, isolated from the original frame.
The installation uses YOLOV5 Object Detection trained on Imagenet categories and Python to analyse 1 million “objects” (short sequences of bounding boxes). MongoDB storage is used for the objects and Unity visualization is controlled with iPad via socket.io. Sound design is done in Ableton, Max MSP.
On view from February 20th to June 26th 2022 at Het Nieuwe Instituut Rotterdam.
Project Page | Richard Vijgen | Nieuwe Instituut Rotterdam
Credits: Installation Design and Production: Richard Vijgen | Curatorial Team: Het Nieuwe Instituut, VPRO Tegenlicht (Bregtje Van Der Haak), Richard Vijgen, Koehorst in ’t Veld | Exhibition Design: Koehorst in ‘t Veld | Programme Manager: Olle Lundin | Production: Floor Berkhout, Babette Zijlstra | Sound Design: Eusebi Jucglà | Supported by Het Nieuwe Instituut, VPRO, Mondriaan Fonds, Nederlands Instituut voor Beeld en Geluid
/++
/++
