Gauging people’s mood in a neighborhood with AI and ML

Statistics

Partners: Ibo Ibelings; Jelle Reith; Ai Applied; RobotDK; A.S.R

Awards: FWA of the day; European Design Award – Silver

Deliverables: Interactive installation

The challenge

Now that we are collecting more and more data, we know more about a specific neighborhood. We can map the average income, the number of soot particles in the air, the number of single people that live there, and even the state of mind of the inhabitants. But, do we really want to know all that? How can we initiate a conversation about this?

Value delivered

With AIML, neighborhoods have a ‘pet’ that can be influenced by people living in them and conveys information about their well-being. With this installation, we make abstract data more tangible and hope to foster a discussion about what role we want data to play in our lives.

Case video

00:00

Background

Aiml is a living breathing animal that reacts to her visitors and their available data in the neighborhood. With the use of social media data and facial recognition, Aiml determines people’s sentiments. This input is used by Aiml to express what she feels by drawing.

With the help of Artificial Intelligence (AI) and Machine Learning (ML) techniques, the smart data creature absorbs as much information as possible about the neighborhood.

This data is analyzed and enables Aiml to judge the mood of her environment and influences how and where she draws. In this way, an abstract digital analysis evolves into a characteristic and unpredictable physical drawing, that becomes part of the public domain.

Datainput

Aiml uses two different data inputs:

Social media messages of a specific neighborhood are converted into a smiley emoticon.
The face of visitors of the installation is converted into an outline.

Drawingstyle

The number of visitors/messages influences the drawing size. If the algorithm defines an input as positive or negative, Aiml will use a different drawing style.

Generalmood

Depending also on the overall 'mood' of Aiml it would portray the images on the left, right, or middle part of her cage.

Scenarios

Interaction scenarios are defined based on all variables that can be influenced.

Phases and toolkit items used

Concept design & visualization

(Rapid) prototyping

Front-end development

Software development

API specification

Maintenance

Solution

The digital ‘creature’ Aiml reflects the sentiment of the neighborhood; she continuously senses her surroundings and reacts to the mood of a neighborhood by drawing. If the neighborhood is happy, then Aiml is a great company. If the neighborhood is not doing well, Aiml will transform into a fierce beast that is difficult to tame.

This ‘data beast’ captures your data and expresses your sentiments by means of drawing while letting you discover how AI and machine learning techniques can be harnessed to gain insight into people’s emotions and shape the future of city living in a positive way.

Visitors can uncover more details about how Aiml is feeling through aimlmonitor.com

. They can follow how she experiences the mood of her environment, what she sees and how she assesses different messages and visitors.

Besides that, visitors will find background information about the way Aiml uses AI and ML techniques to come to her conclusions.

Technology

Software behind Aiml consists of three different parts:

A serverside sentiment analyses and database
The website aimlmonitor.com
The robot, webcam, and light control software

We used several frameworks, programming languages, and software for the development of these parts, including RobotDK, Kuka API, MongoDB, ReactJS, Phython, and Javascript.