Emergent Mind of City: Visualization of Collective Mind in Cities
Jeong Han Kim, Hyun Jean Lee, Hong-Gee Kim, Jinhyun Ahn, and JungDo Kim
The Emergent Mind of City (EMC) project is a collaborative project where art and technology intersect. Using big data in news, social, and image networks, we tried to approach how personal emotion is connected to the social news and visualize the collective mind of city. The artwork EMC is a system consisted of a visualizer as an input/output interface and a middleware software, which collects web data, semantically analyzes them along with emotion database. The result of such analysis finally is sent to the visualizer to reveal the collective mind in three projection screens as a media art exhibition setup.
The Emergent Mind of City (EMC) is a collaborative media art project by artists and scientists. EMC project has been presented as a video and sound installation in several exhibition venues and developed with three different versions from Ver. 1(2012) to Ver. 3(2015).
Background Concept of Artwork
His drawings incorporate ideas of concrete structures as watercourses (canals), waterways, water supply facilities, sewers, and an automatic, self-cleaning stable. He imagined a city as an evolving creature with a very complex system comprises men and systems like various organizations. Emergence also often comes with the concept of 'evolution.' Many physical, biological and even social systems seem to evolve through the space-time dimension. In this respect, emergent properties of a system can be illustrated as an outcome of the evolutionary development of the system. Humans talk to each other to share their thought, emotion and information content every ordinary day. The channels of the interactive communication flow were set in every city, country and society becoming the infrastructure of the social evolution.
With this background concept for artwork in the EMC project, we look at the contemporary cities in the perspective of data flow, particularly as digitally networked societies. It sometimes looks like a human in that its mind manifests collective intelligence and emotions, exploiting new media technologies and transcending time and space – surfacing then itself as an embodied mind that unifies both mind and body.
In the human body, afferent and efferent neural transmissions among nerves enable various organs to work as one interconnected organism. If the city is viewed as a human body, the neural transmissions can be likened to the data flow of our time.
On the other hand, a mental model, so called a culture, is shared among the citizens of a city. For a single event or issue, there exists a myriad of perspectives. If we consider the whole of data emerging daily in the city, it is an opaque, tantalizing, floating chaos. Thus we approached to the data arranged and categorized in light of news, issues, and opinions. Only when someone tries to interpret, or hold a view point on, this unsubstantial data emerges onto a subjective, conscious level, and so the meaning of data can be understood. As for the collective mind, it emerges while thoughts and emotions of its citizens engage in continuous interactions with one another. This idea is similar to the psychological concept of ¡®fringe¡¯ by William James . According to him, ¡®fringe¡¯ is a vague state in which one feels that he knows a certain thing, though he can¡¯t readily express it. Thus it is rather ¡°feeling of knowing¡± states. It is an obscure boundary of cognition, in-between state of consciousness and unconsciousness, or subjectivity and objectivity.
The EMC specifically focuses on three flows of ¡°Fringe¡± data:
¡®event,¡¯ ¡®feeling,¡¯ and ¡®appearance.¡¯ The background of this selection is based on our imagination that the body of
memory holds such virtual organs as events, feelings, and appearances. When perspectives and meanings are projected and focused on
an event, news emerges. When these
are focused on a feeling emerges, emotions, and when focused on an appearance, image
emerges. A news-network that extracts
meaningful structures from the meaningless flow of event data represents a flow
of words that conceptualizes the city, and reveals the collective intelligence.
An emotion-network, consisting of
emotion data, is both a flow and a collective emotion that endow an identity to
the news. We searched over people¡¯s
tweets having emotional words related to current news in Twitter. Added
with the properties of ¡®real-time¡¯ and ¡®unpredictability¡¯ of choice by audience, the three elements—news, emotion
and image—are able to contact with a stratum of presence, and the mind of the
city emerges in the stream of data.
Thus far, through its hybridizing a micro-individual-perspective and macro-social-minds, the EMC series creates the ¡®Virtual Mind Neuron¡¯ of cities for Seoul, Boston, Dublin, and Mumbai and visualizes real-time mind of cities related to a specific issue. In the exhibition setup, the EMC installation shows three different minds of cities simultaneously on three screens selectively in order to show how each city has its own mind and how it is different from the others.
Technology developed in EMC
Overall System Structure
Technically we design the system consisted of a kiosk interface called a ¡®visualizer¡¯ taking user input, and a ¡®middleware¡¯ which collects data from three of online web services, such as Twitter, Google News, and YouTube. Communications in between the middleware and all web services are accessed through Hyper Text Transfer Protocol (HTTP) over the Internet. Through the middleware—a software program that generates semantics networks out of real-time data collected from social network services (Twitter), News and video sharing services (YouTube)—, we indicate a collection of words, emotion words which are related to the context. The context can be determined by both the name of the city and a word input (event of the news) by users through the visualizer¡¯s input interface, built in Flash. After analyzing the data, and finding emotional network attached to the news network, the middleware sends the results to the visualizer and finally the visualizer present images in real time 3D graphics built in the Unity 3D engine. Represented output visual output image— two tree branches stemming from a center— also represents the concepts between individual and collective in the context of human societies.
Data Analysis in the Middleware System
When the user input for the name of the city and the selection of the event among current news was made, in the case of Google News, the middleware sends words to the Google News server that returns a list of recent news mentioning the words. It is likely that news in the city is collected. The title and the summary of news are considered as a whole one text from which it extracts words related to the given words. To do so, each word in the text is tagged with POS (Part-Of-Speech) using a POS tagging program, LingPipe . As related words, we only take into account nouns and verbs. In the case of Twitter, the middleware sends words to the Twitter server that in turn returns a list of most recent tweets (short sentences) mentioning the words. Just like Google News, the tweets are also tagged with POS, and similarly only nouns and verbs are preserved. Then they go through the stemming process that uses the Porter Stemmer, a stemming algorithm. In the stemming process, morphologically-varied lexical items are removed from a given word for normalization. Normalization is to map the tweet into several emotion words stored in the database, since emotion words in the database have already been normalized using the same process.
The emotion word database, which the EMC uses, is adapted from the Emotion database, MEST project #2011K000658, built by the Dept. of Psychology at Choongnam National University in Korea. The original database was written in Korean, and so we manually mapped each Korean word into the corresponding English word for this project. The Emotion database consists of emotion words which were classified by experts into 11 emotion classes, such as ¡®happy,¡¯ ¡®sad,¡¯ etc. Each emotion word has a vector with 11 weights that represent the degree to which the word belongs to each emotion class—for example, the emotion word like ¡®wonderful¡¯ takes 48 weights from the ¡®happy¡¯ class, 31 weights from the ¡®interests¡¯ class, 26 weights from the ¡®surprised¡¯ class and so on. The stemmed words from tweets are lexically matched to the emotion words so that we decide which emotion class is the best fit for the user¡¯s emotion who posted that tweet. Once the middleware determines the emotion class for tweets, it sends back emotion words which have originally been mentioned in the tweets, and other emotion words in the database belonging to the emotion class to the visualizer. Then the visualizer can represent the image output for screen projection based on such results.
In the EMC project, as artists living in a city, we desired to peek into the mind of the city where we live, with the eye of an individual. By making a connection as the individual to the collective mind of the city, the experience of the EMC could be the chance for the audience to reflect themselves as a social-being.
We thank to everyone who provided helpful comments and data for the EMC. As well, author 1 gratefully acknowledge the grant from KOCCA¡¯s Human Resources Research & Development Program 2015. Authors 1, 2, and 5 thank ZKM for their support for upgrading and extending to EMC III for the Karlsruhe version.
References and Notes
1. Evander B. McGilvary. 1911. The 'Fringe' of William James's Psychology the Basis of Logic. The Philosophical Review. 20, 2: pp. 137-164.
2. Leonardo da Vinci's Ideal City Invention.
Retrieved January 12, 2016 from http://www.da-vinci-inventions.com/ideal-city.aspx
3. LingPipe. http://alias-i.com/lingpipe/
Jeong Han Kim is an associate professor in media art at the Department of Contemporary Art at Seoul Women¡¯s University, Korea, where, in addition to being a director of the B-MADE center (Bio-Medical Art & Design Education, http://bmade.org) and Seoul Women¡¯s University Museum.
Hyun Jean Lee is currently Fulbright visiting scholar (Fall 2015–Spring 2016) at Information School and Computer Science and Engineering dpt. at University of Washington, Seattle, USA. She is an associate professor of Media Art major at the Graduate School of Communication and Arts, Yonsei University, Seoul, Korea.
Hong-Gee Kim is the director of Biomedical Knowledge Engineering Laboratory (BiKE), professor, and dentistry library dean at School of Dentistry, Seoul National University.
Jinhyun Ahn is currently a PhD student in the biomedical knowledge engineering laboratory at Seoul National University, Korea.
JungDo Kim is a researcher and CTO in LSR/UX Research Lab, LG Electronics.
Figure 0. Architectural studies, from Leonardo da Vinci, Manuscript B, f. 18v. Paris, Institut de France.
Figure 1. Emergent Mind of City I, interactive installation of data visualization, Jeong Han Kim, Hyun Jean Lee, Hong-Gee Kim, Jin-Hyun Ahan and Jung-Do Kim, Seoul Museum of Art, 2012.
Figure 2. Emergent Mind of City I, interactive installation of data visualization, Jeong Han Kim, Hyun Jean Lee, Hong-Gee Kim, Jin-Hyun Ahan and Jung-Do Kim, Seoul Museum of Art, 2012.
Figure 3. Emergent Mind of City I, interactive installation of data visualization, Jeong Han Kim, Hyun Jean Lee, Hong-Gee Kim, Jin-Hyun Ahan and Jung-Do Kim, Seoul Museum of Art, 2012.
Figure 5. Emergent Mind of City I(Concept Drawing), interactive installation of data visualization, Jeong Han Kim, Hyun Jean Lee, Hong-Gee Kim, Jin-Hyun Ahan and Jung-Do Kim, Seoul Museum of Art, 2012.