DAC 2009

Not_Robert: Collaboration and Co-production in Language Systems

DAC 2009, Theme: Cognition and creativity

Abstract

This paper explores the application of language processing technologies to the creation of interactive systems, particularly the problem of establishing the computer as a meaningful contributor to simple networks composed of artist, computer, and viewer. Throughout, this paper considers the ways in which effects of identification and interpretation are amplified and modulated through narrative framing. Two projects, a chatbot and a language processor, are considered as case studies of simple interactive systems, and as waypoints in a larger investigation of our desire (and predisposition) to ascribe meaning to events, experience, images, and objects. This proclivity is shown to have interesting ramifications with the accumulation of personal information on-line—both in public (Flickr, Facebook, Twitter) and in private (email, cellphone logs, bank accounts, health records)—suggesting the application of language processing and data-mining techniques to create custom, hyper-personal interactive experiences.

The first project, Megahal Grandmommy, is an n-gram Markov chain chatbot, trained as a surrogate for a family member suffering from Alzheimer’s disease. Shifting the computational goal from plausible simulation of conversation to plausible simulation of disease, this piece harnesses the inherent dysfunction of the model as a descriptive asset in the service of creative intent. The role and operation of narrative in this piece is discussed: how the viewer comes to understand the parameters of a narrative space (the artist’s identity, the program’s character and history) through clues uncovered in conversation with the software, and how that frame is essential to maintain the experience of the piece. Applying statistical language tools to form what is a very simple computational mechanism, this project nonetheless creates a rich interactive experience, exploiting the viewer’s predisposition to attach significance to the computer’s utterances on account of the character it performs (the artist’s grandmother).

Having identified language, specifically the formulation of ideas into language, as the most promising site for intervention in the creative process, this paper frames the problem of imagination and choice in studio practice as a series of operations on sets of data (ideas, images, and materials) represented in the space of language. This choice of representation is drawn from the idea of indexical language, as a solution to the problem of representing materially diverse assets in a common space—in this case a multi-dimensional vector space as described in Natural Language Processing (NLP) and Computational Linguistics (CL) literature. Using tools from the fields of NLP and CL, specifically vector representations and relatedness operations, I describe and implement a language processing system, applying it to the domains of video composition and an art-idea machine, creating formulae for studio art objects. Less reliant on narrative framing than the chatbot project, the effort here is a decomposition of the creative process into data and operations, searching for new points of human-computer collaboration. Given a large, rich set of material, can a computer make meaningful creative decisions? If so, what kind?

Finally, this paper discusses the application of statistical language processing tools to the massive sets of personal material accumulated online to create hyper-personal interactive experience. With this online accretion of personal material, our contemporary self-representation of identity has become in some sense the aggregate of these digital-material traces, begging for exploitation via the technologies of index, analysis, and transformation (as described in the previous section) to produce new, personal interactive experience. Initial possibilities are described and discussed.

Outline

• Background/Motivation
  • online culture produces massive, language-annotated personal media datasets
    • some of which is exclusively text (e-mails, twitter, text message, any direct messaging)
    • some of which are language-annotated rich media (video, audio, photos)
  • Text as best index
  • Computational NLP.

• What will the program do?
• Speculative exploration (in the realm of ideas) ... grow something beyond its bounds
  • ideas ?= language
  • (Diagram 1)
• retriveal (humans do that)
  • access/retrieval
  • search engines. key-phrase. (semantically) related search.
• summary
  • key terms
  • key phrases (of the form "VERB to VERB", "NOUN modifier", "modifier NOUN", etc.)
  • Information Retrieval (IR) problem
• ordering logic
  • ruminatory... wandering... associative
  • "ocean"->"sky"->"blue"->"red"->"nail polish"->
• Art object as parameterizable object, template
  • Title
  • Ostensible subject
  • Dimensions
  • Medium
  • Material
• Conceptual Concerns:
  • ...but randomness is not good enough!
    • don't remove the human from the loop, subjective opinion remains key criteria/filter.
    • when do we like randomness? (what point is made in those situations)
    • when do we not?
  • ...is it more interesting with intent?
    • maybe not intent, how about fixations.

• Previous Work