Difference between revisions of "DAC 2009"
(→Outline) |
(→semantic gaussian blur) |
||
| (5 intermediate revisions by the same user not shown) | |||
| Line 31: | Line 31: | ||
site for intervention in the creative process, this paper frames the problem of imagination and choice in | site for intervention in the creative process, this paper frames the problem of imagination and choice in | ||
studio practice as a series of operations on sets of data (ideas, images, and materials) represented in the | studio practice as a series of operations on sets of data (ideas, images, and materials) represented in the | ||
| − | space of language. This choice of representation is drawn from the idea of indexical language, as | + | space of language. This choice of representation is drawn from the idea of indexical language, as an |
| − | + | approach to the problem of representing materially diverse assets in a common space—in this case a | |
multi-dimensional vector space as described in Natural Language Processing (NLP) and Computational | multi-dimensional vector space as described in Natural Language Processing (NLP) and Computational | ||
| − | Linguistics (CL) literature. Using tools from | + | Linguistics (CL) literature. Using tools from those fields, specifically vector representations |
and relatedness operations, I describe and implement a language processing system, applying it to the | and relatedness operations, I describe and implement a language processing system, applying it to the | ||
domains of video composition and an art-idea machine, creating formulae for studio art objects. Less | domains of video composition and an art-idea machine, creating formulae for studio art objects. Less | ||
| Line 49: | Line 49: | ||
== Outline == | == Outline == | ||
| + | |||
=== Text as Best Index === | === Text as Best Index === | ||
| + | |||
Background/Motivation | Background/Motivation | ||
* online culture produces massive, language-annotated media datasets, (e-mails, twitter, text message, any direct messaging), and (video, audio, photos) | * online culture produces massive, language-annotated media datasets, (e-mails, twitter, text message, any direct messaging), and (video, audio, photos) | ||
* Text as best index | * Text as best index | ||
* Computational NLP as tool. | * Computational NLP as tool. | ||
| + | |||
=== What will the program do? === | === What will the program do? === | ||
| + | |||
* Speculative exploration (in the realm of ideas) ... grow something beyond its bounds. (Diagram 1) ideas ?= language. | * Speculative exploration (in the realm of ideas) ... grow something beyond its bounds. (Diagram 1) ideas ?= language. | ||
* retrieval (humans do that). search engines. key-phrase. (semantically) related search. | * retrieval (humans do that). search engines. key-phrase. (semantically) related search. | ||
| Line 60: | Line 64: | ||
* ordering/sequencing. (the production of cinematic experience?) ruminatory... wandering... associative": "ocean"->"sky"->"blue"->"red"->"nail polish"-> | * ordering/sequencing. (the production of cinematic experience?) ruminatory... wandering... associative": "ocean"->"sky"->"blue"->"red"->"nail polish"-> | ||
* art object as parameterizable object, template: <title><ostensible subject><dimensions><medium><material> | * art object as parameterizable object, template: <title><ostensible subject><dimensions><medium><material> | ||
| + | |||
| + | === semantic gaussian blur === | ||
| + | * nearest neighbor replacement | ||
| + | * analagous to associative free-thinking | ||
| + | |||
=== Concerns with my approaches === | === Concerns with my approaches === | ||
| + | |||
* ...but randomness is not good enough! don't remove the human from the loop, HI remains key criteria/filter. when do we like randomness (what point is made)? when do we not? | * ...but randomness is not good enough! don't remove the human from the loop, HI remains key criteria/filter. when do we like randomness (what point is made)? when do we not? | ||
* ...is it more interesting with intent? giving a system intent AI/ugh. If not intent, how about peroccupations/fixations. | * ...is it more interesting with intent? giving a system intent AI/ugh. If not intent, how about peroccupations/fixations. | ||
| + | |||
=== Megahal Grandmommy === | === Megahal Grandmommy === | ||
| + | |||
=== Language Brain === | === Language Brain === | ||
== References == | == References == | ||
Latest revision as of 08:04, 4 August 2009
Contents
Not_Robert: Collaboration and Co-production in Language Systems
DAC 2009, Theme: Cognition and creativity
Abstract
This paper explores the application of language processing technologies to the creation of interactive systems, particularly the problem of establishing the computer as a meaningful contributor to simple networks composed of artist, computer, and viewer. Throughout, this paper considers the ways in which effects of identification and interpretation are amplified and modulated through narrative framing. Two projects, a chatbot and a language processor, are considered as case studies of simple interactive systems, and as waypoints in a larger investigation of our desire (and predisposition) to ascribe meaning to events, experience, images, and objects. This proclivity is shown to have interesting ramifications with the accumulation of personal information on-line—both in public (Flickr, Facebook, Twitter) and in private (email, cellphone logs, bank accounts, health records)—suggesting the application of language processing and data-mining techniques to create custom, hyper-personal interactive experiences.
The first project, Megahal Grandmommy, is an n-gram Markov chain chatbot, trained as a surrogate for a family member suffering from Alzheimer’s disease. Shifting the computational goal from plausible simulation of conversation to plausible simulation of disease, this piece harnesses the inherent dysfunction of the model as a descriptive asset in the service of creative intent. The role and operation of narrative in this piece is discussed: how the viewer comes to understand the parameters of a narrative space (the artist’s identity, the program’s character and history) through clues uncovered in conversation with the software, and how that frame is essential to maintain the experience of the piece. Applying statistical language tools to form what is a very simple computational mechanism, this project nonetheless creates a rich interactive experience, exploiting the viewer’s predisposition to attach significance to the computer’s utterances on account of the character it performs (the artist’s grandmother).
Having identified language, specifically the formulation of ideas into language, as the most promising site for intervention in the creative process, this paper frames the problem of imagination and choice in studio practice as a series of operations on sets of data (ideas, images, and materials) represented in the space of language. This choice of representation is drawn from the idea of indexical language, as an approach to the problem of representing materially diverse assets in a common space—in this case a multi-dimensional vector space as described in Natural Language Processing (NLP) and Computational Linguistics (CL) literature. Using tools from those fields, specifically vector representations and relatedness operations, I describe and implement a language processing system, applying it to the domains of video composition and an art-idea machine, creating formulae for studio art objects. Less reliant on narrative framing than the chatbot project, the effort here is a decomposition of the creative process into data and operations, searching for new points of human-computer collaboration. Given a large, rich set of material, can a computer make meaningful creative decisions? If so, what kind?
Finally, this paper discusses the application of statistical language processing tools to the massive sets of personal material accumulated online to create hyper-personal interactive experience. With this online accretion of personal material, our contemporary self-representation of identity has become in some sense the aggregate of these digital-material traces, begging for exploitation via the technologies of index, analysis, and transformation (as described in the previous section) to produce new, personal interactive experience. Initial possibilities are described and discussed.
Outline
Text as Best Index
Background/Motivation
- online culture produces massive, language-annotated media datasets, (e-mails, twitter, text message, any direct messaging), and (video, audio, photos)
- Text as best index
- Computational NLP as tool.
What will the program do?
- Speculative exploration (in the realm of ideas) ... grow something beyond its bounds. (Diagram 1) ideas ?= language.
- retrieval (humans do that). search engines. key-phrase. (semantically) related search.
- summary. key terms. key phrases (of the form "VERB to VERB", "NOUN modifier", "modifier NOUN", etc.). in engineering, the field of IR (Information Retrieval).
- ordering/sequencing. (the production of cinematic experience?) ruminatory... wandering... associative": "ocean"->"sky"->"blue"->"red"->"nail polish"->
- art object as parameterizable object, template: <title><ostensible subject><dimensions><medium><material>
semantic gaussian blur
- nearest neighbor replacement
- analagous to associative free-thinking
Concerns with my approaches
- ...but randomness is not good enough! don't remove the human from the loop, HI remains key criteria/filter. when do we like randomness (what point is made)? when do we not?
- ...is it more interesting with intent? giving a system intent AI/ugh. If not intent, how about peroccupations/fixations.
