Changes

==Proposal - Solipsist ==

I have been working with voice recognition technologies and Mel Bochner's text 'Serial Art, Systems, Solipsism', developing a device for performance and exchange between human and computer. The device consists of a microphone, speech recognition system, software, and a receipt printer. The format of the conversation is an dialog between human voice and printed receipts in text--the system transcribes (and validates) what it hears in terms of the words it knows. As is characteristic of voice recognition, face recognition, and other kinds of machine perception that operate within explicitly defined or statistically trained spaces of "perception", this is a solipsistic system, "denying the existence of anything outside the confines of its own mind" (Bochner, 1967). This character of the solipsist is one Mel Bochner evoked to describe the autonomy and denial of external reference in minimalist sculpture of the 1960s, but which I find particularly appropriate to describe current "smart" technologies--ultimately the agency of the system still comes down to whatever agency the programmers have embedded in it, a sort of ventriloquism. This idea of a closed, narrowly parameterized space of perception in the machine is an interesting model for (and contrast to) issues of language, vocabulary, and free expression in humans--an exploration I intend to pursue through this project.

Finally, I need to address sonic properties of the piece and its time course as a composition. The voice of the viewer as they speak into the microphone is one sound source in the system, and the receipt printer has a very assertive (and retro) dot-matrix-ey sound as it prints out text and cuts rolls of paper. I need to make some decisions about how to use the sounds of the speaker and the printer over the course of the piece. Also, will I add in additional sound sources such as more printers, pre-recorded voices, voices of past participants, or processed sounds? There are additional possibilities here for rhythmic exchanges between the percussive sound of the printer and the speaker, for long pauses, silences, and repetitions. Additionally, I need to establish some overall arc for the piece--does an encounter with the system travel through to one pre-ordained conclusion? Are there multiple branching possibilities that change depending on what the viewer says and how they respond to the printouts? Finally there is a relationship to be explored between speech as text and speech as sound--a parallel to the roles of printing as text and printing as sound. The fundamental distinctions between text, sound, and speech as kinds of communication and expression can be ripe territory for exploration. I suspect that these conceptual and compositional questions will occupy most of my time this quarter and comprise the bulk of the work that I need to do.

The most obvious technical challenges I foresee at this point are the implementation of sound input and pre-processing with supercollider, and interfacing from supercollider to the speech recognition library and receipt printer. As part of this project is a critical investigation of the strengths and limitations of automatic speech recognition (ASR) technology, I intend to get more involved with the internal mechanisms of speech recognition as implemented in the Sphinx-4 library. A more comprehensive understanding of that technology is necessary to figure out how to tweak it and expose its internal character and assumptions.

**see [[#Code | Code]] section below.

'''<nowiki>*</nowiki>I do not have these items yet.'''

==Timeline==

'''Week 1 - 2'''

Introduction to course and project development.

#Find desk and desk lamp for "me vs. the computer" staging of microphone/speech recognition system.

Work time.

Working model of each of <s>two</s> one tracks:

#Participant speaking to computer.

#<s>Computer/printer speaking to itself (feedback loop). Interpreting printer sounds as speech. Or transforming them into speech.</s>

Realize that the best approach will combine elements of each of the two tracks above.

*Experiments with the characterization of the system:

**Software agent? agency. towards what goals?

I imagine these two go hand in hand--that the choice of particular texts will lend much of the character to the piece.

Have others experience the system, try it out.

Near-final form, near-final realization.

Viewer interaction tests.

Final changes, improvements, last minute blitz.

==Progress==

*Example code for Grammar-based recognition: http://svn.roberttwomey.com/processing/sphinxGrammarTest/

*Example code for Statistical Language Model (SLM) based recognition: http://svn.roberttwomey.com/processing/sphinxSLMTest/

===Supercollider===

*Speech Segmenter tool in supercollider http://svn.roberttwomey.com/supercollider/supercollider/speechtool.scd

===Pocketsphinx command-line Recognizer in OS X===

*http://svn.roberttwomey.com/supercollider/pocketsphinx-osx/

===Build a Language Model Online===

*upload a list of sentences (i.e. text file) here: http://www.speech.cs.cmu.edu/tools/lmtool-new.html

===Build a Language Model Locally===

*see week 10 and final week above... not yet done

== How To Compile on OS X ==

*Reference Home http://cmusphinx.sourceforge.net/wiki/

*download http://sourceforge.net/projects/cmusphinx/files/

*training a SLM, http://www.speech.cs.cmu.edu/tools/lmtool-new.html

*make x86_64 version of libsphinxbase:

<code>

*Charles O. Hartman. Virtual Muse: Experiments in Computer Poetry. http://www.amazon.com/Virtual-Muse-Experiments-Computer-Wesleyan/dp/0819522392/ref=ntt_at_ep_dpt_2 / http://www.upne.com/0-8195-2238-4.html

*Brief History of the Oulipo. Jean Lescure. In ''New Media Reader'', Noah Wardrip-Fruin, Nick Montfort 2003.