Art Lab

Interactive Installation

Viewing posts from the Interactive Installation category

Aural Aura


The Aural Aura Project


GlowingBodysuit1


Aural Aura is a computer vision and sound installation/performance that visualizes color frequencies and sonifies resonances of key energy points of the body, generated through movement and triggered using sensors. The project aspires to enhance energy management through a mapping of color, sound and movements based on the Shaolin 18 Lohan hands, the fundamental chi Kung exercises used for meditation since 527 BCE in China.


Active and/or passive bodily movements trigger specific sound frequencies and colors mapped to body energy centers (chakras).The goal of Aural Aura is to to unite sound, color and dance in a fluidly attuned communion that transcends the performer, and ideally the audience into a hyper reality that approaches a spiritual experience.


The Project Structure


The nature of Aural Aura is fivefold involving computer vision, a wearable interface, an astral body projection, musical compositions and sensors.

structure


The Bodysuit


The bodysuit is the controller triggering sounds and visual effects according to the readings of the attached sensors. A choreography of 8 moves measured by the sensors, determines which sounds and images are generated. The sensor bodysuit acts as an instrument capable of playing the 8 tone scale resembling the lydian mode scale, that is, the white keys of the piano. The wearable interface maps human body movements to the associated body erergy center (chakra), sound and color.


wearable1


wearable4


wearable5


wearable7


wearable2


wearable8


wearable9


wearable3


wearable6


The Gloves


Musical gloves for Aural Aura

Musical gloves for Aural Aura


A pair of gloves use two Flex sensors, one soft potentiometer, one FSR and soft switches, to provide continuous controllers and switches affecting rhythm, pitch, reverb and delay.  The gloves went to multiple design variations.  In the final design of the glove I used soft switches made of conductive fabric and sowed the remaining wire wrap onto the surface of the glove. For an update please check the link Musical Gloves


The Astral Body


Sketch of Aural Aura extension of the bodysuit for performance

Sketch of Aural Aura extension of the bodysuit for performance



An exoskeleton wearable layer, the astral body dress, physically extends outwards from the central bodysuit, representing the aura and serving as a canvas for projecting images. The astral body dress is used for projection to visualize the aura in three dimensional space rather than on a screen.  Visual effects projected on screen merge with the astral body dress color effects to simulate the aura.

Computer Vision Projection


Image projection from Aural Aura

Image projection from Aural Aura



As camera and computer track movement, the screen/body projection reflects body energy centers (chakras ) triggered by glowing the equivalent colors, while the corresponding sounds are generated in realtime

The Sound


The musical compositions attempt to capture the ethereal aspect prevalent in the visual representation of the aura. The estoreric aspects of music inform the process of sound design as different energy points are mapped to distinct instrumental sounds and musical intervals based on the Pythagorean tuning system. Sounds convolve harmoniously into each other as gestural movements trigger different energy points (chakras). Pure oscillators are synthesized into alpha and delta vibrations, intervals and resonances that accompany the choreography of movements. The low vibrational frequencies used help induce meditative states. Sound does not accompany the dance but is instead created by the dancer.


Mapping Movement, Color and Sound


Aural Aura relies on a mapping between energy point (chakra), color, sound frequency and movement, to produce a synesthetic experience that physically manifests the invisible and inaudible nature of the aura.  The mapping of color to sound is based on “Rainbow Resonance”, a project I developed in Spring 2008, which assigns sound frequencies to colors in an intelligible manner by transposing light wavelengths 40 octaves below their spectrum to match the audible spectrum of the electromagnetic field.  Aural Aura adds the element of movement to achieve an immersive synaesthetic experience.


Chi Kung (Qigong) healing exercises


ikaria

The performance seeks a balancing of the energies of the body, through an alignment of the chakras using sound vibrations in conjunction with corresponding colors, as chi kung movements activate the body energy centers.


Chi Kung began when prehistoric man discovered how to manipulate breath, as a form of cosmic energy, and according to Chinese records by 2700 BCE it had become an important aspect of chinese medicine. The earlier type of Chi Kung was probably a form of meditative dance which encouraged energy balance in the body. Chinese martial arts, especially Shaolin Kungfu make use of Chi Kung to enhance fighting abilities. Carrying the Moon is one of the best exercises in Chi Kung and can be practiced in many styles. A Chinese saying tells us “You need not worry about growing old, so long as your spine is young”. Carrying out the exercise daily enhances youthfulness and health at 60 and beyond and strengthens the kidneys, thus enhancing sexual vitality.


The Moves


When the 8 shaolin moves are performed the chakra centers are activated allowing for a connectedness between body, mind and spirit, that enables a “balancing” of the energies to occur.

Op On Screen Interactive 2009

Lower East Side Performing Arts

New York Public Library, Hamilton Fish Park

opons09

 

Rainbow Resonance | Sofia Paraskeva

October 15, 2009

From camera input a computer vision installation produces images and sounds according to the movement of the participator’s body
Programming by Sofia Paraskeva

Rainbow Resonance is a computer vision installation that generates colors and musical sounds of the equivalent sound frequencies in the electromagnetic spectrum, according to the motion of the participator. The audience is encouraged to participate in a playful performance that engages the body to produce colorful images and sound resonances through simple movements.

Rainbow Resonance 1

Rainbow Resonance 1,
Participators trigger sound and visuals with their body movement at
Op On Screen Interactive,
New York Public Library,
Hamilton Fish Park,
October 15, 2009

Rainbow Resonance 2

Rainbow Resonance 2

Rainbow Resonance 3

Rainbow Resonance 3

Rainbow Resonance 4

Rainbow Resonance 4, Children playing with Rainbow Resonance at
Op On Screen Interactive,
New York Public Library,
Hamilton Fish Park,
October 15, 2009

 

Voices for Speakers | Lesley Flanigan

October 17, 2009

Interactive video display, Pamela Stein and Sarah Moulton, sopranos

Voices for Speakers uses vocalizations and feedback technology to create an interactive sound environment. This performance of Lesley’s work will include some new sounds using the dynamic voices of sopranos Sarah Moulton and Pamela Stein along with the video artistry of Luke DuBois.

Voices for Speakers 1

Voices for Speakers 1,
Lesley Flanigan and sopranos Pamela Stein and Sarah Moulton performing Voices for Speakers at
Op On Screen Interactive,
New York Public Library,
Hamilton Fish Park,
October 17, 2009

Voices for Speakers 2

Voices for Speakers 2,
Lesley Flanigan and sopranos Pamela Stein and Sarah Moulton performing Voices for Speakers

Voices for Speakers 3

Voices for Speakers 3,
Lesley Flanigan and sopranos Pamela Stein and Sarah Moulton performing Voices for Speakers

Voices for Speakers 4

Voices for Speakers 4,
Lesley Flanigan and sopranos Pamela Stein and Sarah Moulton performing Voices for Speakers

 

Trickster | Sofia Paraskeva, Elodie Lauten, Khoi Bao Le

October 22, 2009

The Trickster with Sofia Paraskeva’s interactive environment, Elodie Lauten’s
music and Khoi Bao Le’s choreography

Audio-visual Interactive Dance Performance involving computer vision motion tracking and a sensor glove interface

The Trickster is a collaborative sound and interactive installation with choreography by Khoi Bao Le and visuals and sound effects by Sofia Paraskeva. This performance is an offshoot from Lauten’s new work The Two-Cents Opera, involving interactivity between movement, music, and image projections triggered by dance and a wireless musical glove.

Trickster 1

Trickster 1

Trickster 2

Trickster 2,
Khoi Bao Le performing for The Trickster at
Op On Screen Interactive,
New York Public Library,
Hamilton Fish Park,
October 22, 2009

Trickster 3

Trickster 3

Trickster 4

Trickster 4

Trickster 5

Trickster 5

Trickster 6

Trickster 6

Trickster 7

Trickster 7,
Elodie Lauten playing her custom Trine, Khoi Bao Le improvising choreography, and Sofia Paraskeva programming interactivity for The Trickster performance, October 2009

Trickster 8

Trickster 8

Trickster 9

Trickster 9,
Elodie Lauten,
Khoi Bao Le, and Sofia Paraskeva working together for The Trickster
performance at
Op On Screen Interactive,
October 2009

 

Echolations | Arleen Schloss and Emma Zakarevicius

October 24, 2009

Note: Projections by Arleen Schloss interact with graffiti art by local artist on the library wall and children’s art displays

Echolations, a performance-sound environment for electronically processed voices. Arleen Schloss & Emma Zakarevicius will present a collaborative performance piece for voices and electronic echolations investigating the primitive desire with alphabetic sanskrit vocal sounds and digital techno-poetics (signs) as a mode of communication and connection.

Echolations 1

Echolations 1,
Arleen Schloss and Emma Zakarevicius performing Echolations at
Op On Screen Interactive,
New York Public Library,
Hamilton Fish Park,
October 24, 2009

Echolations 2

Echolations 2

Echolations 3

Echolations 3

Echolations 4

Echolations 4

Interactive Therapy


Interactive Therapy with Rainbow Resonance


extremeKids1

Extreme Kids and Crew, Brooklyn, February 23rd, 2012
Photography by Kate Milford


Rainbow Resonance is an interactive installation that generates colors and musical sounds of the equivalent sound frequencies in the electromagnetic spectrum, according to the motion of the participator. The audience is encouraged to participate in a playful performance that engages the body to produce colorful images and sound resonances through simple movements. The color scheme is based on the visible light spectrum and the sound is inspired by Pythagorean mathematics.




Technology & User Experience

The Microsoft Kinect 3D motion controller is used to detect motion in space, while software interprets body movements into a mapping of color and sound. Using the Kinect camera allows for capturing and interpreting movement along the x, y and z-axes. The colorful mirror images and sound accompaniments comprise an immersive user experience that facilitates play and social interaction. The user is empowered as simple movements of the body generate harmonic intervals and sounds. Through dance, play and movement the body becomes an instrument of expression. This interactive experience benefits children and adults with special needs.




Queens Museum of Art – Autism Awareness Month



Rainbow Resonance - QMA 1

Rainbow Resonance 1, Queens Museum of Art, Autism Awareness Month, April 6, 2013

Rainbow Resonance - QMA 2

Rainbow Resonance 2, Queens Museum of Art, Autism Awareness Month, April 6, 2013

Rainbow Resonance - QMA 3

Rainbow Resonance 3, Queens Museum of Art, Autism Awareness Month, April 6, 2013





Solomon R. Guggenheim Museum – Mind’s Eye program



Partially sighted and blind adults have also enjoyed interacting with Rainbow Resonance at Solomon R. Guggenheim Museum, Mind’s Eye Program, on June 4, 2012


guggenheim1

Rainbow Resonance at Mind’s Eye Program,
Solomon R. Guggenheim Museum,
June 4, 2012
Photography Copyright © 2012 Solomon R. Guggenheim Museum

guggenheim2

Rainbow Resonance at Mind’s Eye Program,
Solomon R. Guggenheim Museum,
June 4, 2012
Photography Copyright © 2012 Solomon R. Guggenheim Museum

guggenheim3

Rainbow Resonance at Mind’s Eye Program,
Solomon R. Guggenheim Museum,
June 4, 2012
Photography Copyright © 2012 Solomon R. Guggenheim Museum

Rainbow Resonance at Mind’s Eye Program, Solomon R. Guggenheim Museum,  June 4, 2012 Photography Copyright © 2012 Solomon R. Guggenheim Museum

Rainbow Resonance at Mind’s Eye Program,
Solomon R. Guggenheim Museum,
June 4, 2012
Photography Copyright © 2012 Solomon R. Guggenheim Museum





Extreme Kids and Crew



Rainbow Resonance has been presented to autistic children at Extreme Kids and Crew, Brooklyn, a playground haven for autistic children, on February 23rd, 2012, at Queens Museum of Art, Autism Awareness Month, on April 22, 2012 and at Brooklyn Conservatory of Music, Music Therapy Program Open House, on June 28, 2012. The goal is to provide a safe and engaging environment where children can interact to reinforce and build social skills through dance, play and music.


extremeKids2

Extreme Kids and Crew, Brooklyn, February 23rd, 2012
Photography by Kate Milford



extremeKids3

Extreme Kids and Crew, Brooklyn, February 23rd, 2012
Photography by Kate Milford





Special children interacting with Rainbow Resonance at MS Viewpoint Ltd



On July 13, 2010 two special children, Alexandros and Zoe, were invited with their families and friends to experience Rainbow Resonance at MS Viewpoint Ltd, Nicosia. This was the first time Rainbow Resonance was presented to children with special needs. It was a thrill to discover that the children responded vividly and enjoyed the piece. I did not know until the end of the evening that Zoe was blind. This moving experience of the two children’s reaction was the start of my work in the special needs community. Below are images and a video of the evening’s event.

Rainbow Resonance presented to special children, Ms Viewpoint Ltd, Nicosia, July 13th 2010

Rainbow Resonance presented to special children,
Ms Viewpoint Ltd, Nicosia,
July 13th 2010

Rainbow Resonance presented to special children, Ms Viewpoint Ltd, Nicosia, July 13th 2010

Rainbow Resonance presented to special children,
Ms Viewpoint Ltd, Nicosia,
July 13th 2010

Rainbow Resonance presented to special children, Ms Viewpoint Ltd, Nicosia, July 13th 2010

Rainbow Resonance presented to special children,
Ms Viewpoint Ltd, Nicosia,
July 13th 2010

Rainbow Resonance presented to special children, Ms Viewpoint Ltd, Nicosia, July 13th 2010

Rainbow Resonance presented to special children,
Ms Viewpoint Ltd, Nicosia,
July 13th 2010

Rainbow Resonance presented to special children, Ms Viewpoint Ltd, Nicosia, July 13th 2010

Rainbow Resonance presented to special children,
Ms Viewpoint Ltd, Nicosia,
July 13th 2010