Art Lab

Art Lab

Aural Aura


The Aural Aura Project


GlowingBodysuit1


Aural Aura is a computer vision and sound installation/performance that visualizes color frequencies and sonifies resonances of key energy points of the body, generated through movement and triggered using sensors. The project aspires to enhance energy management through a mapping of color, sound and movements based on the Shaolin 18 Lohan hands, the fundamental chi Kung exercises used for meditation since 527 BCE in China.


Active and/or passive bodily movements trigger specific sound frequencies and colors mapped to body energy centers (chakras).The goal of Aural Aura is to to unite sound, color and dance in a fluidly attuned communion that transcends the performer, and ideally the audience into a hyper reality that approaches a spiritual experience.


The Project Structure


The nature of Aural Aura is fivefold involving computer vision, a wearable interface, an astral body projection, musical compositions and sensors.

structure


The Bodysuit


The bodysuit is the controller triggering sounds and visual effects according to the readings of the attached sensors. A choreography of 8 moves measured by the sensors, determines which sounds and images are generated. The sensor bodysuit acts as an instrument capable of playing the 8 tone scale resembling the lydian mode scale, that is, the white keys of the piano. The wearable interface maps human body movements to the associated body erergy center (chakra), sound and color.


wearable1


wearable4


wearable5


wearable7


wearable2


wearable8


wearable9


wearable3


wearable6


The Gloves


Musical gloves for Aural Aura

Musical gloves for Aural Aura


A pair of gloves use two Flex sensors, one soft potentiometer, one FSR and soft switches, to provide continuous controllers and switches affecting rhythm, pitch, reverb and delay.  The gloves went to multiple design variations.  In the final design of the glove I used soft switches made of conductive fabric and sowed the remaining wire wrap onto the surface of the glove. For an update please check the link Musical Gloves


The Astral Body


Sketch of Aural Aura extension of the bodysuit for performance

Sketch of Aural Aura extension of the bodysuit for performance



An exoskeleton wearable layer, the astral body dress, physically extends outwards from the central bodysuit, representing the aura and serving as a canvas for projecting images. The astral body dress is used for projection to visualize the aura in three dimensional space rather than on a screen.  Visual effects projected on screen merge with the astral body dress color effects to simulate the aura.

Computer Vision Projection


Image projection from Aural Aura

Image projection from Aural Aura



As camera and computer track movement, the screen/body projection reflects body energy centers (chakras ) triggered by glowing the equivalent colors, while the corresponding sounds are generated in realtime

The Sound


The musical compositions attempt to capture the ethereal aspect prevalent in the visual representation of the aura. The estoreric aspects of music inform the process of sound design as different energy points are mapped to distinct instrumental sounds and musical intervals based on the Pythagorean tuning system. Sounds convolve harmoniously into each other as gestural movements trigger different energy points (chakras). Pure oscillators are synthesized into alpha and delta vibrations, intervals and resonances that accompany the choreography of movements. The low vibrational frequencies used help induce meditative states. Sound does not accompany the dance but is instead created by the dancer.


Mapping Movement, Color and Sound


Aural Aura relies on a mapping between energy point (chakra), color, sound frequency and movement, to produce a synesthetic experience that physically manifests the invisible and inaudible nature of the aura.  The mapping of color to sound is based on “Rainbow Resonance”, a project I developed in Spring 2008, which assigns sound frequencies to colors in an intelligible manner by transposing light wavelengths 40 octaves below their spectrum to match the audible spectrum of the electromagnetic field.  Aural Aura adds the element of movement to achieve an immersive synaesthetic experience.


Chi Kung (Qigong) healing exercises


ikaria

The performance seeks a balancing of the energies of the body, through an alignment of the chakras using sound vibrations in conjunction with corresponding colors, as chi kung movements activate the body energy centers.


Chi Kung began when prehistoric man discovered how to manipulate breath, as a form of cosmic energy, and according to Chinese records by 2700 BCE it had become an important aspect of chinese medicine. The earlier type of Chi Kung was probably a form of meditative dance which encouraged energy balance in the body. Chinese martial arts, especially Shaolin Kungfu make use of Chi Kung to enhance fighting abilities. Carrying the Moon is one of the best exercises in Chi Kung and can be practiced in many styles. A Chinese saying tells us “You need not worry about growing old, so long as your spine is young”. Carrying out the exercise daily enhances youthfulness and health at 60 and beyond and strengthens the kidneys, thus enhancing sexual vitality.


The Moves


When the 8 shaolin moves are performed the chakra centers are activated allowing for a connectedness between body, mind and spirit, that enables a “balancing” of the energies to occur.

Rainbow Resonance

Rainbow Resonance is a computer vision installation that generates colors and musical sounds of the equivalent sound frequencies in the electromagnetic spectrum, according to the motion of the participator. The audience is encouraged to participate in a playful performance that engages the body to produce colorful images and sound resonances through simple movements. Anyone can potentially create a personal experience within the installation. The color scheme is based on the visible light spectrum.

Rainbow Resonance at the New York Hall of Science

Rainbow Resonance at the New York Hall of Science, October 2008, Spring 2009

Light wavelengths are transposed 40 octaves below, into audible sound frequencies and mapped to an 8 tone musical scale based on Pythagorean mathematics. The resulting eight-tone scale corresponds to the “white keys” on the piano, and represents the seven predominant colors of the light spectrum.
An all encompassing composition of the frequencies occurs when the arms extend upwards towards the sky. The movement is reminiscent of the meditation exercise “Carrying the Moon” in Chi Kung, an ancient Chinese method of healing and balancing energy in the body.


User Experience


Rainbow Resonance utilizes technology to engage, entertain and empower the participant by enabling them to playfully interact with a software mirror image to produce sounds and visuals.

Participants interacting with Rainbow Resonance at MS Viewpoint Ltd

Participants interacting with Rainbow Resonance at MS Viewpoint Ltd, Nicosia, July 8th 2010

A lingering sound invites the visitor to enter the space. Upon entering the installation area, the participant generates his/her mirror image mapped to the colors of the rainbow, projected on a life-size screenwall. Simple motions generate harmonic intervals. Movement along the vertical axis (upward or downward) iterates through the color musical scale, while movement along the horizontal axis (left or right) or the z-axis (forward or backward), adds richness to the sounds with additional chord variations. An all-encompassing composition of the frequencies occurs when the arms are extended towards the sky. The movement recalls the meditation exercise “Carrying the Moon” in Chi Kung, an ancient Chinese method of healing and strengthening the body, mind and spirit.




Technology


Rainbow Resonance CYBC

Sofia Paraskeva demonstrating Rainbow Resonance for CYBC (Cyprus Broadcast Corporation) at ARTos cultural and research foundation, Nicosia, Cyprus, Nov 3, 2010

The Microsoft Kinect 3D motion controller is used to detect motion in space and motion tracking processing maps “zones” of movement in virtual and 3D space. The “zone” with the most vivid motion triggers the sound generated by the participant. Artificial intelligence algorithms are utilized to visualize and sonify the participant’s movements. MAX/MSP/Jitter interprets body movements into a mapping of color and sound.




Musical Sounds


Rainbow Resonance sounds are inspired by the ancient Pythagorean symbol of the Tetractys. The tetractys consists of the first four numbers 1, 2, 3, and 4 which reveal the primary intervals 1:2 (musical octave; geometrical proportion), 2:3 (musical fifth; arithmetic proportion), and 3:4 (musical fourth; harmonic proportion), upon which musical harmony is based.



Tetractys

“Through Vibration comes Motion
Through Motion comes Color
Through Color comes Tone”

Pythagoras


In ancient Greece the Tetractys was a metaphysical symbol that embodied the principles of the natural world, the mysteries of the divine realm and the harmony of the cosmos. The sacred Tetractys informed color and music theories prevalent in ancient Greece.

The top first three dots represent the threefold White Light, containing all sound and color.
The remaining seven dots represent the colors of the spectrum and the notes of the musical scale.




Chi kung (Qigong) – A healing exercise



ikaria

Chi kung was discovered when prehistoric man learned how to manipulate breath, as a form of cosmic energy, used for various purposes. By 2700 BCE chi kung had become an important aspect of chinese medicine, according to Chinese records. The earlier type of chi kung was probably a form of meditative dance which encouraged energy balance in the body. Chinese martial arts, especially Shaolin Kungfu make use of chi kung to enhance fighting abilities.

“Carrying the Moon” is an exercise that focuses on enhancing youthfulness by keeping the spine young. Carrying out the exercise daily enhances youthfulness and health at 60 and beyond, strengthens the kidneys and enhances sexual vitality.


Op On Screen Interactive 2009

Lower East Side Performing Arts

New York Public Library, Hamilton Fish Park

opons09

 

Rainbow Resonance | Sofia Paraskeva

October 15, 2009

From camera input a computer vision installation produces images and sounds according to the movement of the participator’s body
Programming by Sofia Paraskeva

Rainbow Resonance is a computer vision installation that generates colors and musical sounds of the equivalent sound frequencies in the electromagnetic spectrum, according to the motion of the participator. The audience is encouraged to participate in a playful performance that engages the body to produce colorful images and sound resonances through simple movements.

Rainbow Resonance 1

Rainbow Resonance 1,
Participators trigger sound and visuals with their body movement at
Op On Screen Interactive,
New York Public Library,
Hamilton Fish Park,
October 15, 2009

Rainbow Resonance 2

Rainbow Resonance 2

Rainbow Resonance 3

Rainbow Resonance 3

Rainbow Resonance 4

Rainbow Resonance 4, Children playing with Rainbow Resonance at
Op On Screen Interactive,
New York Public Library,
Hamilton Fish Park,
October 15, 2009

 

Voices for Speakers | Lesley Flanigan

October 17, 2009

Interactive video display, Pamela Stein and Sarah Moulton, sopranos

Voices for Speakers uses vocalizations and feedback technology to create an interactive sound environment. This performance of Lesley’s work will include some new sounds using the dynamic voices of sopranos Sarah Moulton and Pamela Stein along with the video artistry of Luke DuBois.

Voices for Speakers 1

Voices for Speakers 1,
Lesley Flanigan and sopranos Pamela Stein and Sarah Moulton performing Voices for Speakers at
Op On Screen Interactive,
New York Public Library,
Hamilton Fish Park,
October 17, 2009

Voices for Speakers 2

Voices for Speakers 2,
Lesley Flanigan and sopranos Pamela Stein and Sarah Moulton performing Voices for Speakers

Voices for Speakers 3

Voices for Speakers 3,
Lesley Flanigan and sopranos Pamela Stein and Sarah Moulton performing Voices for Speakers

Voices for Speakers 4

Voices for Speakers 4,
Lesley Flanigan and sopranos Pamela Stein and Sarah Moulton performing Voices for Speakers

 

Trickster | Sofia Paraskeva, Elodie Lauten, Khoi Bao Le

October 22, 2009

The Trickster with Sofia Paraskeva’s interactive environment, Elodie Lauten’s
music and Khoi Bao Le’s choreography

Audio-visual Interactive Dance Performance involving computer vision motion tracking and a sensor glove interface

The Trickster is a collaborative sound and interactive installation with choreography by Khoi Bao Le and visuals and sound effects by Sofia Paraskeva. This performance is an offshoot from Lauten’s new work The Two-Cents Opera, involving interactivity between movement, music, and image projections triggered by dance and a wireless musical glove.

Trickster 1

Trickster 1

Trickster 2

Trickster 2,
Khoi Bao Le performing for The Trickster at
Op On Screen Interactive,
New York Public Library,
Hamilton Fish Park,
October 22, 2009

Trickster 3

Trickster 3

Trickster 4

Trickster 4

Trickster 5

Trickster 5

Trickster 6

Trickster 6

Trickster 7

Trickster 7,
Elodie Lauten playing her custom Trine, Khoi Bao Le improvising choreography, and Sofia Paraskeva programming interactivity for The Trickster performance, October 2009

Trickster 8

Trickster 8

Trickster 9

Trickster 9,
Elodie Lauten,
Khoi Bao Le, and Sofia Paraskeva working together for The Trickster
performance at
Op On Screen Interactive,
October 2009

 

Echolations | Arleen Schloss and Emma Zakarevicius

October 24, 2009

Note: Projections by Arleen Schloss interact with graffiti art by local artist on the library wall and children’s art displays

Echolations, a performance-sound environment for electronically processed voices. Arleen Schloss & Emma Zakarevicius will present a collaborative performance piece for voices and electronic echolations investigating the primitive desire with alphabetic sanskrit vocal sounds and digital techno-poetics (signs) as a mode of communication and connection.

Echolations 1

Echolations 1,
Arleen Schloss and Emma Zakarevicius performing Echolations at
Op On Screen Interactive,
New York Public Library,
Hamilton Fish Park,
October 24, 2009

Echolations 2

Echolations 2

Echolations 3

Echolations 3

Echolations 4

Echolations 4

Interactive Therapy


Interactive Therapy with Rainbow Resonance


extremeKids1

Extreme Kids and Crew, Brooklyn, February 23rd, 2012
Photography by Kate Milford


Rainbow Resonance is an interactive installation that generates colors and musical sounds of the equivalent sound frequencies in the electromagnetic spectrum, according to the motion of the participator. The audience is encouraged to participate in a playful performance that engages the body to produce colorful images and sound resonances through simple movements. The color scheme is based on the visible light spectrum and the sound is inspired by Pythagorean mathematics.




Technology & User Experience

The Microsoft Kinect 3D motion controller is used to detect motion in space, while software interprets body movements into a mapping of color and sound. Using the Kinect camera allows for capturing and interpreting movement along the x, y and z-axes. The colorful mirror images and sound accompaniments comprise an immersive user experience that facilitates play and social interaction. The user is empowered as simple movements of the body generate harmonic intervals and sounds. Through dance, play and movement the body becomes an instrument of expression. This interactive experience benefits children and adults with special needs.




Queens Museum of Art – Autism Awareness Month



Rainbow Resonance - QMA 1

Rainbow Resonance 1, Queens Museum of Art, Autism Awareness Month, April 6, 2013

Rainbow Resonance - QMA 2

Rainbow Resonance 2, Queens Museum of Art, Autism Awareness Month, April 6, 2013

Rainbow Resonance - QMA 3

Rainbow Resonance 3, Queens Museum of Art, Autism Awareness Month, April 6, 2013





Solomon R. Guggenheim Museum – Mind’s Eye program



Partially sighted and blind adults have also enjoyed interacting with Rainbow Resonance at Solomon R. Guggenheim Museum, Mind’s Eye Program, on June 4, 2012


guggenheim1

Rainbow Resonance at Mind’s Eye Program,
Solomon R. Guggenheim Museum,
June 4, 2012
Photography Copyright © 2012 Solomon R. Guggenheim Museum

guggenheim2

Rainbow Resonance at Mind’s Eye Program,
Solomon R. Guggenheim Museum,
June 4, 2012
Photography Copyright © 2012 Solomon R. Guggenheim Museum

guggenheim3

Rainbow Resonance at Mind’s Eye Program,
Solomon R. Guggenheim Museum,
June 4, 2012
Photography Copyright © 2012 Solomon R. Guggenheim Museum

Rainbow Resonance at Mind’s Eye Program, Solomon R. Guggenheim Museum,  June 4, 2012 Photography Copyright © 2012 Solomon R. Guggenheim Museum

Rainbow Resonance at Mind’s Eye Program,
Solomon R. Guggenheim Museum,
June 4, 2012
Photography Copyright © 2012 Solomon R. Guggenheim Museum





Extreme Kids and Crew



Rainbow Resonance has been presented to autistic children at Extreme Kids and Crew, Brooklyn, a playground haven for autistic children, on February 23rd, 2012, at Queens Museum of Art, Autism Awareness Month, on April 22, 2012 and at Brooklyn Conservatory of Music, Music Therapy Program Open House, on June 28, 2012. The goal is to provide a safe and engaging environment where children can interact to reinforce and build social skills through dance, play and music.


extremeKids2

Extreme Kids and Crew, Brooklyn, February 23rd, 2012
Photography by Kate Milford



extremeKids3

Extreme Kids and Crew, Brooklyn, February 23rd, 2012
Photography by Kate Milford





Special children interacting with Rainbow Resonance at MS Viewpoint Ltd



On July 13, 2010 two special children, Alexandros and Zoe, were invited with their families and friends to experience Rainbow Resonance at MS Viewpoint Ltd, Nicosia. This was the first time Rainbow Resonance was presented to children with special needs. It was a thrill to discover that the children responded vividly and enjoyed the piece. I did not know until the end of the evening that Zoe was blind. This moving experience of the two children’s reaction was the start of my work in the special needs community. Below are images and a video of the evening’s event.

Rainbow Resonance presented to special children, Ms Viewpoint Ltd, Nicosia, July 13th 2010

Rainbow Resonance presented to special children,
Ms Viewpoint Ltd, Nicosia,
July 13th 2010

Rainbow Resonance presented to special children, Ms Viewpoint Ltd, Nicosia, July 13th 2010

Rainbow Resonance presented to special children,
Ms Viewpoint Ltd, Nicosia,
July 13th 2010

Rainbow Resonance presented to special children, Ms Viewpoint Ltd, Nicosia, July 13th 2010

Rainbow Resonance presented to special children,
Ms Viewpoint Ltd, Nicosia,
July 13th 2010

Rainbow Resonance presented to special children, Ms Viewpoint Ltd, Nicosia, July 13th 2010

Rainbow Resonance presented to special children,
Ms Viewpoint Ltd, Nicosia,
July 13th 2010

Rainbow Resonance presented to special children, Ms Viewpoint Ltd, Nicosia, July 13th 2010

Rainbow Resonance presented to special children,
Ms Viewpoint Ltd, Nicosia,
July 13th 2010