The first pianist in the world in the metaverse is Italian and plays Steve Reich – La Voce di New York

From today you can make music in duo with your own avatar. Giusy Caruso is the first pianist in the world to have made a musical foray into the metaverse with the performance “MethaPhase, contrapuntal dialogue between a pianist and her avatar”, made in world preview on 23 September at Cremona Musica International Exhibitions and Festival on music by American composer Steve Reich.

It is a project developed in collaboration with the engineer Paolo Belluco, the designer Samuele Polistina and the VR developer Andrea Randone of LWT3, a very Italian company that deals with the acquisition, management, analysis and understanding of data and human performance.

The metaverse is destined to change not only human behavior and interactions on a planetary level, but also the ways in which art is used, allowing the public to explore new sound worlds beyond the threshold of reality thanks to the integration of digital technology in the performing arts.

The conceptual roots of the performance, inspired by the most recent studies in neuroscience on performative movement, bio-mechanics and man-machine interaction, derive from the research path undertaken some time ago by the Italian concert artist residing in Brussels, who is currently a post -doc at the Royal Conservatory of Antwerp, at the IPEM – Institute of Psychoacoustics and Electronic Music within the Department of Art, Music and Theater Sciences of Ghent University as well as at the Laboratoire de Musicologie, Université libre de Bruxelles, and visiting professor at the London Performing Academy of Music where he holds the Mirroring Creative Lab, a course on the practice and technology of musical performance.
The pianist explains the details of her innovative project that sees her dialogue with her avatar in the metaverse and the new directions of her incessant activity as a research artist.

Giusy Caruso

First, what connects you to the United States and, in particular, to New York City?

“The recording of the “72 Etudes Karnatiques” by the French composer Jacques Charpentier was released in 2018 by the American record label Centaur Records. This meant that I was reviewed by several American critics, including Jed Distler, who later became a great friend of mine. It was he who asked me to hold a marathon-concert in New York in the Spectrum Hall in Brooklyn, known for hosting experimental music exhibitions, where I performed all 72 Carnatic Studies, about three hours of music, also presenting the CD he reviewed for the online magazine Classics Today”.

Can you explain to us what your world premiere performance at Cremona Musica consisted of?

“This performance is the result of a scientific study that I carried out in recent years at the Institute of Psychoacoustics and Electronic Music of the University of Ghent, in Belgium where, working precisely on Charpentier’s Studies, I used the Motion Capture technology, based on a system of infrared cameras that manage to capture the movement in the space of the performer, which is covered with light-reflecting markers. This technology allows not only to collect quantitative data (i.e. the measurement of amplitude, speed and acceleration) regarding the performative gesture of the musician, in particular in the approach to contemporary writing, but also to visualize, through the creation of a virtual agent, therefore of an avatar, his own body in three-dimensional space. This allows for an even more in-depth observation of bodily expressiveness in piano performance”.

How did you end up in the metaverse then?

“The next step in my research concerns the use of this technology from the point of view of the performer and his bodily perception, through the use of a particular eyewear, the Oculus, while he is immersed in a virtual scene. This experimentation, I recall, has above all a scientific purpose, but it then led me to think of realizing a real performance in the metaverse, in the wake of the idea of ​​”augmented-performance”.

The composer you entered the metaverse with is New Yorker Steve Reich…

“Exactly. For this performance we used the song Plan Phase by Steve Reich, based on twelve notes that are repeated by a first pianist, while a second pianist repeats the same notes initially in unison; subsequently a small acceleration is created, a delay between the two musicians and then they come back in sync again. A counterpoint therefore develops between the two pianists between moments of phase, i.e. of synchronism, and moments in which one goes out of sync: this creates a phase shift and therefore new musical patterns until, at the end of the cycle, one returns again to ‘unison. I thought of reinterpreting this composition by pre-recording the first part through Motion Capture, thus capturing my body expressiveness, and then, during the live performance, in real time, I started having a dialogue in the metaverse with my avatar. The public, which was physically present, followed the performance within this virtual space and the next day, in a post-production phase, in our stand at Cremona Fiera, people could review the performance created on day before precisely in the metaverse”.

Giusy Caruso

Let’s talk about the particular instrument with which you made the performance. What is the man-machine interaction, or better, woman-floor interaction?

“I used the Yamaha disklavier, which is a partner in the project, which allows us to record the expressive touch and also the pedaling of the musician. With this instrument I first performed piano pieces of contemporary music in augmented reality, such as the first movement of the collection “Memento Mori” by the Belgian composer Wim Henderickx and a selection of Charpentier’s “72 Etudes Karnatiques”, but the real foray into the metaverse was had, as I explained, with the song by Steve Reich”.

From the point of view of the public, what does this new form of immersive and, at the same time, interactive use of music consist of?

“Having the possibility of entering the space of the metaverse to enjoy something that is the result of an interaction with the machine, which however is a creative interaction, I think can offer a glimpse into the future. Without taking anything away from the traditional concerts of classical music, which must continue to exist, it is necessary to create new performative codes in which people can identify. This performance in the metaverse also had a strong purpose of disseminating contemporary music, linked to the concept of public engagement”.

Your artistic experimentation continues in the T*ActiLE project at the Antwerp Conservatory with CREATIV – Artistic Research Group, a research team based in the Royal Belgian Conservatory. What should we expect more advanced?

“Soon there will be the possibility of implementing sensors in my performance that can put the public in direct contact with the metaverse so that, with a particular gesture, they can interact with the virtual scene making personal changes, changing the colors or adding of the sounds. This project, precisely T*ActiLE, focuses precisely on the way in which body movement, touch, can be translated into something visual or auditory, not only from the point of view of the performer, but also of the spectator, who becomes an active part of the performance and, in some way, co-creator. This new phase of my new project starts from the concept of the performative space as a space of social interaction”.

You have published a book called Musical Artistic Research. Languages ​​and Methods (LIM, 2022), the first in Italian on artistic-musical research. What does the figure of the research musician consist of?

“Music for a performer must always be a mission aimed at creating something new. The concept of research is therefore inherent in the creative path of each artist. For twenty years, artistic research has been institutionalized in northern Europe, taking up the American model of performance studies, for which it is possible to access doctorates where artistic research is recognized as scientific research, understood as a harbinger of new knowledge. The goal of artistic research must not be that of only produce a performance, but also to create new knowledge, which can also be disseminated through scientific articles. That of the artist researcher is a new professional figure that in Italy has yet to be recognized and placed even within Conservatories”.

Source link

About David Martin

David Martin is the lead editor for Spark Chronicles. David has been working as a freelance journalist.

Check Also

Chadia Rodriguez Releases Irreverent New Single ‘Bitch 3.0’

Goes out Friday 23 December “Bitch 3.0” (Sony Music Italy)the new single from the …

Leave a Reply

Your email address will not be published. Required fields are marked *