Bringing music, art and tech together to deliver an immersive music experience that moves you. This digital experience is designed to be seamlessly integrated directly into her new website.

Overview

The connection between Hybrid Music Vibes and arfected had already been established, when they reached out to us for a creative collaboration. When they pitched the idea to us, we were immediately excited about the endless possibilities. The project's end result was meant to integrate with a new website that RHIANNIN would launch, so it had to be Web-based. Other than that, the team at HMV gave complete creative freedom to our development team, and RHIANNIN. RHIANNIN herself already had some ideas of what the experience should look like, and we added our own technical expertise to it. We used a process of co-creation, while incorporating a lot of RHIANNIN’s existing artwork and music. The result is a hybrid immersive experience, in which music, art and tech come together seamlessly. For this project we collaborated with Gijs Habraken for the design of the 3D artwork.

Challenges

It was key for the project to find a balanced way of combining development with the available artworks. The experience relies primarily on Three.js, a JavaScript library built around WebGL. WebGL is a JavaScript API created to render 2D and 3D graphics in (supported) browsers on both desktop and mobile devices. The experience incorporates several specific features developed for RHIANNIN, including:

Vertex Displacement: Each song is represented by a bubble.' Each bubble has its own 'characteristics,' which is displayed through differences in movement, intensity of this movement, etc. Originally, each bubble is a normal three-dimensional sphere constructed from triangles. By displacing the points of these triangles to and from the center of the sphere using Perlin noise, the effect seen in the experience is created. By adjusting this Perlin effect through JavaScript, the 'characteristics' of each bubble is modified.

Shaders: Shaders are code written in a scripting language called OpenGL Shading Language (GLSL), which you can use to control various parts of the rendering process. This allows you to shape and control different processes in a way that has little impact on performance. 

Position-based Environments: Depending on the user's position relative to a bubble, the environment changes to one of RHIANNIN's artworks. This submerges the user into song-specific environments and moods.

Spatial Audio: The audio of the individual tracks is based on the user's position relative to an object within the 3D space.

Rianne Wilbers - artist
"Thanks to arfected who did a great job in creating this immersive experience: RHIANNITY. You guys have really taught me a lot about such an experience and to think outside the box. Thanks for all the great inspiration and conversations during the whole process."

‍

‍