Do you know how to play any instruments? If not, what’s holding you back? Is it time? The cost of instruction? Do you find music theory too difficult? Or is music education severely underfunded in your area? As technology becomes more and more ingrained into our daily lives, the potential applications for virtual and mixed reality to advance both music education and an appreciation of the craft are increasing dramatically.
Though holographic technology is still on the verge of becoming widely available to the everyday consumer, recent efforts by manufacturers to drop the price of headsets means it’s only a matter of time before there’s one in every household. While there’s no mass market holographic music instruction app currently on the market taking people by storm, we can look to current trends in music education and technology to see that it’s only a matter of time before you’re learning piano with the help of a Microsoft Hololens.
How companies have already managed to find success in the digital realm when it comes to musical education can tell us a lot about the ways we can expect holographic technology to also meet and exceed those needs. Consider the success of YouTube-channel-turned-tech-company, The Hoffman Academy, which helped pioneer the idea of online music lessons before they were even thought possible.
Starting in their living room, Joseph Hoffman and his wife Kelly grew a small children’s music lesson company in Portland, Oregon into a proper school that now teaches 300 students, all the while giving their lessons away for free online.
A few years into the private lesson business, Hoffman explains their epiphany: “My wife and I realized that some people were getting priced out of piano lessons.” That’s when Kelly Hoffman suggested putting lessons online despite the scarcity of online education options ten years ago and Joseph’s initial hesitancy.
The Hoffman Academy channel now has over 100,000 subscribers from all around the world. In the 8 years since they uploaded their first video in 2010, their library has grown to over 200 free video lessons. Additionally, the Hoffman Academy incorporated a web app into their teaching practice, which gamifies the drilling, repetitive aspects of piano education — much like language app Duolingo — allowing users to track their progress and practice their skills between video lessons. The games combined with videos lessons further the “Hoffman Method,” an approach to teaching they pioneered which encourages students to learn the fundamentals of piano playing by using contemporary songs and gives students the space to write their own music.
At the moment, there’s no holographic component to the Hoffman Academy’s approach, but Hoffman says that his company “believes strongly in making use of all of the best available tools for the good of the students. So any time a new tool comes out, technology-wise, we want to be at the forefront of taking those tools and seeing how we can use that to have better learning.” Once virtual or mixed reality headsets are adopted en masse, don’t be surprised if Hoffman Academy are some of the first to embrace it.
In the meantime, there is an app on the horizon for the Microsoft Hololens that is taking a similar approach to music education. Teomirn is a project of the Nippon Television Network Corporation in Japan that will use mixed reality technology like the Hololens to project a piano instructor’s own hands on a keyboard so that a student can follow along with the movements.
Ayato Fujii, the chief art director and lead on the project, sees the idea as an improvement on the ways he used to record his instructor’s piano tracks as MIDI files, then follow along at home with the MIDI file playing on an automatic performance piano. With that method, he was never able to see his teacher’s finger movements. So VR offered a possibility, yet wearing VR headsets meant not being able to see one’s own fingertips. “I was waiting for an MR device,” he says, which can now integrate the both into one visual element.
Now that the technology exists to meet Fujii’s demands, development for Teomirn is in full swing. Teomirn uses an infrared light to scan all 88 keys of a piano, scanning a player’s finger movements and allowing them to follow along at their own pace with a professional’s recorded movements which appear visually onto the keyboard via the Hololens.
“There are people who give up on musical instrument performance because they cannot read the score. I think that the problem can be solved by MR,” Fujii explains, meaning that although Teomirn doesn’t take as rigorous an approach to piano lessons as the Hoffman Academy does, it opens the possibility of piano lessons for people who wish to learn a particular song. Fujii likens the experience to a map and car GPS, guiding the user direction by direction.
In fact, the UX that projects the notes of the score in front of the user recalls the colored scrolling notes of Guitar Hero, a game which anticipated the current trend in gamification in music education.
Though it doesn’t get into the finer details of musical theory, Teomirn could be used to augment a beginner’s education program like the Hoffman Academy’s or by an experienced player who wishes to learn or practice a song without the score itself. Unlike a video, the beauty of holographic technology is the fact that it can be responsive, and help correct students in real time and based on Hoffman’s own experiences, it wouldn’t be surprising that once an app like Teomirn launches, users start to demand the ability to create and publish their own music for others to be able to learn and follow along to.
The Hoffman Academy’s success proves that there’s already a great demand for technology in the home that makes music education both accessible and appealing, especially to kids. Once the issue around affordability of mixed reality technology like the Hololens is solved, you’ll inevitably start to see applications like Teomirn being embraced by the greater public. Like the Hoffman Academy, mixed reality will then be poised to advance musical education by emphasizing the user’s own ability to shape and drive their learning, and to do so without the barriers that come with “traditional” musical education.
