If you just walk into the Scotiabank Arena practice facility on Monday night, you’ll find a room full of journalists, developers, and representatives from Amazon Web Services (AWS) and Maple Leafs Sports and Entertainment (MLSE).
You’ve never seen a 3D rendering of mannequins representing the Toronto Raptors and Milwaukee Bucks, recreating the latest NBA game. That is, until you put on an augmented reality (AR) or virtual reality (VR) headset.
With the help of headphones, you can immediately immerse yourself in reality. Players come to life in front of you, allowing you to follow them from baseline to baseline, or simply sit on the sidelines – whatever you experience gives you the best view of defensive breakdowns and offensive bursts.
“It’s about looking into the future,” said Humza Tehrani, director of technology and digital at MLSE.
MLSE Digital Labs and AWS have teamed up to launch a new research and development program called SpaceX. His first initiative is Augmented Reality (AR) and Virtual Reality (VR) to change the way we play professional sports. It’s not just an experience for fans, it’s also designed to help improve team performance, as players and coaches can relive moments to help them train.
As part of the Jan. 22 launch, MLSE and AWS demonstrated multiple scenarios for how AR and VR headsets can be used to display the data they collect. This allows those wearing the headsets to attend live games or receive real-time stats, such as during Monday night’s matchup between the Toronto Maple Leafs and the New York Islanders. .
“Honestly, nowhere else in the world has this been done,” Tehrani said.
“The biggest thing about this program is to create the future of sports from Toronto to the rest of the world and make these great things available within MLSE. I hope it sets an example for the rest of the sports world.”
How technology works
The “Immersive Basketball Experience” uses optical data to track body position, including 3D models, including the position of each joint and limb, and a video game development engine. The result is a life-size way to recreate the game as the players are rendered around you in 3D.
For the NBA experience, MLSE and AWS are using in-game cameras located throughout the court to collect the necessary data. The same experience can be applied to NHL games, and with advances in biomechanical data acquisition in other professional sports such as Formula 1, we expect this experience to come to all sports.
Because the league uses “NHL Edge,” the experience can be more realistic for the NHL. To collect data, the NHL uses infrared technology embedded in the ball and players’ jerseys, which in this case is transmitted to an AWS outpost at Scotiabank Arena. This technology allows you to control every part of the game’s movement and allows you to use an “Augmented Reality Statistical Layer” on an AR or VR headset.
While watching an NHL game, viewers can watch a replay of a team’s stats, such as puck possession, speed, and distance covered. By using the laser function on the remote control that comes with the headset, viewers can select individual players, such as Auston Matthews, and get specific statistics.
“You can enjoy the game and see the analytics and data all in one place,” Tehrani said, adding that it can be used by a hockey fan or someone learning the game.
Future applications: Arenas for everyday life
Currently, the cost of several thousand dollars and the size of the headset still make it difficult to reach a large number of people. Tehrani says that when innovations such as Apple’s augmented reality glasses come out, it will change the market, and we expect these features to be available in the size of your everyday glasses.
Eric Gales, manager of AWS Canada, says that a big change in the market in recent years is that you needed a lot of hardware to deliver an AR or VR experience. Now, with all the data they’ve collected in the cloud, they’re ready to take advantage of the opportunity to popularize this experience.
“Augmented Reality Stats Overlap” is an experience MLSE wants to give its fans an in-field experience. This will help enhance the viewing experience, especially for those sitting high up in the crowd. A similar statistical experience is displayed on a table-sized board at Scotiabank Arena without the headphone element, which can serve as a fixture inside the luxury suit.
Rogers and Bell owns the broadcast rights, so finding licenses for AR and VR for these games outside of the arena is a challenge.
MLSE and AWS members want to bring the Immersive Basketball experience for professional NBA games into people’s homes for everyday use. After talking to a number of developers, a realistic timeframe is 5-10 years, given the cost and size of the headset.
For a life-size version of the players, you need a bigger, more convenient space, explains MLSE vice president of arenas and digital technologies Christian Magsisi. For example, if you want to be right next to Kawhi Leonard’s infamous 2019 touchdown, you’ll need a large space, such as a gym or arena.
But with AR and VR capabilities, users will be able to place their experiences on smaller surfaces like desktops and help them see the action unfold in front of them.
Create a competitive advantage
One of the advantages of VR and AR experiences is for professional sports organizations that are part of MLSE. Magsisi said it allows players and coaches to relive key moments in their games and learn from them.
“It gives us a competitive advantage,” Magsisi said, noting that members of the Raptors and Leafs have already tested the technology.
Instead of needing 10 real-life players to recreate a specific play on an NBA court, you can use AR or VR to literally bring the moment to life. Players and coaches can watch and re-watch games with the help of headset technology, which shows vital statistics as they unfold and allows them to see exactly what is affecting their team’s performance.
Gales also notes how it can help them increase awareness of injuries and how to avoid them to improve player safety. This type of data and innovative research has been a focus of the NFL since 2019 with AWS.
Get people involved
While enhancing the viewing experience and helping teams gain a competitive advantage, SpaceX is driven by the concept of community engagement. Not only do they want their fans to help lead them to future ideas through submissions, but they also want to experiment with AR and VR experiences.
“We’ve been watching sports the same way for so long. Giving people virtual reality glasses is a big change. So we need to make it easier for people to do that. So, however soon it’s ready, I think it’s going to take a while. People really it’s time to adopt.”
Those interested can register immediately at spacex.io and test the technology for themselves.
More from Yahoo Sports