MLSE aims to bring virtual reality, other tech innovations to live sports viewing

The NHL Extended Reality Stats Overlay glasses, developed by SportsX, a research and development program funded by MLSE and AWS. (MLSE)

What if you could strap on a headset and immediately find yourself in the middle of a professional basketball game, rather than just watching it on TV?

It’s an ambitious idea, but its one that might not be all that farfetched, thanks to a new research and development partnership between Maple Leaf Sports and Entertainment (MLSE) and Amazon Web Services (AWS) that could change the way fans watch live sports.

The program, called SportsX, aims to create “new, innovative digital solutions that achieve technological advancement in sports to enhance team performance and the fan experience.”

And some of those advancements are already here – in the form of virtual (VR) and augmented reality (AR) technology.

At a launch event at Scotiabank Arena Monday night, the SportsX team unveiled three headsets that use VR and AR, two of which allowed users to step into a simulated basketball game, while the third provided an augmented viewing experience of the live Toronto Maple Leafs game happening on the ice below.

“We’re thrilled to announce SportsX, a new incubator in collaboration with AWS rooted in research, applied sciences and product development,” said Humza Teherany, Chief Technology & Digital Officer at MLSE.

“The program is an extension of our ongoing work with AWS and combines the expertise from both of our organizations with ideas shared by our passionate fans, to create a powerhouse in sports innovation.”

THE IMMERSIVE BASKETBALL EXPERIENCE

In the first demonstration, users could strap on a headset and step into a life-sized, virtual basketball game being played on the Toronto Raptors practice court in the arena.

The headset uses optical data to create the illusion that the game is going on around you, with players passing, dribbling and shooting the ball on the practice court nets.

The development team took a sample of a real basketball game and used optical tracking body pose data captured using computer vision, and processed it to generate a 3D rendering of the data that runs at around 60 or more frames per second, providing a smooth game simulation.

While the players in the simulation, for the time being, aren’t detailed or identifiable aside from the different jersey colours of the two teams, Teherany says that fans could soon be able to use VR to step into a simulated game being played by their favourite NBA teams.

The simulation can also be scaled up or down depending on the size of the user’s space, meaning fans could eventually use these types of headsets in their homes or even outside. 

Courtesy of MLSE.

“It could get to the point where you’re running outside next to [Raptors player] Pascal Siakam,” Teherany said.

THE NHL EXTENDED REALITY STATS OVERLAY

The second demonstration took place in an upper bowl suite overlooking the live hockey game between the Maple Leafs and New York Islanders.

There, users were given a pair of glasses connected to a smart phone. The glasses provided AR overlay as you watched the game, allowing you to track the puck or a specific player from either team.

The glasses displayed player statistics and other information and also featured an option to display a roaring polar bear in a Leafs jersey at centre ice.

The real-time statistics are taken from NHL Edge – the League’s Puck and Player tracking technology system, which has been deployed in all 32 NHL arenas and is used in all NHL games.

In addition to the glasses, the SportsX team unveiled a digital table top that displayed a mini version of the Leafs game being played below, which tracked player and puck movement in real time.

Courtesy of MLSE.

During the demonstration, you could hear the puck slam against the boards in the arena as you watched its location on the screen.

With the press of a button, you could see which players had skated the fastest that night or who had won the most face-offs.

THE FUTURE OF SPORTS TECHNOLOGY

While these two demonstrations showed some of the innovative ways that SportsX aims to transform the live sports viewing experience, in many ways, this is just the beginning.

“We don’t want to just bring out solutions that have been done before, we want to invest in early solutions and invest in solving problems early and then show them to the world, which is what you’re seeing here today,” said Christian Magsisi, Vice President of venue and digital technology at MLSE.

SportsX says that in addition to using technology to create new and exciting fan experiences, they want to be at the forefront of using it to give MLSE’s teams, which include the Raptors, Leafs, Toronto FC and Toronto Argonauts, a competitive advantage.

“I don’t know how many other teams, in all the leagues, can say that they have a dedicated research and development team partnered with one of the largest technology companies in the world, working on a technology that will at some point disrupt sports,” Magsisi said.

“So that is where we believe the competitive advantage will come from, but the point was to invest in it early.”

The program was also created with the goal of creating positive social and environmental impact by making AR and VR technologies more accessible and by using fans’ feedback to solve problems that already exist or may come up in the future.

“The [SportsX] program is designed with a bottom-up approach to capture, analyze, and build out the best ideas from key stakeholders, which include coaches, fans, partners, and employees,” MLSE said in a press release.

“Chosen ideas will be evaluated for viability, developed, and tested in the MLSE environment with the goal of creating solutions that can benefit the sports industry at large.”

Courtesy of MLSE.

As for when fans can expect to see VR and AR headsets in the stands at sports games or on shelves to be purchased, Magsisi says it’s coming sooner than people might think.

“Part of the reason why we wanted to do this public launch today is we want feedback from fans. So we want you to go on our website, give us your information, and you’ll be able to see this technology early so that we can incorporate your feedback into the final product,” he said.

“Research and development programs are created to solve the early technology problems so that we can release this, whether it’s on the Leafs app or the Raptors app or on your broadcast TV; we want this to be ready for product and the real world very soon.”