Big Picture delivers landmark live xR broadcast
- Details
“Usually the Dally M Awards are staged in a large event venue, but with COVID-19 we had to rethink how we would stage the 2020 production,” remarks Paul Slater, senior producer at Fox Sports. “We still wanted to have a big auditorium feel, but we didn’t have the budget or resources to shoot something like that in reality.”
Paul turned to Big Picture for inspiration, who in turn demonstrated how an xR (extended reality) broadcast could work. The technology impressed, marking a vote of confidence for Big Picture, as a live broadcast xR production had never been done anywhere in the world.
xR technology allows users to blend virtual and physical worlds in live production environments to create fully immersive experiences.
“We were approached by Fox Sports who had been looking at several technologies to integrate a virtual component into a live multi-camera environment,” explains Josh Moffat, special projects & business development at Big Picture. “With only a short lead time up until the event, we designed a technical solution around the disguise xR workflow.”
The event took place at Fox Sports in Sydney where the main Studio setup comprised a 12m x 3m ROE Black Onyx 2.8mm LED Wall integrated into the disguise xR workflow. The seamless extension of Big Picture’s real-world LED screens to the virtual world environments could only be done by disguise’s xR multi-camera registration workflow, allowing switching between camera perspectives and the LED content. Multi-camera switching between perspectives also allowed them to switch cameras, and the LED content.
Three Panasonic 4000 camera chains ran a 1080P workflow then integrated downstream into Fox Sports 1080i transmission path. Big Picture’s UHD-2 PPU System with a ROSS Ultrix and Carbonite Ultra was at the heart of the system managing routing, monitoring and conversion.
“Two Panasonic AW-HD150 UHD PTZ cameras were in Clean Zone studio green screen to bring players into the main set virtually to interact with hosts,” adds Josh. “There were virtual objects in real-time in the 3D Scene such as the leader board, player profiles and a virtual big screen fed from a mixture of disguise timeline and external EVS feeds.”
Five disguise gx 2c servers were used with Lightware HDMI 2.0 Matrix and Ross Ultrix SDI Matrix for failovers.
Stype RedSpy camera tracking devices were mounted on all three cameras, one studio pedestal, a jib and one Stedicam operated by Fox’s Glenn Steer who did an amazing job with all the virtual graphics. The Stype RedSpy figures out where the camera is in 3D space and where it’s pointing - its position, angle, and field of view.
The Notch real-time engine then quickly drew the background scene from that point of view, before rendering the resulting image onto the screens on stage thus creating the illusion that the performers and props are actually in the virtual scene.
The Notch programming and integration was flawlessly executed by Ryan Sheppard, based in Canada. Big Picture had Ryan in communications in real-time for rehearsals using a low latency multiview stream and ClearCom Agent-IC for communications. He was then connected with remote desktop to the Notch file onsite, making changes as required.
“We used Notch to render the 3D word designed by Fox Sports in real-time,” comments Ryan. “Performance of the Notch scene was of utmost importance for a live to air multi-camera xR broadcast as there was no tolerance for dropped frames. We were able to achieve this level of performance by using a strategic combination of baked and unbaked textures that allowed for some dynamic elements while maintaining maximum performance of the scene.”
Working remotely on any project has its challenges, as Ryan is located in Toronto the biggest challenge for him was shifting his sleep schedule to match that of the Big Picture team in Sydney.
“However, once I was on the same sleep schedule we were able to develop a workflow that allowed me to communicate via ClearCom Agent-IC as well as a VNC connection over AnyDesk to a computer on the network that allowed me to network edit the block during programming and rehearsals if necessary. Once we had this workflow developed with Agent-IC, VNC and low latency multi-views, I might as well have been sitting in the studio.”
Nathan Barnier, senior technician & content specialist at Big Picture, observed that there were several challenges with this project; time being the major challenge but also no LED floor and having a flat LED wall had an impact on what is a more traditional xR workflow.
“Fortunately we were able to build the cameras along with a test strip of the LED in the corner of the studio so we could get ahead to achieve frame accuracy and seamless switching before the main LED wall build the night before going to air,” he reveals. “Anyone with experience in the disguise xR workflow would acknowledge that frame accuracy and perfectly timed camera, tracking and composite delay is extremely important and the underlying factor to the successful execution of multi-camera xR.”
Having no LED floor is something that Ryan and Nathan tested a few weeks before the broadcast. Fox Sports wanted a gloss finish floor which would allow for reflections from the LED wall and virtual scene to be realised in the real environment. A combination of alpha blend mode in the Notch block and a dummy LED floor in disguise was able to deliver a great outcome. Notch reflections of front plate objects such as players worked well on the high gloss acrylic chosen by Fox Sports.
The set up also ensured everyone was COVID-safe as disguise xR is ideal for avoiding non-essential contact, mitigating the risks posed by traditional approaches to filming immersive visuals which would involve high-level, real-time in-camera shoots, green screen and other VFX.
Douglas Heriot ran the content from a MA Lighting grandMA2 console.
“Fox Sports were very happy with the result and I’m sure most people watching would have no idea of the technology used as it worked so seamlessly,” says Paul. “It was the perfect solution. We cover such a wide array of different sports and we like all of our hostings to look different to match the code, so to be able to change your set at the flick of a button is exciting to us and something we’ll be looking to use more in the future.”
Paul notes that Big Picture went above and beyond to rig and set up the system in a near impossibly short time frame: “They guys worked around the clock to make it happen which gave me all the confidence in the world that it was going to work!”
