Sub-Framing Technology in Virtual Production and XR Broadcast

Over the past few years, terms such as GhostFrame, Frame-remapping, and PhantomTrack have emerged as defining pillars of modern cinematography and broadcast. Each reflects variations of a core innovation: Sub-Framing Technology. This technology enables more seamless integration of virtual environments, addresses persistent technical constraints, and enhances the overall experience for both crew and talent on set. By utilizing industry-leading hardware, such as that provided by ROE Visual, production teams can achieve higher levels of visual fidelity and workflow efficiency.

The Power of Sub-Framing Technology

What is Sub-Framing Technology? 

In standard LED volume setups, the LED screen displays content at a specific frame rate, for example, 50 or 60fps. The camera captures each frame in sync with the screen's frame rate. Sub-Framing Technology introduces a more dynamic system, where each frame divides into multiple sub-frames (or "slices") that can display different content sequentially, such as a static color alongside video footage.

The number of sub-frames is hardware-dependent and can range from 2 to currently max. 15 at a 60fps frame rate, which increases at lower frame rates. Using sub-frames creates an environment where multiple visual elements—such as video content and chroma-key layers—are displayed simultaneously. By adjusting the camera's shutter angle and/or timing, you can select which sub-frames you capture, allowing for varied use cases and configurations.

There are two primary setups:

• Multi-Camera Setup: One camera captures the normal video feed, while another camera, using an offset reference signal, captures a different sub-frame, such as a static color or a different video feed.
• Single Camera Setup: One camera records multiple sub-frames, producing separate video streams for each, creating various output streams in real-time.

Applications of Sub-Framing Technology

Sub-framing opens up a wide range of possibilities in virtual production and broadcasting. While each application can be used independently, they are often combined to address specific technical needs on the ROE Visual volume.

Naked Eye View

This feature enhances crew and talent interaction by displaying content such as AR markers or text overlays on the LED screen that are invisible to the camera but visible to the talent. 

• AR Markers: Helps actors interact with virtual elements by providing visual markers that don't interfere with camera recordings.
• Text Overlays: Displays helpful information such as auto-cue, countdowns, or instructions for on-screen talent, improving performance without needing additional reference monitors.

Camera Tracking

Traditionally, camera tracking requires additional hardware like special cameras or reflective markers. Sub-framing technology simplifies this by displaying tracking markers directly on the LED wall, invisible to the audience but detectable by tracking systems (e.g., Mo-Sys, Stype, Zeiss), eliminating the need for extra markers and simplifying setup.

Chroma Recording

Green screens are often used to film scenes with post-production effects. Sub-framing allows simultaneous capture of virtual content and a chroma-key layer using the same camera, blending the advantages of real-time virtual production (ICVFX) with the flexibility of green screen post-production.

Multi-Source Broadcasting

Sub-framing technology can broadcast multiple versions of content simultaneously. For example, different languages or regional sponsor advertisements can be displayed and captured, tailored to different audiences in real-time across different global locales.

Multi-Camera Workflows

Broadcasting with multiple cameras on a virtual set typically requires precise coordination between the cameras and the media server. Sub-framing allows each camera to capture its unique view simultaneously, eliminating delays and enabling seamless transitions between multiple camera angles.

High Frame-Rate Capture

The technology has significantly improved LED panels' ability to display high frame-rate content, achieving frame rates up to 360fps with moving content and 900Hz with static content, without issues like scan lines or flickering.

Hidden Feed (GhostFrame)

GhostFrame displays hidden video content or chroma keys by showing an inverted sub-frame that blends into a white background when viewed by the naked eye. This feature is handy for regulatory compliance, such as hiding specific advertisements from live audiences while displaying them on a broadcast.

This feature should only be used on the best-performing LED panels, like the ROE Visual BP2V2 or RB1.9BV2 or -V3, as maximum performance is required to tune the sub-frames in a way that minimizes flickering.

Required Equipment

LED Screen Quality

The performance of the LED screen is pivotal to the success of sub-framing technology, involving two key components: the LED panels and the LED processor.

LED Panels

The performance of the LED panels is crucial. Selecting the appropriate panel driver configuration and ensuring precise PCB (Printed Circuit Board) design allows the panels to handle high data rates and maintain accurate timing. This is why many professionals choose ROE Visual for high-stakes productions.

LED Processor

The processor manages the incoming video signal and controls the LED panels. There are two primary approaches:

1. Brompton, Colorlight, and Novastar Solutions: These use "Frame-remapping," where sub-frames are generated directly on the processor. This requires a higher data load and additional processor capacity.
2. Megapixel's Camera Mode (GhostFrame): This solution offloads much of the sub-framing workload to the receiver card within the LED panel. It reduces data transmission requirements while maximizing the LED panel's potential, reaching speeds up to 900Hz.

Camera Specifications

Rolling vs. Global Shutter Cameras

Global shutter cameras are the preferred option for working with LED screens. Rolling shutter cameras require more careful configuration to avoid scan lines when the shutter angle is reduced to capture specific sub-frames.

Single-Speed vs. High-Speed Cameras

Standard cameras are "single-speed." However, high-speed cameras (like the Sony HDC5500 or Grass Valley LDX150) offer additional capabilities by generating "phase outputs" offset by fractions of a frame, ideal for multi-source sub-framing recording.

Synchronization and Lighting

• Genlock: It is essential to synchronize all equipment using a high-quality Sync Pulse Generator (SPG).
• Lighting: Reduced shutter angles require precise, flicker-free lighting setups to provide enough brightness for the stage.

Conclusion

By leveraging Sub-Framing Technology, production teams can achieve greater flexibility and streamline virtual production workflows. Whether it's camera tracking or multi-source broadcasting, sub-framing represents a crucial innovation. With hardware from ROE Visual, the possibilities for immersive storytelling continue to expand.