Persistent Coordinate Frames

WebXR is a browser API that allows you to experience immersive sessions. Helio is compatible with the latest WebXR version (Sept 30, 2019), which can be found here. In addition, Helio has support for augmented reality sessions.

Magic Leap specific extensions

To allow the placement of content to persist across sessions and devices, Helio added a custom API.

Lumin Runtime has a concept called Persistent Coordinate Frames(PCFs) which are predefined positions in the real world. Each PCF has a unique ID which an author can store in the browser session or on the server. The ID can also be sent to other devices.

For instance, if an author places digital content during an immersive session, they can request the nearest PCF and store the unique ID of that PCF as well as the location of a chair relative to the PCF. The next time the browser starts, the author can retrieve the unique ID of the PCF and request its location. This location is relative to the new coordinate system so the object can be placed in the exact same location. The unique ID can also be sent to another device along with other state information so an object can then be reconstructed in the same physical location.

API

NOTE: The PCF API is experimental and subject to change or to be replaced by WebXR Anchors

partial interface XRSession {
  Promise<FrozenArray<DOMString>> getPCFs();
  Promise<DOMString> getNearestPCF();
  Promise<DOMMatrix> getPCFTransform(DOMString PCF);
};

getPCFs

This is a new method on the XRSession object that returns a list of unique IDs describing all known PCFs. This list has a maximum of 100 entries.

  let known_pcfs = await session.getPCFs();

getNearestPCF

This is a new method on the XRSession object that returns the unique ID of the nearest PCF.

  let nearest_pcf = await session.getNearestPCF();

getPCFTransform

This is a new method on the XRSession object that takes a unique ID and returns the position of the PCF in the current coordinate space.

let nearest_pcf = await session.getNearestPCF();
let location = await session.getPCFTransform(nearestpcf);

API to request PCFs

Access to PCFs is not granted automatically. Instead, an author has to request the feature during the WebXR session creation.

RequestSession allows a list of required and optional features. If an author passes mlPCFs as a string in that list, PCF support is enabled.

navigator.xr.requestSession( 'immersive-ar',  {
            requiredFeatures: ["mlPCFs"],
            optionalFeatures: [] } ).then( onSessionStarted );

Notes

Make sure the perception system is localized

When you start your device, it attempts to locate the device in the real world. Make sure that the landscape is recognized when you start an immersive session.

Delay of the perception subsystem

The perception subsystem might take a couple of seconds to start working after entering the first immersive session. We recommended you wait 2-4 seconds before calling the PCF APIs. The system is also continuously learning, which might slightly shift the location of the PCF points. We recommend that you sometimes re-request the location of the PCF you are using.

Sharing of PCFs with other users

Each PCF has a unique ID. However, not every PCF is known across devices. Certain PCFs are only known to your devices while others are known across all Magic Leap devices. There is currently no way to determine which type of PCF is returned to the user.