XRAuthor

February 27, 2025

Avatar for Chek Tien TAN
Chek Tien TAN    
Associate Professor

Read More 

Avatar for Songjia SHEN
Songjia SHEN    
Researcher
Avatar for Tuck Wah LEONG
Tuck Wah LEONG    
Researcher
Avatar for William RAFFE
William RAFFE    
Researcher
Avatar for Tim CHEN
Tim CHEN    
Researcher

The core design of the tool is to help educators create interactive learning content in multiple formats through a simple and familiar process.

It enables users to record demonstration videos with markers and requires only minimal user input for video segmentation to automatically generate the following three formats (see Fig. 1) of interactive learning content:

  • Video Format: footage recorded by a webcam, augmented with virtual object overlays, segmented into steps, with multiple-choice questions for each step, used on a computer
  • Interactive Format: animations of virtual objects with/without a video background, with interactive tasks for each step, used on a computer
  • VR Format: animations and tasks within immersive VR environments (classroom or other relevant surroundings), with the ability to show or hide the recorded video.
learner-task-768x216.png

Fig. 1. (A) Video Format: a participant answering a multiple-choice question in a standalone web app, (B) Interactive Format: a participant interacting with cubes to complete a step of the learning content embedded on the Wikipedia page for tree sort, and (C) VR Format: a participant completing a step in VR.

author-ui-1-768x634.png

Fig. 2. A video recording is being edited in the tool.

Considering accessibility, the team implemented the tool to run in a web browser on a computer without requiring any installation. It has a minimalist UI (see Fig. 2), with four buttons for the tool’s major modes at the top, a viewing area for displaying authored content in the middle, and a progress bar with editing and playing controls at the bottom.

author-process.png

Fig. 3. Tool Authoring Process

After setting up the markers, the authoring process can start with recording a regular demonstration video and then proceed through an edit-review loop on the video footage until the author is satisfied with the content and uploads it to the server (see Fig. 3). The content can be used as a standalone web app or embedded in existing web pages. Learners can access the content online in their preferred format (see Fig. 1).

Live demo:

Author app (tree sort) 

Learner app (tree sort) 

The team is currently conducting studies with educators, learners and developers to evaluate their user experience with XRAuthor.