Game Control

The game control page allows control of external applications from a remote device. Generally these are Unity/Unreal apps that are integrated with the Igloo Toolkit.

Usage

Click or tap the “Control game engines” button to begin controlling external applications.

You can control orientation, movement, and game controller button presses in a number of ways which are outlined below.

“Orientation” in an immersive environment is actually controlling the on-screen, world-space cursor. You’ll see this move on the ICE Canvas when you change your orientation through any of the control methods.

If you’re using a mouse and keyboard, your mouse will be locked to the Control Panel window, and all further mouse movement and keyboard interactions will be forwarded. Press [Escape] to unlock your mouse and stop controlling the ICE cursor.

Mouse movement will adjust your rotation within the scene. Left clicking will send an A button press.
Keystrokes will send movement messages or button presses. See below for a keymap of these controls.

If you’re using a touchscreen device with gyroscope support, and your device is in a portrait orientation, the page will try and use data from your device’s gyroscope to control your orientation.
You can control your movement with a virtual joystick. Tap + drag anywhere on the dotted area to start using this joystick.
Buttons are shown at the bottom of the screen.

It’s possible when controlling orientation with a gyroscope that you’ll be physically facing a different direction to the on-screen cursor.
To adjust for this, hold down the cube in the top right and rotate your device/body to point at the on-screen cursor. When you release the cube, you’ll begin controlling orientation from this new direction instead.

On iOS, you’ll be asked to approve use of the gyroscope when pressing the “Control game engines” button. If you deny this, the page will fall back to its no-gyroscope behaviour (described below).

If you’re using a touchscreen device without gyroscope support, you pressed the “Control game engines” button with your phone in a landscape orientation, or denied permission to use the gyroscope, the page will provide two areas for virtual joysticks - one for orientation and one for movement.

Tap + drag anywhere on the screen to start using a virtual joystick.
Joysticks on the left hand side of the screen will control movement.
Joysticks on the right hand side of the screen will control orientation.
Buttons are shown in the top-right of the screen.

Keyboard controls

Control method selection

Pre-requisites

As for all ICE Control Panel usage, the Igloo Core Service must be installed and running on the media server.

The Igloo Controller Service also needs to be installed and running on the media server.

See Desktop UI - Setting up a Game Engine Layer for instructions on setting up an ICE layer to display the application, and correct configuration of that application (if using the Igloo Toolkit).

External applications that you want to control need to either be using the Igloo Toolkit (for Unity/Unreal), or accept the OSC messages outlined below.

OSC output

Messages to the Igloo Controller Service are sent on port 10000. Messages to external applications are sent on ports 9007 and 9008.

Movement → Igloo Controller Service

/controller/0/leftThumb [x] [y]

x and y are the virtual joystick’s coordinates, either from an on-screen virtual joystick or calculated from WASD presses.

Results in software Xbox controller moving its left thumbstick

Orientation → External application

/ximu/euler [y] 0 [x]

x and y are components of a rotation Euler describing the user’s current world-space rotation.
This is initialised at (0, 0, 0) when the user presses the “Start controlling game engines” button, and reset whenever they stop casting.
Changes to the rotation are tracked by the page and the resulting world-space rotation is provided. This means that the game engine’s camera will jump back to a (0, 0, 0) rotation between control sessions.

The Igloo Toolkits for Unity and Unreal accept this OSC by default.

Buttons → Igloo Controller Service

/controller/0/[button] where button is one of [ 'buttonX', 'buttonY', 'buttonA', 'buttonB' ]

Results in software Xbox controller pressing the button.