3D User Interfaces

Question 1

3D Interaction

Answer 1

• Human-computer interaction in which the user’s tasks are carried out in a 3D spatial context
• 3D input devices, 2D input devices mapping into 3D

Question 2

3D User Interface (UI)

Answer 2

A UI that involves 3D interaction

Question 3

3D Interaction technique:

Answer 3

A method (hardware and software) allowing a user to accomplish a task in a 3D UI

Question 4

What makes 3D Interaction difficult?

Answer 4

Spatial input
Lack of constraints
Lack of standards
Lack of tools
Lack of precision
Fatigue
Layout more complex
Perception

Question 5

Mine’s Fundamental Forms of Interaction in Virtual Environments

Answer 5

Movement: for a user to move through the virtual world
- Parameters: speed, direction

Selection: ways to indicate the target of the desired interaction
- Requires: Method to 1) identify object of interest (Pointing), 2) Signal or command to indicate act of selection (button press)

Manipulation: Specification of an object’s position and/or orientation in the virtual world.
- Parameters: change in position, change in orientation and center of rotation.

Scaling: to get components at their correct relative score, and allow a user to view some small detail
- Parameters: the center of scaling and the scaling factor

Menu: powerful way to add function to a virtual environment application
- Parameters: Dimensionality (1D, 2D, 3D), among many others

Question 6

Object Selection: Goals of Selection

Answer 6

• Indicate action on object
• Query object
• Make object active
• Travel to object location
• Set up manipulation

Question 7

Selection Technique Classification

Answer 7

Question 8

Simple Virtual Hand Technique

Answer 8

Process
* One-to-one mapping between physical and virtual hands
* Object can be selected by “touching” with virtual hand
* “Natural” mapping

Limitation:
* Only select objects in hand reach

Question 9

GO-GO Technique

Answer 9

• Arm-extension technique
• Non-linear mapping between physical and virtual hand position
• Local and distant regions (linear < D, non-linear > D)

Question 10

Ray-Casting Technique

Answer 10

“Laser pointer” attached
to virtual hand
* First object intersected by ray may be selected
* User only needs to control 2 DOFs
Proven to perform well for remote selection
Variants:
* Cone casting
* Snap-to-object rays

Question 11

Virtual Hand vs Pointer

Answer 11

Virtual hand:
* Direct, physical, natural, resembles the real world
* Fast performance
* As accurate as hand/finger
* Only for reachable objects
* Occlusion of view through own hand

Virtual pointer:
* Interact with out-of-reach objects
* Requires pointing & confirmation
* Slower, but accurate
* Less occlusion

-> Depends on interaction technique, task, …

Question 12

Image Plane interaction

Answer 12

Question 13

Object Manipulation: Goals of Manipulation

Answer 13

• Object placement
  *Design
  *Layout
  *Grouping
• Tool usage
• Travel

Question 14

Taxonomy of Manipulation Techniques

Answer 14

Question 15

Egocentric vs Exocentric metaphor

Answer 15

Question 16

Common Manipulation Techniques

Answer 16

• Simple virtual hand
• HOMER
• Scaled-world grab
• World-in-miniature
• Gaze + Pinch

Question 17

Simple Virtual Hand Manipulation

Answer 17

Question 18

HOMER Technique

Answer 18

H: Hand-Centered
O: Object
M: Manipulation
E: Extending
R: Ray-Casting

• Selection: Ray-casting
• Manipulate: directly with virtual hand
• Include linear mapping to allow wider range of placement in depth

Question 19

Scaled-World Grab Technique

Answer 19

• Often used w/ occlusion
• At selection, scale user up (or world down) so that virtual hand is actually touching selected object
• User doesn’t notice a change in the image until he moves

Question 20

World-In-Miniature (WIM) Technique

Answer 20

• “Dollhouse” world held in user’s hand
• Miniature objects can be manipulated directly
• Moving miniature objects affects full-scale objects
• Can also be used for navigation

Question 21

Gaze + Pinch Interaction

Answer 21

• Hands act: Gaze only indicates targets, but hands manipulate it.
• Target snapping: The target closest to gaze is always selected.
• 1:1 translation: Object moves like hand moves

Question 22

Design Guidelines

Answer 22

• No single best manipulation technique
• Optimise the interaction technique vs the device/task
• Reduce degrees of freedom, clutching
• Pointing techniques for selection vs grasping technique for manipulation
• Established techniques vs designing a new application-specific method
• Trade-off between technique design vs environmental design

Question 23

Conclusions

Answer 23

User interface key for good XR experience
* Usability one of the most crucial issues facing VE applications
* Implementation details critical to ensure usability
* Ease of coding not equal to ease of use
* Need 3D user interface techniques

Design for
* Selection/Manipulation
* Navigation
* System control

Follow good design guidelines
* Cannot just implement 2D techniques in XR