ARCore tutorial

How to add UI elements to AR scene in ARCore

Getting Android UI elements in augmented reality scene

Developing for mobile AR is very different from a standard mobile app with buttons and lists and the same goes for user interfaces and experience.

If you are planning to develop or design an AR feature I recommend to check out ARCore Elements app on Google Play Store. ARCore Elements can give you an introduction to principles and patterns for building high-quality augmented reality with simple examples and illustration scenes in AR.

Screen capture from ARCore Elements app

Types of UI elements for AR scene

There are 2 types of UI that we can add to our AR feature:

Elements on the screen:

While elements on the screen is very familiar pattern for many users, there might be limited space available to fit all the action there. User should get maximum screen space to enjoy AR experience and excessive UI might be in the way (picture below, illustration to the right).

Elements in AR scene:

While working with AR, we place objects in 3D environment. We can interact with 3D objects, walk around and collaborate. That is why it feels natural to add UI elements as part of AR scene. This can be actions to manipulate a 3D object and some additional information and actions.

Screenshot from ARCore Elements app

Adding UI elements with ARCore

To be able to add 2D Android views in 3D space we need to use ViewRenderable class from ARCore. Use setView to bind Android layout file to a renderable object. Like any other layout, you can set ClickListeners to UI elements to be able to handle any user interaction.

Add ViewRenderable to scene as any other Renderable:

Final result

Source code:

Kristina-Simakova/ar_ui_interface

Next:

Customise Instruction View in ARCore

How to attach UI elements to ARCore Node

Shapes, textures and materials with ARCore and Sceneform

Originally posted in creativetech.blog:

How to add UI elements to ARCore scene