Oculus Passthrough: Part 3
Previously we saw how to set up a Unity project to work with Oculus Passthrough API and how we can create an augmented interactable object. If you haven’t checked out the first two parts of this series, we highly recommend checking it out as the API will not work without the right settings and the knowledge from previous parts is required here as well.
In this blog post, we’ll learn how to customize the passthrough layer. There are various ways to customize passthrough. It could be through styling or composite layering or by surface-projected passthrough implementation. Our focus, for now, is going to be on styling, where we will write a script that will allow us to change the brightness, contrast, alpha and posterize values of the passthrough.
1. Creating UI Canvas
Let's start by creating a canvas with four sliders. The values from these sliders will later be used to change the brightness, contrast, alpha and posterize values of the passthrough.
-
Right-click on the Hierarchy window and select
UI
→Slider
. -
Select the
Canvas
GameObject and rename it as PassthroughStyler → change the Render Mode toWorld Space
→ adjust its position and scale, I have considered the position as (0,0,0) and the scale as (0.01, 0.01, 0.01) → for the EventCamera, drag and drop theCenterEyeAnchor
GameObject from theOVRCameraRig GameObject
into that field. -
Select the
Slider
GameObject → adjust its position and scale. -
Duplicate the
Slider
GameObject thrice and name them as BrightnessSlider, ContrastSlider, AlphaSlider and PosterizeSlider → adjust their position such that there is a sufficient gap in between them. -
Right-click on
PassthroughStyler
GameObject and selectUI
→ selectText - TextMeshPro
→ click onImport-TextMeshPro Essentials
, that popup will be seen if you don't have it imported already, as seen in the gif→ reduce the Font Size and Center align the text. -
Duplicate the
Text(TMP)
GameObject thrice → adjust their position from top to bottom and name them as Brightness, Contrast, Alpha and Posterize respectively. -
Copy and paste the same names into the respective Text Input field.
-
Select the
PassthroughStyler
GameObject and move it in front of the camera. -
Disable the
Cube
GameObject.
The result of what we have created is the UI Canvas. Now, you hit play and view the canvas.
Note: While in play mode make any position or size adjustment if needed → copy the values → exit the play mode and paste the values back into the component.
2. Making UI Interactable
At this current state, we’ll be just able to view the UI canvas but not interact with it. To make it interactable we need to add a few components and helper prefabs to our scene.
-
From the Hierarchy window delete the
EventSystem
GameObject. -
Select the
OVRCameraRig
GameObject and add theOVR Physics Raycaster
component. -
In the Project window, search for UIHelpers → drag and drop the
UIHelpers
prefab into the Hierarchy window. -
Open the
UIHelpers
prefab’s children→ select theLaserPointer
GameObject → enable theLineRenderer
component. -
Select the
PassthroughStyler
GameObject → remove theGraphic Raycaster
component → add theOVR Raycaster
component. -
With this, we should be able to use the UI with the controller. But when you test it, you’ll not be able to interact with the UI. The reason is that the Canvas of the individual text is big and it's placed in front of the sliders. So to rectify that by:
- Right-click on
PassthroughStyler
GameObject and selectUI
→ selectCanvas
. - Name it as TextCanvas.
- Select the four GameObject
Brightness
,Contrast
,Alpha
andPosterize
, and move them into theTextCanvas
GameObject.
- Right-click on
You can now hit play and test the UI interaction.
3. Scripting
Now that we have the slider UI ready, we can write a script that will take the slider values to adjust different parameters of the passthrough layer. Thus, we will be able to customize it.
3.1 Code
In the next section we have the breakdown of the code as well, so don't worry if you are not able to understand the code immediately.
Create a new C# script, name it as StylingPassthrough and copy the following code. The code will use the events from the slider to update the parameters of the passthrough layer.
using UnityEngine;
public class StylingPassthrough : MonoBehaviour
{
[SerializeField] private OVRPassthroughLayer passthroughLayer;
private float setBrightness = 0.0f;
private float setContrast = 0.0f;
private float setPosterize = 0.0f;
private Color setEdgeColor = Color.white;
private void Start()
{
setEdgeColor.a = 0.0f;
passthroughLayer.colorMapEditorBrightness = setBrightness;
passthroughLayer.colorMapEditorContrast = setContrast;
passthroughLayer.edgeColor = setEdgeColor;
passthroughLayer.colorMapEditorPosterize = setPosterize;
}
public void OnBrightnessChanged(float newValue)
{
setBrightness = newValue;
passthroughLayer.colorMapEditorBrightness = setBrightness;
}
public void OnContrastChanged(float newValue)
{
setContrast = newValue;
passthroughLayer.colorMapEditorContrast = setContrast;
}
public void OnAlphaChanged(float newValue)
{
setEdgeColor.a = newValue;
passthroughLayer.edgeColor = setEdgeColor;
}
public void OnPosterizeChanged(float newValue)
{
setPosterize = newValue;
passthroughLayer.colorMapEditorPosterize = setPosterize;
}
}
3.2 Code Breakdown
Declaration
using UnityEngine;
public class StylingPassthrough : MonoBehaviour
{
[SerializeField] private OVRPassthroughLayer passthroughLayer;
private float setBrightness = 0.0f;
private float setContrast = 0.0f;
private float setPosterize = 0.0f;
private Color setEdgeColor = Color.white;
Variable Name | Type | Use |
---|---|---|
passthroughLayer | float | To store the value between 0 and 1 from the slider. The value will determine the brightness of the passthrough layer. |
setBrightness | float | To store the value between 0 and 1 from the slider. The value will determine the brightness of the passthrough layer. |
setContrast | float | To store the value between 0 and 1 from the slider. The value will determine the contrast of the passthrough layer. |
setPosterize | float | To store the value between 0 and 1 from the slider. The value will determine the posterize of the passthrough layer. |
setEdgeColor | Color | To store the alpha Color of the rendered edges. |
Initialization
private void Start()
{
setEdgeColor.a = 0.0f;
passthroughLayer.colorMapEditorBrightness = setBrightness;
passthroughLayer.colorMapEditorContrast = setContrast;
passthroughLayer.edgeColor = setEdgeColor;
passthroughLayer.colorMapEditorPosterize = setPosterize;
}
At the start when this script is enabled for the first time, we want the Passthrough Layer’s fields to be set to zero. In particular, we want to set the brightness, contrast, alpha value of edge color and posterize to zero. So using the properties of the Passthrough Layer the respective field is initialized to zero.
Methods
public void OnBrightnessChanged(float newValue)
{
setBrightness = newValue;
passthroughLayer.colorMapEditorBrightness = setBrightness;
}
public void OnContrastChanged(float newValue)
{
setContrast = newValue;
passthroughLayer.colorMapEditorContrast = setContrast;
}
public void OnAlphaChanged(float newValue)
{
setEdgeColor.a = newValue;
passthroughLayer.edgeColor = setEdgeColor;
}
public void OnPosterizeChanged(float newValue)
{
setPosterize = newValue;
passthroughLayer.colorMapEditorPosterize = setPosterize;
}
The UI sliders have an event called On Value Changed which gets called when the value of the slider changes. The event can send the current value as a float type dynamic argument. So by making use of that feature we can change the Passthrough Layer’s fields dynamically using public methods.
When the OnBrightnessChanged method is invoked by the On Value Changed event of the slider, it passes the slider values as float parameters to this method. Which then sets the Passthrough filter’s brightness to that value.
Other methods OnContrastChanged, OnAlphaChangedwork and OnPosterizeChanged work in the same way as well. The only then we need to be care full about is to assign the right method to the right slider
Customizing the Passthrough Layer
In this section, we’ll see how to set up the OVRPassthroughLayer component. We’ll also see how to assign the StylingPassthrough component’s methods to the slider's events.
-
To customize the Passthrough Layer, we need to enable the Layer’s control map. To do that, click on
OVRCameraRig
GameObject → under theOVRPassthroughLayer
component, select the Color Map asControls
. -
Add the
StylingPassthrough
component to thePassthroughStyler
GameObject by dragging and dropping the script on the GameObject. -
To add the parameter for Passthrough Layer, drag and drop the
OVRCamraRig
GameObject into the field. As the OVRPassthroughLayer is attached to the OVRCamraRig, it will automatically assign that to the field. -
Select all the four sliders and click on the plus ➕ button to add an action to the event list → drag and drop the
PassthroughStyler
GameObject into the event field. -
Click on
BrightnessSlider
GameObject→ choose the functionStylingPassthrough
→ under Dynamic float, select theOnBrightChanged
method.
Similarly, select one slider at a time and assign the respective function under Dynamic float for the rest of them.
With that, we have finished setting up our scene for customizing the Passthrough during run time. You can hit play and test it.
Conclusion
The whole intent of this blog was to understand and learn about the passthrough filter. When you play around with this scene you will be able to have a better understanding of the passthrough filter. What we saw above was just one part of this. There are other ways of using a passthrough filter as well as we mentioned earlier. You can find different types of example scenes showing the various implementation of passthrough in the Oculus Integration package. In the project folder navigate to Assets
→ Oculus
→ SampleFramework
→ Usage
→ Passthrough
, to find the example scenes. To learn more you can check out this documentation.
__________________
Thank you
Thanks for reading this blog post. 🧡 If you are interested in creating your own AR and VR apps, you can learn more about it here on immersive insiders. Also, if you have any questions, don't hesitate to reach out! We're always happy to help.