Mar 14 • Ashray Pai

Ready Player Me Avatars in AR

Previously we learned to integrate Web View on the mobile app to create an avatar and add it to the scene. If you haven’t checked out the first part of this series, we highly recommend checking it out, since the skills, knowledge and project settings from it are required here as well.
In this blog post, we’ll learn how to augment the imported avatar on a horizontal plane. We’ll also add animations and move them around by tapping at the desired location on the plane.
YouTube

Want to watch the video instead?

1. Installing Packages and AR Setup

Let’s begin by downloading the AR packages.

  • Click on WindowsPackage Manager → select the ARFoundation v4.1.9 and click Install .
  • Also, select ARCore XR Plugin v4.1.9 and click Install.
  • Close the packager manager and now we can enableAR Core by navigating to Edit Project Settings. Here, select the XR Plug-in Management tab and check AR Core.

2. Setting up the scene for AR:

Now let’s set up the scene for AR by adding a few components.

  • In the Hierarchy, right-click and select XRAR session. This will create a new AR Session GameObject with AR Session component and AR Input Manager component
  • Next, right-click in the hierarchy window once again and select XRAR Session Origin. This will create a new AR Session Origin GameObject with the AR Session Origin component. It also creates an AR Camera Gameobject as a child with required components like a Camera, AR Pose Driver, AR Camera Manager, **and AR Background.
  • Additionally, add AR Plane Manager component to the AR Session Origin GameObject
  • The AR Plane Manager component requires a plane prefab. This prefab will be the visual representation of the planes getting detected. So, to create a plane prefab:
    • Right-click in the Hierarchy window and select XRAR Default Plane.
    • Drag and drop it in the project folder to convert it as a prefab.
    • From the project folder, drag and drop the AR Default Plane prefab into the AR Plane manager component.
    • Finally, delete it from the scene.
  • Later on, we'll be detecting horizontal planes and when a plane of a certain size is found, we’ll have to assign it a tag. So, create a tag called Floor by navigating to Edit Project settingsTags and Layers → under Tags, click on the plus button to add a new tag and name it as Floor.

3. Downloading Animations

The Ready Player Me SDK comes with an inbuilt idle animation and a walking animation. We need to download the animation to rotate the avatar either on the left side or the right side.

  • Before we download the animation, create an empty folder and name it Animations in the Project window.
  • On your web browser, visit the Mixamo website and upload your avatar → search for turn animation → select the appropriate animation for right turn→ click on Download → select the Format as FBX for Unity and save it on your device.
  • Then, check the box ✅ for Mirror and download it once again → Rename it from Right turn to Left turn and save it on your device.
💡Not sure how the animation is downloaded, feel free to check out this video.
  • Now switch back to Unity and import the downloaded FBX files in the Animations folder. You can do that either by dragging and dropping or simply right-click in the Project window, select Import New Asset and then select the two files.
  • As we want the animation to match with the Avatar’s rig, select both the models → click on the Rig tab → select the Animation Type as Humanoid → select the Avatar Definition as Copy From Other Avatar → select MaleAnimationTarget V2 or FemaleAnimationTarget V2, as per your choice → click Apply.
  • Select the models one at a time → click on the Animation tab → check the box for Loop Time

4. Setting Up The Animation Controller

We’ll have to create and set up an Animation controller with different boolean parameters. With the help of those parameters, we’ll be able to transition between different animations states.

  • Navigate to AssetsPluginsWolf3d Ready...ResourcesAnimatorControllers.
  • Create an Animator controller by right-clicking in the Project window → CreateAnimator Controller → name it ‘AvatarAnimator’
  • Open the AvatarAnimator→ click on the Parameters tab → add a Bool parameter by clicking on the plus sign + and name it isMoving. Similarly, add two more bool parameters and name them as isTurningLeft and isTurningRight.
  • We need to create four animation states for our avatar and those are Idle state, Walking state, Turning Left state, and Turning Right state. To do that :
    • Right-click on the base layer→ Create StateEmpty→ name it as Idle → add the Breathing Idle animation in the motion field.
    • Create another empty state → name it as Walking → add the Walking animation in the motion field.
    • Similarly, create two more empty states → name it as LeftTurn and RightTurn → add the LeftTurn and RightTurn animation in the motion fields respectively.
  • Now, to make transitions between the states:
    • Right-click on the Idle state → select Make Transition and then click on Walking state. Next, right-click on the Walking state, click on Make Transition and then click on Idle state.
    • Similarly, make the transitions between Idle state and LeftTurn state, and also between Idle state and RightTurn state
  • Let’s add the parameter which will define the condition as to when the transition should happen :
    • Click on the transition arrow from Idle to Walking state → in the Inspector window, uncheck Has Exit Time box → click on the Plus button ➕ under Condition and make sure the parameter chosen is isMoving and the value is true. Next, click on the transition arrow from Walking to Idle state → uncheck Has Exit Time box → click on the Plus button + and make sure the parameter chosen is isMoving and the value is false.
    • Click on the transition arrow from Idle to LeftTurn state → in the Inspector window, uncheck Has Exit Time box → click on the Plus button ➕ under Condition and make sure the parameter chosen is isTurningLeft and the value is true. Next, click on the transition arrow from LeftTurn to Idle state → uncheck Has Exit Time box → click on the Plus button + and make sure the parameter chosen is isTurningLeft and the value is false.
    • Click on the transition arrow from Idle to RightTurn state → in the Inspector window, uncheck Has Exit Time box → click on the Plus button ➕ under Condition and make sure the parameter chosen is isTurningRight and the value is true. Next, click on the transition arrow from RightTurn to Idle state → uncheck Has Exit Time box → click on the Plus button + and make sure the parameter chosen is isTurningRight and the value is false.

5. Scripting

In order to place the avatar on the detected surface and move them to the touch location, we’ll have to write a few scripts. We already have a script to import the avatar from the Ready Player Me website using web view. We need to modify it a little to meet our requirements.

5.1 Updating Avatar Importer

  • Open Avatar Importer script and add the following code above the Start method:
  • Modify the ImportAvatar and StoreAvatar methods as follows:
  • In the ImportAvatar method, the reason for writing the LoadAvatar API with a different overload is because earlier we used to receive the avatar GameObject when it was imported but we need the avatar when it's loaded.
  • In the StoreAvatar method, we are shrinking the size to 2/10th its original size so that It can be easily seen In AR. You can scale it as per your choice.
  • We’ll also be invoking an event when the avatar gets stored.
The entire script should look like this :

5.2 AR Plane Detection 

Create a new script named ARPlaneDetection and copy the following code. The code will make a list of all the planes ARPlaneManager has detected, filter out the 1st plane that has an area of more than 0.5 units i.e 0.5m square.

Once that plane is found,

  • The tag “Floor” will be assigned to the found plane
  • The AR Plane Manager component will be disabled so that further planes are not detected.
  • The already detected planes will be disabled.
  • Finally, this component will be disabled as well to unsubscribe from the events.
As we are disabling the AR Plane Manager, the shape and size of the plane get fixed. The plane will not grow bigger in size even if you scan other areas. As AR plane scanning is not perfect, leaving the AR Plane Manager enabled will allow the found planes to grow but it will detect new planes which can overlap the existing one. When you tap at that particular location the avatar will not move, this can break the experience.

5.3 Tap To Place:

Create a new script named ARTapToPlace and copy the following code. The code will store the new touch position on the found plane and invoke an event. Later on, this new touch position will be used to move the avatar to that position using the event.

5.4 Avatar Animator Controller:

Create a new script named AvatarAnimationController and copy the following code. The code will be used to start and stop the appropriate animation when the respective methods are called. i.e start turn animation when the avatar has to turn, walk when the avatar has to move, etc

5.5 Movement Controller :

Create a new script named MovementController and copy the following code. The code will listen to the Unity event declared in the ARTapToPlace script. When the event gets invoked, the avatar is made to rotate and move it to the new position.

Angle calculation Explained:

  • The API SignedAngle will return the smaller of the two possible angles between the two vectors, therefore the result will never be greater than 180 degrees or smaller than -180 degrees.
  • The SignedAngle method requires 2 vectors to calculate the angular difference and a third vector around which the other vectors are rotated.
  • Legend for the image below :
    • Vector A is Avatar's forward vector
    • Vector B is the vector joining Avatar’s position to Touch position 1
    • Vector C is the vector joining Avatar’s position to Touch position 2.

Case 1: Considering touch position 1 For this case, the angle will be calculated between Vector A and Vector B in a clockwise direction as that’s the shortest path. So the returned value will be positive between 0 and +180.

Case2: For this case, the angle will be calculated between Vector A and Vector C in a clockwise direction as that’s the shortest path. So the returned value will be positive between 0 and -180.

Dot product for rotation explained

  • When the rotation animation gets played the whole avatar rotates and so does its local transform.
  • If the dot product of two normalized vectors is:
    • 1 then, the vectors are in the same direction.
    • -1 then, the vectors are in opposite direction.
    • 0 then, the vectors are 90deg to each other.
  • Legend for the image below :
    • Vector A is Avatar's forward vector
    • Vector B is the vector joining Avatar’s position to Touch position.
    • Vector C is the Normalized vector joining Avatar’s position to Touch position.
  • As the avatar rotates, the avatar’s forward vector changes and so does the dot product.

6. Adding Components 

Let’s now add the components we have created to get the desired result.

  • Add the ARPlaneDetection component to the AR Session origin GameObject→ drag and drop the required fields i.e the AR Plane Manager and the Floor prefab.
  • Add the ARTapToPlace component to the AR Session origin GameObject → drag and drop the ImportAvatar and the AR Camera components in the respective fields.
  • Add the AvatarAnimationController component to the AR Session origin GameObject → drag and drop the required fields i.e the ImportAvatar and the AvatarController we had created earlier
  • Finally, add the MovementController component to the AR Session origin GameObject → drag and drop the ImportAvatar, the ARTapToPlace and the AvatarAnimationController into the respective fields.
With that, we have completed creating our scene. All that’s remaining now is to test it and see if it works as intended. Ps: It will work correctly, I have already tested it and made sure all the above code is correct... haha.

7. Testing

To test the app,

  • Connect your phone to your laptop/ PC. Also, make sure USB Debugging is enabled on your phone.
  • Navigate to FileBuild Settings → click on Add Open Scenes → click on Build and Run.
Wait till the build gets completed and the app will automatically be launched on your phone. Now you can test it by tapping at a different location on the floor and by tapping outside the floor area. While testing, you can see that the avatar moves to the location of touch on the floor. If you touch anywhere apart from the floor the avatar will not move.

Conclusion

In this blog, we saw how we can use the imported Avatar for AR. So what next? Well, what we saw was just one application the Ready Player Me Android integration. There are many other ways you can make use of the avatar as well, for example, you can use it as your main character in a game.
You can try out different things in AR as well. You can have a joystick to control your Ready Player Me Avatar in AR, make it run, jump, etc.

Thanks for reading this blog post 🧡

If you've enjoyed the insights shared here, why not spread the word? Share the post with your friends and colleagues who might also find it valuable.
Your support means the world to us and helps us create more content you'll love.