USDZ in AR QuickLook flies on camera level, does not acknowledge the world/camera

I’m trying to use QuickLook AR to show a 3D ThreeJS model in AR on iOS. Exporting my 3D object to USDZ using the Three USDZ exporter works fine. When opening the USDZ file in XCode everything seems to be fine.

Though, when opening the USDZ in QuickLook AR, the 3D model is flying above the ground, on my camera’s Y level. The camera PoV is positioned exactly in the middle of the X and Z axis of the 3D model and at the bottom of the Y level.

I have another problem with opening the USDZ in QuickLook AR, which is; When opening the USDZ in QuickLook AR, the model is invisible at first. Then when I scale the model down to < 10%, the model becomes visible, though it does not scale in size at all. Also, the “Model” tab in QuickLook does not even show the 3D model. When switching between the “Model” and “AR” tabs, the model flies by really quick.

For reference, I’ve added my USDZ model below.

What I’m trying to accomplish is to position the 3D model in front of me, and for the 3D model to acknowledge the world shown by the camera. The 3D model should stick to walls, or at least the floor to begin with.

Button click code:

newScene.add(sceneRef)
              const pivot = new THREE.Object3D()
              newScene.add(pivot)
              pivot.add(sceneRef)
              // position the object on the pivot, so that it appears 5 meters
              // in front of the user.
              pivot.position.z = -50

              const yaxis = new THREE.Vector3(0, 1, 0)
              const zaxis = new THREE.Vector3(0, 0, 1)
              const direction = zaxis.clone()
              // Apply the camera's quaternion onto the unit vector of one of the axes
              // of our desired rotation plane (the z axis of the xz plane, in this case).
              direction.applyQuaternion(cameraRef.quaternion)
              // Project the direction vector onto the y axis to get the y component
              // of the direction.
              const ycomponent = yaxis
                .clone()
                .multiplyScalar(direction.dot(yaxis))
              // Subtract the y component from the direction vector so that we are
              // left with the x and z components.
              direction.sub(ycomponent)
              // Normalize the direction into a unit vector again.
              direction.normalize()
              // Set the pivot's quaternion to the rotation required to get from the z axis
              // to the xz component of the camera's direction.
              pivot.quaternion.setFromUnitVectors(zaxis, direction)
              // Finally, set the pivot's position as well, so that it follows the camera.
              newScene.getWorldPosition(cameraRef.position)

              newScene.updateMatrixWorld(true)
              iosExporter.parse(newScene).then((result) => {
                saveUSDZString(result, 'scene.usdz')
              })

saveUSDZString function:

  function saveString(text: any, filename: any) {
    save(new Blob([text], { type: 'application/json' }), filename)
  }

save function:

  function save(blob: any, filename: any) {
      link.href = URL.createObjectURL(blob)
      link.download = filename
      link.rel = 'ar'
      let img = document.createElement('img')
      img.alt = 'hi'
      img.src = 'https://google.com/img'
      link.appendChild(img)
      link.click()
  }

USDZ Model: https://wetransfer.com/downloads/2d2d2e840f9f964e036cd6077094c33220220630095321/a7f94b9f2bc730fead9107bf133e175220220630095338/193b81?utm_campaign=WT_email_tracking&utm_content=general&utm_medium=download_button&utm_source=notify_recipient_email

Answered by Vision Pro Engineer in 719333022

When you open AR Quick Look and see the model float aligned to the camera, this means that it hasn't been placed in the real world yet. Depending on your environment, it might take a while until AR Quick Look finds a plane that is sufficiently large to fit your virtual object.

Looking the your specific model in Xcode, I can see that its bounding box is in the order of magnitude of thousands. ARKit's unit is meters, which means that the object's extent is several kilometers. So as a first step, you should recreate a version of your USDZ with fixed units. A length of 1 would mean that the object will appear one meter long in AR Quick Look.

Once you have done that, try previewing the model again. If you run AR Quick Look in an environment that has a enough space to fit your model, your object should automatically appear anchored in the real world.

Accepted Answer

When you open AR Quick Look and see the model float aligned to the camera, this means that it hasn't been placed in the real world yet. Depending on your environment, it might take a while until AR Quick Look finds a plane that is sufficiently large to fit your virtual object.

Looking the your specific model in Xcode, I can see that its bounding box is in the order of magnitude of thousands. ARKit's unit is meters, which means that the object's extent is several kilometers. So as a first step, you should recreate a version of your USDZ with fixed units. A length of 1 would mean that the object will appear one meter long in AR Quick Look.

Once you have done that, try previewing the model again. If you run AR Quick Look in an environment that has a enough space to fit your model, your object should automatically appear anchored in the real world.

USDZ in AR QuickLook flies on camera level, does not acknowledge the world/camera
 
 
Q