How can I increase the image tracking accuracy?

Hello,
I'm building an AR image tracking application that detects some images and play video on each image.
In most cases there is no problem, but some images are not detected.

So, I tried these.

  1. when I init ARReferenceImage, I set the physicalWidth value to be same as the actual size. (It is 10cm)
  2. I change the contrast of image.

But it didn't work very well.
I would be very grateful if you could give me a small hint.
Here is my code.

///Model
struct PhotoCard {
  var imageName: String
  var image: UIImage
  var videoURL: URL
}

struct TrackingItem {
  var referenceImage: ARReferenceImage
  var videoURL: URL
}

final class CameraModel {
  private let dataManger = DataManger.shared()

  private func getPhotoCards() -> [PhotoCard] {
    //get photocard from server
    return dataManger.getPhotoCards()
  }
   
  func trackingItems() -> [TrackingItem] {
    let photocards: [PhotoCard] = getPhotoCards()
    var items: [TrackingItem] = []
     
    photocards.forEach { photoCard in
      guard let cgImage = photoCard.image.cgImage else { return }
      let referenceImage = ARReferenceImage(cgImage, orientation: .up, physicalWidth: 0.1)
      referenceImage.name = photoCard.imageName
      let item = TrackingItem(referenceImage: referenceImage, videoURL: photoCard.videoURL)
      items.append(item)
    }
    return items
  }
}
///viewController
import SceneKit
import ARKit
import AVFoundation
import SpriteKit
import RxSwift


final class CameraViewController: VideoPlayerPresentableViewController, ARSCNViewDelegate {
        
    @IBOutlet private var sceneView: ARSCNView!
    private let model = CameraModel()
    private var data: [TrackingItem] = []
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        setupSceneView()
        setupTrackingConfiguration()
    }
    
    override func viewWillDisappear(_ animated: Bool) {
        super.viewWillDisappear(animated)
        
        sceneView.session.pause()
    }
    
    private func setupSceneView() {
        sceneView.delegate = self
        let scene = SCNScene(named: "SceneAssets.scnassets/photocard.scn")!
        sceneView.scene = scene
    }
    
    private func setupTrackingConfiguration() {
        data = model.trackingItems()
        
        let configuration = ARImageTrackingConfiguration()
        configuration.isAutoFocusEnabled = true
        let trackingImages = data.map { $0.referenceImage }
        configuration.trackingImages = Set(trackingImages)
        configuration.maximumNumberOfTrackedImages = trackingImages.count
        
        sceneView.session.run(configuration)
    }
    
    func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
        guard let current = data.first(where: { $0.referenceImage.name == imageName }) else { return }
      
        
        let referenceImage = imageAnchor.referenceImage        
        let planeGeometry = SCNPlane(width: referenceImage.physicalSize.width,
                                     height: referenceImage.physicalSize.height)
        let plane = SCNNode(geometry: planeGeometry)
        plane.transform = SCNMatrix4MakeRotation(-.pi/2, 1, 0, 0)
            
        let videoSceneAndPlayer = makeVideoSceneAndPlayer(with: current.videoURL)
        planeGeometry.materials.first?.diffuse.contents = videoSceneAndPlayer.0
        node.addChildNode(plane)
        
        anchorAndPlayerMap[anchor] = videoSceneAndPlayer.1
        model.updatePlayYN(of: current)
    }
    
    func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
        guard let imageAnchor = anchor as? ARImageAnchor,
              imageAnchor.isTracked == false else { return }
                        
        sceneView.session.remove(anchor: anchor)
    }
    
    private func makeVideoSceneAndPlayer(with url: URL) -> (SKScene, AVPlayer) {
        let size = CGSize(width: 500, height: 500)
        let scene = SKScene(size: size)
        scene.scaleMode = .aspectFit
        let player = AVPlayer(url: url)
        let videoSpriteNode = SKVideoNode(avPlayer: player)
        
        videoSpriteNode.position = CGPoint(x: size.width/2, y: size.height/2)
        videoSpriteNode.size = size
        videoSpriteNode.yScale = -1
        scene.addChild(videoSpriteNode)
        
        addObserver(of: player)
        player.play()
        return (scene, player)
    }
}

Replies

If you use ARImageTrackingConfiguration then the cards need to completely visible in the camera feed. If you use ARWorldTrackingConfiguration then your content is placed relative to the detected portion of the real world, meaning that your video will display in the same place even if the found image anchor is not visible in the camera feed. That would be the first thing to try.

let worldConfig = ARWorldTrackingConfiguration()
worldConfig.detectionImages = detectionImages(for: viewModel.detectionImage, width: viewModel.width)
worldConfig.isAutoFocusEnabled = true
worldConfig.maximumNumberOfTrackedImages = 4
worldConfig.isLightEstimationEnabled = true

Not all images are equally suited for using them as ARReferenceImages. Good reference images typically have high contrast, good texture, clear edges, and are printed on a flat, non-glossy surface. Low-texture and low-contrast images will likely not work so well. When you add reference images to an ARResourceGroup in an Xcode asset catalog, Xcode will show quality estimation warnings, which may provide additional insights about the specific issue with that particular image.

In some cases, you can try using an image editing tool to tweak contrast, exposure, or emphasize edges. Depending on the subject, cropping may work as well. For example, a picture showing mostly a blue sky with a small airplane could be cropped to just the region showing the airplane. If you can't get detection to work at all for a particular image, you probably need to consider replacing it.