Framework

ARKit

Integrate iOS device camera and motion features to produce augmented reality experiences in your app or game.

Overview

Augmented reality (AR) describes user experiences that add 2D or 3D elements to the live view from a device's camera in a way that makes those elements appear to inhabit the real world. ARKit combines device motion tracking, camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience.

Topics

First Steps

About Augmented Reality and ARKit

Discover supporting concepts, features, and best practices for building great AR experiences.

class ARSession

A shared object that manages the device camera and motion processing needed for augmented reality experiences.

class ARConfiguration

The abstract base class for AR session configurations.

World Tracking

Create AR experiences that allow a user to explore virtual content in the world around them with a device's back-facing camera.

Building Your First AR Experience

Create an app that runs an AR session and uses plane detection to place 3D content using SceneKit.

Handling 3D Interaction and UI Controls in Augmented Reality

Follow best practices for visual feedback, gesture interactions, and realistic rendering in AR experiences.

class ARWorldTrackingConfiguration

A configuration that uses the back-facing camera, tracks a device's orientation and position, and detects real-world flat surfaces.

class AROrientationTrackingConfiguration

A configuration that uses the back-facing camera and tracks only a device's orientation.

class ARPlaneAnchor

Information about the position and orientation of a real-world flat surface detected in a world-tracking AR session.

Face Tracking

Use the TrueDepth camera on iPhone X to create AR experiences that respond to the user's face and facial expressions.

Creating Face-Based AR Experiences

Use the information provided by a face tracking AR session to place and animate 3D content.

class ARFaceTrackingConfiguration

A configuration that tracks the movement and expressions of the user’s face with the TrueDepth camera.

class ARFaceAnchor

Information about the pose, topology, and expression of a face detected in a face-tracking AR session.

Standard Views

class ARSCNView

A view for displaying AR experiences that augment the camera view with 3D SceneKit content.

class ARSKView

A view for displaying AR experiences that augment the camera view with 2D SpriteKit content.

Custom Views

Displaying an AR Experience with Metal

Build a custom AR view by rendering camera images and using position-tracking information to display overlay content.

Image Detection

Recognizing Images in an AR Experience

Detect known 2D images in the user’s environment, and use their positions to place AR content.

class ARReferenceImage

An image to be recognized in the real-world environment during a world-tracking AR session.

class ARImageAnchor

Information about the position and orientation of an image detected in a world-tracking AR session.

Real-World Objects and Positions

class ARHitTestResult

Information about a real-world surface found by examining a point in the device camera view of an AR session.

class ARAnchor

A real-world position and orientation that can be used for placing objects in an AR scene.

protocol ARTrackable

A real-world object in a scene for which ARKit tracks changes to position and orientation.

Camera and Scene Details

class ARFrame

A video image, with position-tracking information, captured as part of an AR session.

class ARCamera

Information about the camera position and imaging characteristics for a captured video frame in an AR session.

class ARLightEstimate

Estimated scene lighting information associated with a captured video frame in an AR session.

class ARDirectionalLightEstimate

Estimated environmental lighting information associated with a captured video frame in a face-tracking AR session.

Advanced Topics

Using Vision in Real Time with ARKit

Manage Vision resources for efficient execution of a Core ML image classifier, and use SpriteKit to display image classifier output in AR.

Creating an Immersive AR Experience with Audio

Use sound effects and environmental sound layers to create an engaging AR experience.