Now available for iOS and OS X, SceneKit is an inventive high-level graphics framework enabling your apps to efficiently load, manipulate, and render 3D scenes. Get reintroduced to SceneKit and check out how your apps can take advantage of its latest capabilities. Learn about advanced capabilities for physics and animation, support for subdivision surfaces, new rendering effects, integration with SpriteKit, and much more.
Thank you. So, good morning, everyone, and welcome to the "What's New in SceneKit" session.
So, SceneKit is a framework to help you load and integrate 3D objects or 3D scenes in your application.
It can be used for different purposes, like UI, data visualization, or presentations, like here, and now with this release also to build casual 3D games. It was first introduced on the Mac on 10.8, and it is now available on iOS.
Yeah, thank you.
Regarding the layering, SceneKit is a high-level Objective-C API.
It's built on top of OpenGL and OpenGL ES, and it can collaborate well with those aerographic technologies, like Core Image and Core Animation, and also, especially now, with SpriteKit, and I will detail that in the next slide. In addition of being available in iOS, which is really great, we also have a set of cool new features ready for this release, like physics, particles in physics fields. I will present them in the next slides.
Before that, some related sessions.
So, I invite you first to have a look at last year's presentation, since even if SceneKit was OS X only, everything represented last year is applicable now on iOS.
Then this session is about "What's New in SceneKit." We also have a more hands-on session at 11:30, same room, where we will show how we build a little but fully functional game, and we'll explain how we did that.
Also, in case you missed it, have a look at the SpriteKit session that was earlier this week, since SceneKit and SpriteKit can really work well together to achieve great results. Okay, for this presentation, I will start with a very brief overview of the SceneKit main principles.
It will be very quick. If you want more details, please refer to last year's presentation. And then I will present the steps to create an application that uses SceneKit, and I will present the new features as I go along.
So, the overview.
So, a scene in SceneKit is represented by the SCNScene class. If you are already familiar with SpriteKit, that's the equivalent of SKScene.
So that's object, and the scene has a root node of the SCNNode class, and a node represents a location in 3D space.
A node may have some child nodes that are related to their parent node, just like SKNode are related to their parent node. And then, a node by itself doesn't render anything onscreen. It's just a location in 3D space. And unlike SpriteKit, where you use subclasses to render things onscreen, here with SceneKit, you attach attributes to your nodes.
So attributes you can attach as the following: You can attach a light or a camera or geometry.
And attaching a geometry to a node means that you want to render that geometry at the node location.
So, if you want to render the same geometry in multiple places, you simply share the same geometry instance to multiple nodes.
So, let's go quickly over these attributes. So, the first one is SCNGeometry. It represents a geometry, so a set of triangles, vertices, no more than textural coordinates, and it has also a of materials that controls the appearance of the surface.
Then, the second attribute is SCNLight. It represents a light source. We support four types of lights: to illuminate from a point, in a direction, as a spot directions, and with the ambient light.
Then, next attribute, SCNCamera.
A node with a camera attached can be used as a point of view to render your scene, so it's simple as setting the pointOfView property of your view to one node that has a camera attached. For example, here is a point of view to render the scene, and here is another point of view.
So, to sum up, when you create a scene with SceneKit, you first start by creating nodes and placing them in 3D space.
Then you can add a child node that are relative to their parent node.
Then you attach geometries to these nodes to render surfaces onscreen.
Then you configure the materials to change their colors and textures. And finally, you can also attach a light to one node if you want to illuminate your scene. So that's our very brief introduction. Now let's see how to get started.
So, in Xcode 6, there is now a new game template from which you can choose to create a SceneKit base game, and this will create a new application with the SCNView already set up for you. If you already have an application, you can also use the Interface Builder and drag and drop SCNView from the Interface Builder Library into your application. Note that SceneKit gives you a view if you want to render into a view, but there is also SCNLayer on OS X, and SCNRenderer if you want to render off-screen into an arbitrary OpenGL context. Now you know how to render a scene; let's see how to create one.
So, to create a scene, you have basically two options: You can user-create everything programmatically, or you can load a scene from a file.
To create a scene programmatically, you can use one of our built-in parametric geometries, like a box, a plane, a torus, a cylinder, etcetera.
You can configure the parameters, like width, length, height, , segment count, etcetera.
You also have SCNText to create 3D text with extrusion and chamfer and multiple materials, and it supports basically all the fonts and layouts supported by Core Text.
You can also use SCNShape to create a 3D object from a 2D Bezier pass by extruding it and chamfering it. So, for example, here is a 2D Bezier pass, and I can extrude it to create a 3D object from it very easily. And also, if you need, you can also create your fully custom geometry by placing your custom vertices, normal and texture coordinates. So here you have the full control and with SCNGeometry API. And also new in this release, you can also now subdivide your geometry with the subdivisionLevel property of SCNGeometry.
This uses Pixar OpenSubdiv technology to achieve this. So, for example, here on the left you have the model not subdivided yet, and on the right with one iteration of subdivision. Okay, so that to create scenes programmatically. Now, you can also load a scene from a file.
And loading a scene from a file lets you load a scene fully configured by your artist that will include all the geometries, all the nodal hierarchies, the lightings, the textures, the position of your cameras, all the skin information, etcetera.
SceneKit supports COLLADA documents with the extension .dae. It's an XML-based file format.
We now also support the Alembic file format with the extension .abc.
So, Alembic is quite popular in the video industry, for example.
And with Alembic you can load geometries and animations exclusively.
So, once you get your 3D file from your artist, you can have a preview of it in Preview, or directly in Finder with QuickLook, and you can open it and have a preview and do more with Xcode.
Xcode has a built-in COLLADA editor that will let you inspect your 3D files.
To be clear, it's not a modeling tool; this is not where you will create your models and create your scene. This is more a tool to inspect the hierarchy of nodes, their names, you can change the materials, adjust the lighting, and do the scan of things, typically. You can tune your scene for your application with Xcode.
Once you have your scene ready to load it at one time, it's easy. You first add your COLLADA file to your resource as a normal resource, and then, at runtime, you use SCNScene sceneNamed to load it, or Scene with URL. New in this release, we also provide an alternative way to manage your assets, and we call that SceneKit Assets Catalogs.
An Asset Catalog is simply a folder with a .scnassets extension, and what it does is that, if you put your assets in it, Xcode will copy this folder at build time by preserving your folder hierarchy.
So, the first advantage is that it lets you organize your assets the way you want -- for example by having your models in one folder, your textures in another one, your particles in yet another one -- or organize your assets by level if you are building a game.
The second advantage is that Assets Catalogs will give you some options because the Assets Catalogs are processed at build time, and so you have options to optimize your assets at build time by, for example, interleaving the geometry data to be more efficient on iOS, or by converting the orientations of your models, too, so that they are all consistent across your game.
Okay, once you have your scene loaded to display it, it's easy. Simply assign your scene to your view with the scene property. The view will automatically display the scene, and also, if you modify your scene graph, the view will automatically reflect your changes. There is no need to call display, for example.
Okay, so now we have seen how to load and display your scene; let's see how to give life to those scenes.
To animate your objects, you have several options. You can update the position of your objects programmatically at every frame.
You can configure animations for a given duration and let it play.
New in this release, we introduce the concept of actions. I will present that. You can also set up some constraints between your 3D objects. And last I will present how you can animate your scenes using physics.
So first, the per-frame update. If you need to update your objects on a per-frame basis, it is recommended to use one of the delegate methods here. So here is how the game loops look like; so, this is what is done at every frame when your game runs.
So it's the combination of delegate method that we call, and you implement, and some SceneKit internal processes, like our rendering typically.
So basically, you will implement the update method to implement your game logic, and you can here move your objects programmatically. It is recommended to use these callbacks essentially for performance reasons, because when these callbacks are called, the scene is already loaded for you to modify it.
If you do it in your own thread, your changes will be committed on the next transaction and it won't be necessarily synchronized with the current frame. Note that you are also have some callbacks to be notified when the animations did apply and when the physics did simulate, if you need to do whatever actions after these operations.
So, that's for per-frame updates.
Now, another way is to configure actions - animation, sorry.
So, for animations, SceneKit uses the same programming model as Core Animation, which supports implicit and explicit animations, and almost all the properties of the scene graph are animatable.
So, for implicit animations, it works just like Core Animations: You first start a transaction with a given duration, then you modify whatever property you want on the scene graph.
And last, you commit your changes and it animates implicitly. For explicit animations, here we use directly the APIs of Core Animation. We support CABasicAnimation, CAKeyFrameAnimation, and CAAnimationGroup. For example, here I set up a simple basic animation that targets the rotation of my node, then I configure its duration, its destination value, and set it to repeat forever. And finally, I add my animation with addAnimation forKey - so, just like Core Animation. And it makes my nodes to rotate forever like this.
Okay, don't worry.
So, note that we also have some extensions, some additions in Core Animation animations. The first one is animation events. Animation events lets you trigger a custom block of code at some specific progress of you animation.
A typical example, for example, is to play a sound at some progress in the animation. For example, here, I'm playing a sound at 60 percent of my animation, so if I press Next, you can see it calls my block at 60 percent and plays that sound.
So that's the first addition, and the second addition is smooth transitions.
So this is useful to blend one animation onto another. So, for example, here, the monster is playing an idle animation.
If I press Next, I will start an attack animation on top of it. By default, without any fade-in and fade-out duration, it looks like this. Okay, I will do it one more time; look carefully at the end of the attack animation.
Okay, so this is not super smooth, not really great.
And if I simply configure the fadeInDuration and fadeOutDuration of the attack animation, it will make the effect of the attack animation to smoothly increase at the beginning and smoothly decrease at the end. So, now look at the same thing.
Okay, one more time. So now it's smooth, so it's much better. Yeah, thank you.
So that's much better, and also that's super easy to configure.
So that's for animations.
So, if you're already familiar with SpriteKit, you already know SKAction. Here, SCNAnimations just works the same. And the main advantage over animations is that it makes really easy to sequence group and repeat simple actions.
However, it's limited to a set of predefined actions that works on SCNNode. With animations, you can really target whatever property of the scene graph.
So to give you an example of how easy it is to write an action, here is an action that makes my node to rotate forever.
So with just one line of code, it's the equivalent of the previous slide with Core Animation with six lines of code. So that's much simpler.
Here are some of the predefined actions. You can move your objects, rotate, make them fade in, fade out, remove from the scene graph. And note that you can also implement your custom actions by providing your custom block if you want. One difference between animations and actions is that actions directly target the presentation tree.
If you are familiar with Core Animation, you know already the concept of the model tree versus the presentation tree. So, model tree contains the value you set and get programmatically, and the presentation tree contains the in-flight value of any running animation.
So, the red dot here represents node.position. You can see that, with actions, node.position corresponds to the position of the cube on the screen.
With animations, node.position corresponds to the position of the cube at the end of the animation.
If you want to know the position of the cube onscreen with animations, you have to do node.presentationNode.position, just like Core Animation.
Okay, so that's about actions. Now, physics.
SceneKit now has a built-in physics engine, zeroprode . So, the API is very similar to SpriteKit. To make a node physicalize, you set the physics body property of your node to a SCNPhysicsBody instance.
For example, here I have a cube that is not physicalized yet, and by setting the physicsBody property to a dynamic body, it will automatically animate with the physics simulation. Now, dynamic bodies automatically bounce and collide together based on their physicsBody properties, it's automatic.
And then if you want to manipulate those objects, you should not set the position programmatically directly. Instead, you have to use forces to manipulate those objects. SCNPhysicsBody has some methods for you to apply forces and angular forces to your objects. So for example here, if I apply a force from the center of the scene to all these cubes, I will make them go away. Okay.
So, that's dynamic bodies. We also have static bodies. Static bodies participate to the physics simulation, but they never move; they are static.
So for example here, these boxes here are set to a static body, and if I have some dynamic spheres, you can see that they collide together but that the static objects don't move.
My son loves that, too.
One last kind of physics body: kinematics body. Kinematics bodies participate to the physics simulation.
They don't move in response to collision. However, the difference here is that you can move kinematics bodies programmatically by setting their position or rotation. For example here, I have a rotating box that is set as a kinematic body. I can make it to rotate with an action, for example. And you can see that it automatically collides with dynamic spheres here and makes them bounce.
Okay, so that's for physics bodies.
Now, one note about PhysicsShape. The PhysicsShape is a geometry that is used by the physics engine to perform the physics simulation. By default, SceneKit will automatically create a shape for you. So for example, if I had some teapots, let's say, SceneKit will automatically build a shape for these teapots by building the convex hull for you.
That said, you can still customize the physics shape if you want. The main reason can be for performance.
If you use a built-in primitive for the shape, it will be more efficient than the convex hull. And the second reason can be for better correctness.
Let's say you want to have an object that rolls on the floor. You will have better correctness if you use a sphere or cylinder than using a discretized geometry.
So, here are the primitives that are natively supported by the physics engine.
So spheres as usual, spheres and boxes, etcetera.
And, okay, next, PhysicsBehavior. The PhysicsBehavior lets you customize the physics simulation. For example, we have a set of joints available in the APIs for you to connect objects together, for example.
For instance here, these boxes are connected by hinge joints.
To add a joint, simply use addBehavior on the physicsWorld of your scene, and the joint will reference the two nodes it has to connect.
To remove the behavior, simply use removeBehavior, and it will remove the joints.
Okay, so there are definitely much more things I didn't cover about physics, like PhysicsFields for example, but definitely a lot of cool things to play with.
Now I will just show you a quick demo, still about physics.
So, here is a little sample code available on the developer website that is to illustrate the physics in general and vehicles in particular. Vehicles have a specific behavior.
When I touch the screen, I will apply a force on the wheels, and so we'll be able to start to drive.
Then I can control the steering of my vehicle with the accelerometer and also with the game controller.
And the first thing you can see, well, the first thing you can see is that I'm not a good driver, but the second thing you can see is that vehicles automatically collide with dynamic bodies and make them bounce.
Second thing, it also works with static bodies. So, for example, if I hit the wall, the walls are static, ah, oops, missed, woo, like that.
And - okay, then you can also notice the train here is made of several objects that are connected with joints, so if I hit the train you can see that the objects move like that.
Also notice the smoke effect, and we explain how we do that later in the slides.
Now, we'll try to do one gentle. Let me restart the game like that and try to jump.
Woo! Okay, quite good. Thank you, thank you.
Okay, and also note that this game has some little overlays on top of it. They are made with SpriteKit; we will explain how we do that later.
So I'm talking about the speed gauge on the right and the camera button on the left. And so they are rendered by SpriteKit, but they are also fully functional. So, for example, if I click the camera button here, it is clickable, and I can change the point of view now to drive like that. Ah! Okay, and you can see that the transition between the two points of view is smooth because it's just done between using an implicit transaction, and so that's really super easy to do. So this sample code is available. It is very small - sorry - it's about 600 lines of code, so very easy to do.
Okay, so still related to animations and interactivity in general, now constraints.
You can set an array of constraints to a node. These constraints will be applied sequentially, and they will be applied at render time. So, that means that the constraints will never modify your model tree, so models that you set and get programmatically.
It will be applied at the very last moment of the rendering.
We support custom constraints where you can program your block and apply whatever transform you want on the node's transform.
We also support some predefined constraints, like the LookAtConstraint.
So, LookAtConstraint forces a node to look at another node. For example here, if I set LookAtConstraint to these arrows, it will make my arrows to look into the direction of the sphere here, and note that I'm doing it inside an implicit transaction to make a smooth transition. And since it is done at render time, I can just move my sphere and you can see that the constraints automatically update.
Then a typical use case is to use this constraint on the node that owns the camera. For example, now to have my camera to follow that node. And since everything are nodes in SceneKit, you can also set this constraint on the node that owns the spotlight, and now you can see that I have my spotlight that can follow this node as well.
New in this release, we are adding one more constraint that we named SCNIKConstraint. IK stands for inverse kinematics.
And this one is hard to explain by a French guy, but I'll try. IK constraints apply on a node chain. The node chain is specified as a base node and a root node, and the two nodes must be in the same hierarchy.
The IK constraints will force your node to keep the same original related distance between each other.
So to give an example here, I have a character that is playing an idle animation, and when I press Next, it does an attack animation like this.
So IK are often used to control articulated things like the arm of robots or characters.
Now, let's say I want to do the same attack animation, but this time I want to hit this target.
By default, it will just miss the target. Now, I can use an IKConstraint to move the lift of my node chain in the direction of that target, and the IKConstraint will automatically make sure that the hand and the elbow and the shoulder stay consistent. And I can also modulate the effect of the IKConstraints over time by modifying the influence factor of the IKConstraint to make its effect increase during the animation. So for example here, by combining the attack animation and the IKConstraint, I can do something like this. Hit the target.
Note that here I'm also using a LookAtConstraint for the head of the character to make it to look in the direction of the target.
So, if I move the target down, you can see that the head of the character follows the target. I can still use the IKConstraints to hit the target like this. I will do it one last time because it's fun.
And beam! Okay.
Okay? So that's all for animations, and so there are definitely a lot of things I did not cover yet, like the skinning, for example, the morphing as well. For this, please refer to last year's presentation.
Now, I hand it over to Aymeric for the rest of the presentation. Thank you.
Good morning, everyone. My name is Aymeric, and I'm going to talk about rendering.
Thomas already showed you how to load a scene or create it from scratch.
We now see how you can make it visually nice using material. While geometry defines the shape of a node, material defines its appearance. That means its color, its transparency, and interactions with light like diffuse and specular reflections.
SceneKit uses the SCNMaterial class to represent material.
You can imagine it as a toolbox full of properties controlling an underlying optimized shadow.
Here is an example of a nicely configured material.
A satellite image is used as a diffuse map while the oceans are masked as the only specular areas.
A normal map is used to simulate a mountain, while an emission map displays the city lights shining even in the dark.
Finally, a cloud layer has been added using a transparent map on a separate object.
All these properties uses of SCNMaterialsProperty class.
Let's see how we can fill our contents. First, we can set a plain color using an NSColor or CGColorRef.
Then, of course, we can set an image. The easiest way to set an image is using an image pass, either using an NS string or an NS URL.
This is the most optimized way.
Simply save an extra copy of the image in the system memory. Of course, you can set image using NSImage or UIImage, but new in this release we also support SKTexture.
This will allow you to use the cool features of SpriteKit, like atlases, procedural node generation, and automatic normal map generation.
For dynamic contents, we still support CALayer and, new in this release, SpriteKit Scene.
This video is played using a SKVideo node.
For the reflective property, we cube maps.
Cube maps are useful to represent a complete environment.
To set a cube map on the property, simply set an array of six separate images.
If you want to customize the standard materials of SceneKit, you may want to use shader modifiers.
Modifiers are little snippets of GLSL code that can be injected at specific stages.
You can apply modifiers directly into materials or onto a geometry.
In this case, every material used by the geometry will be modified.
Here are some examples of shader modifiers.
The first, Ripple Effect, is working at the geometry stage.
The Liquid Noise Effect's working at the surface stage, while the Rim Lighting Effect is obviously at the lighting stage. Finally, we fake a mode using a fragment modifier.
For more information, for more details about shader modifiers, please refer to last year's session. If you want to completely replace a specific material, you can do so by using SCNProgram.
These are GLSL programs like Vertex and Fragment shaders, and new this year, but only on OS X, geometry and shaders are supported too. This will allow you to take the full control of the rendering.
SceneKit only provides you geometry attributes like position, normal, and texture coordinates, and transform uniforms like model view and projection matrices.
For even more control, you can inject OpenGL code directly before or after the scene rendering.
This allows effects like this vortex tunnel that was done using a full screen quad on a custom fragment program.
You can also inject OpenGL code on a specific node basis, allowing you to inject code during scene rendering among those objects.
So, new in this release also, we support overlaying a SpriteKit scene directly onto the SceneKit scene. This is how we did the speedometer in the toy car demo Thomas showed you earlier.
This is the preferred way to all your UI needs, because you only will have to write once your event and link code in SpriteKit, and it will work on iOS, on OS X, and it also quite performant because SpriteKit nodes are directly rendered in our buffer, saving on extra compositing.
So, that's all for rendering and rendering customization.
Let's now browse the effects available in SceneKit to dress up your scenes, starting with particles.
Particle systems are commonly used in movies and video games to simulate a large class of effects, like fire, rain, smoke and explosions.
In this release, we introduce a new SCNParticleSystem class that will allow you all of these effects and more.
Because particles have a large number of properties, we also provide you with this new release of Xcode, a new 3D particle editor that will allow you to tweak almost all properties of ParticleSystem.
So, let's see what kind of effect we can achieve with it.
Here is a hot fire effect that is done with a flame texture set as a particle image.
We animate the color and the size of the particle over the lifespan.
We also use an additive blending, but we could also have used a screen blending among other blend modes.
ParticleSystem can be local, like the one in the background moving as a or global like the one in the front, where particles are directly emitted in workspace.
Particles can be affected by gravity.
They can also collide with nodes of any shapes.
Just beware of your , though.
Particles can be affected by physics fields, like this turbulence field or this vortex field.
ParticleSystem can also generate subsystems on specific events, like particle bursts, deaths or collision.
Here, each raindrop generates a subsplash system when hitting the floor. You can even apply custom block on this particle event or on each simulation step.
Here, the confetti are laid down and stand still when they touch the floor.
ParticleSystem can be emitted from inside or from the surface of any shape.
This shape I represented using standard geometries.
Please note that we recommend that you use parametric geometries as they will provide the best performances and a better spatial distribution.
Here are some kinds of parametric geometries.
As I've said earlier, it is the recommended way to create ParticleSystem is through the 3D Particle Editor built into Xcode.
Starting from the provided templates, you will be able to tweak and customize your system.
Then, you will save them as .scnp files that you'll be able to later load in your app.
One of the most important visual cues regarding depths and contact is shadows, and SceneKit provides several ways to do shadows.
The first way is to directly bake your shadow map into your altering tool.
Then, you will use them on the Multiply properties of your receiving materials.
While this provides the best-looking shadows, and they are quite efficient as well, the effect will break if you try to move your shadow casters or even the light.
But SceneKit also supports dynamic shadows.
All you have to do is enable castsShadow on the spotlight or, new this release, a directional light. Then your receivers, caster, and even the light will be able to move as the shadow map will be generated each frame. If you want to move your shadow casters but have a limited GPU budget, you can also use projected shadows, where a simple texture will fake soft shadow when set on a gobo with a ShadowModeModulated. The next session will cover this in more detail.
New in this release, we also added a global fog on the scene. Just set a fog color, the range you want it to operate, and you're done.
You can even control or animate the density of the fog.
Another commonly used effect in games is the use of depth of field.
Here again, with SceneKit, it's really easy to do. All you have to do is tweak a few focal properties on the camera, and you can achieve effects like out of focus in the background or in the foreground. It's that simple.
The integration of Core Image also allows you to use several screen-space fancy effects.
Applying on a single node or hierarchy, you can do effects like Gaussian blurs, distortion like this Pixelate Effect, or any kind of color processing you want.
The last big new thing in this release that I wanted to talk about is the addition of multipass technique. Shadows, Core Image filter, depth of field - all of these are internally built using techniques. And now, with this release, we offer you to create your own. So, what is a technique? A technique is a sequence of passes. Each pass can render the scene, a specific object or full-screen quad, using custom vertex and fragment programs.
Techniques have outputs, like a color or depth render target. They usually have inputs that can be constant textures or other passes render target.
They can also modify states like blend states.
So, this passes from a graph where render targets are produced and consumed.
Techniques were designed to be data-driven.
They can be entirely configured through a property list referencing external vertex and fragment shader.
This setup allows you quick iterations and tweaking your effects without rebuilding your application. Let's see how to create a technique.
The easiest way is to load it from a dictionary, usually coming from a property list, but you can also create technique by chaining several techniques together.
When you have a technique, simply set it to a view, and you're done.
Here's an example of a simple depth of field done using techniques.
The first pass here renders the scene normally, writing into a color on a depth buffer. Then a second pass takes the color buffer from the first one and generates a blow-up version of it. Finally, the third pass combines the normal color buffer and the blowed one using the depth buffer as a blending factor.
So, that's all for techniques. So let's sum up what you should take away from this session.
SceneKit now is available on iOS. It is ready for casual games thanks to physics, actions and animations. It's also of rendering possibilities using standard materials and effects like particles, and it has a lot of control for you to customize its rendering with things like shader modifiers and techniques.
One last thing is, as you may have guessed, this session was created entirely in SceneKit, and we are happy to give it to you as a sample code.
So, I strongly encourage you to download it and give it a try. It's almost as fun as Swift.
For more information, please contact our Evangelists Allan and Filip. We have a brand new documentation for you in the developer website and also a dedicated forum.
Looking for something specific? Enter a topic above and jump straight to the good stuff.
An error occurred when submitting your query. Please check your Internet connection and try again.