SceneKit is a high-level graphics framework that helps you create 3D animated scenes and effects in your apps. See how to take advantage of the new capabilities and performance gains enabled by SceneKit's new Metal-based renderer. Check out the innovative Scene Editor, enabling you to build game levels and other scenes directly within Xcode. Learn about new features for realistic shading, enhancements for physics, and dynamic spatial audio.
So good afternoon, and welcome
to the enhancements to SceneKit session.
My name is Thomas Goossens.
If you don't already know what SceneKit is, here is an overview
of the different frameworks on iOS 10.
SceneKit and SpriteKit are the high-level APIs.
SpriteKit for 2D and SceneKit for 3D.
You can use them with other game technologies like Model I/O,
GameplayKit, and GameController, and give you access
to the low-level APIs, which are OpenGL and Metal.
SceneKit was introduced on OS X on Mountain Lion
and last year on iOS in iOS 8.
Last year, we added a few new features to SceneKit
to make it ready for casual games, like particle systems,
physics, force fields, and the integration with SpriteKit,
for example, for all the 2D game overlays.
This year, the two main additions
to SceneKit are the transition to Metal
and the new Scene Editor.
In this presentation we will start
by showing you the new Scene Editor,
then explain why we are transitioning to Metal
and what it means for you in terms of adoption.
And we will finish by presenting a few new features that we added
to the engine this year.
The new Scene Editor.
It is available in Xcode 7, so we can run it in El Capitan
or Yosemite, and it is available in the force field.
To open it, simply open a 3D file with SceneKit, for example,
DAE, OBJ, olympic, or you can create using template chooser.
In that case, it will create a new empty SceneKit document
with the extension .scn,
and an scn file is simply an scn scene instance archived
with an archiver.
That means that you can produce your own SceneKit document
programmatically if you want, and this is very handy
for the production flow.
Then the main goal of the Scene Editor is
to let you build the levels of your game.
You will see that it's really easy to combine multiple assets
from different files into a single scene
by keeping references to the original assets.
The Scene Editor also includes some visual editors for some
of the key features of SceneKit, for example,
to let you configure particle systems live in a 3D scene,
configure your physics, inspect and configure the physics shapes
of your objects, and preview a simulation for example,
sensing for force fields, and it includes an action editor
to let you build action sequences
with a graphic user interface.
Scene Editor also includes some more advanced features,
for example, if you want
to editor your shared modifiers live in your scene,
and some baking tools, for example,
to bake ambient occlusion, and for this please refer
to the Model IO session for more information about this.
Okay. But nothing better than a demo to show you this,
so let me bring Amaury on stage for presentation of Xcode 7.
AMAURY BALLIET: Hello.
My name is Amaury, and I am a software engineer
with the SceneKit team.
We dramatically improved the SceneKit editor in Xcode 7,
and with its new editing capabilities,
it's now really easy to build scenes from the ground
up in a visual and interactive manner.
Later on in the session, I will show you how we used SceneKit
in editor to build Red Panda, the puzzle game,
but right now we will start with something more simple,
a small game of physical scale, which consists
of building a tower of blocks that will eventually collapse
as we remove blocks from it.
So what you see here is a SceneKit Scene Editor showing an
On the right side, you have the usually utility area,
with its inspectors and libraries.
In the object library, you can find many SceneKit objects,
such as modes, lights, cameras, parametric geometries,
as well as physics fields.
Now, for our wooden block, we use a simple box geometric.
To customize its appearance, I can drop and imagine it.
Okay. So now just like in Interface Builder,
inspectors as well can edit all the properties
of your 3D objects.
For instance, in the attributes inspector, I can change the size
of my geometry to be 2 by 1 by 6.
And to center it in the scene, I can use the node inspector
and set its position to zero.
But actually, there is a more simple way to do that.
Using the 3D manipulator,
I can freely move the object in the scene.
Okay. So now that our wooden block is ready, we will use it
as a template to build other blocks.
Using the 3D manipulator, I can easily make copies
by holding the alt key.
I can repeat the operation, make a selection,
duplicate that selection, rotate it, make a new selection,
repeat the operation, and one last time to finish the tower.
Okay. We are now ready to use the physics engine that's built
into SceneKit to make the scene more interesting.
I will simply select all the objects,
head to the physics inspector,
and add a dynamic body to each block.
We are now ready to run the simulation.
As you can see, our blocks are now subject to gravity
and fall indefinitely.
To prevent that, we will add a floor to the scene,
so I open the objects library, put a floor, position it,
and I need a static body so that it participates
in the simulation but is not allowed to move.
If we run the simulation again, nothing happens.
But as we remove a few blocks from the tower,
we get a fair idea of what the game could look like.
One additional thing I would
like to show you is a scene review.
You can review the scene review
in the lower left corner of the editor.
The scene review is where you can see all the nodes
in your scene, and it is where you can edit the node hierarchy.
For instance, we have our floor, the many wooden boxes,
but we also have a spotlight.
I placed this light in the scene right
after I created the project.
It is currently hidden and has no effect, so let's show it.
To further improve the lighting,
and to avoid these completely dark areas,
I will simply add an ambient light to the scene.
And that's it.
Just through the editor, we built an entire scene
with 3D objects, textures, lighting, and dynamic bodies
that could make a nice game after we implement gestures.
So the SceneKit Scene Editor really allows you to build more
of your game up by writing less code.
And there's actually much more things that you can do
with Editor, and to cover that, let's switch
to the Red Panda demo, and let's see the demo again.
So Red Panda is a puzzle game in the 3D world,
where you can manipulate the camera with pan gestures
and control the character on his path.
The goal of this game is to collect these flowers
and here is the first one.
On his path, Red Panda will also have to collect items.
See how the world is rich and animated with many details?
We have lots of particle systems and animated vegetation.
Now, Red Panda can go everywhere in the level.
For instance, he can climb hills.
But on his path Red Panda will also have to avoid obstacles.
Notice the sound became louder as we pushed.
It's because we used one of SceneKit new API's 3D audio.
Many of the features are in play here.
We have the physics engine for navigation
and collision detection.
Lots of particle systems.
And shadow modifiers and actions help to bring the scene to life.
So we just collected our seventh flower,
and by moving the camera, we can see where objects hide.
We have a quick tour under the arch,
and the level is completed.
Thank you, Thomas.
So we are now going to see how we built this game using the
SceneKit Scene Editor.
So this is a starting point for our game.
A quick and easy way
to immediately improve your scene is to add a skybox.
A skybox is simply a collection of six images
that represent the environment.
After we add a skybox to the scene, we have a context
in which we can build the level.
So if we now take a look at the resources in our project,
we can see that our artist provided us
with many building blocks, and by that I mean
that we have separate files for the bamboos, for the flowers
to collect, for the pearls to collect,
as well as the many structural elements.
So now how do you use these external building blocks
in your scene?
Well, you go to the scene, and if you drag
and drop an external building block in the scene,
you can see that the reference node has been created
as indicated by this little arrow icon.
It is very nice and convenient to have these external resources
as external files because that way you can design the level
and still have a chance to save the assets,
and you can also reuse these assets
to build other levels for your game.
So as you can see, I've already included other reference nodes
for other industrial elements.
So this is how you typically build a level.
You would add your building blocks one
after the other and iterate.
You would do the same thing for the flowers to collect.
And finally, the vegetation.
So here you can see how far we went
with just a simple operation.
So as you remember, our scene is rich
and animated with many details.
We have a lot of particle systems,
but we also have animated vegetation.
Many were animated with skimming
which is a fairly Complicated animation technique that has
to be done in a special tool.
But for we use shadow modifiers.
Actually, there is one last element to animate here
in the small waterfall.
So to do that, I will open the scene for our waterfall,
and the SceneKit's editor now allows you
to add modifiers right into this scene for immediate feedback
and live compilation issues.
You can add and edit each of the modifiers
in the assistant editor.
So here I select the waterfall and choose
to add a geometry modifier.
I add the modifier, and you can immediately see its effect.
You can play with your shadow modifier and try
to tweak the final rendering.
You can try to find what the final modifiers are.
Once you are happy with your scene,
you close the assistant editor and go back to your main level.
As you can see, our mountain is now animated.
This is because we used reference nodes,
and when you edit assets,
they are automatically reflected in the scene.
One last thing we want to do here is
to add an enemy in the scene.
So this is a scene for our enemy.
As you can see, it is static.
It has no behavior.
To add the behavior, we will use actions.
The Scene Editor now allows you to add
and edit actions right in the scene.
So this is the action editor.
I will select my enemy.
Choose a row detection.
Add it. Set it to rotate along the Y axis.
And make it repeat forever.
So this is a simple behavior that we have for our enemy,
but the action editor allows you to build some complex scenarios,
and for more information about that, I invite you
to watch the What's New in SpriteKit session.
We can now go back to our level, add a reference to our enemy,
and our scene is finished.
So just -- we just saw how the editor allows you
to build an entire level from external building blocks
and how you can bring life to the scene
with shadow modifiers and actions.
And with that, I hand over to Thomas, who will talk
about the architecture of the game.
THOMAS GOOSSENS: So yeah, we are very happy
to give you this nice demo as a sample code.
It's available on the developer website.
You will see it's very simple and straightforward.
It's about 1,000 items of code for both iOS and OS X,
it includes everything like the sound, the controls,
everything you just saw.
So Amaury showed you what was done
in Xcode 7 to build this demo.
I will quickly present the rest, what was done
on the inside and the outside.
The first thing we did was to define a direction
and a simple game play for our game.
So we started with our main character.
Then we defined the shape of our level,
including where the collectible items should be
and where the enemy should be.
And we defined the final appearance we wanted
to achieve using SceneKit.
Then the next step was to scope the 3D models in the 3D tool,
to paint the different textures, configure,
and animate our character.
And the final polish was done by adding some particles,
vegetation, and fog, and we finished the game
by adding some 2D overlays on top of that.
Then the first challenge on the code side was
to manage the collisions.
For example, we don't want our character to cross the walls.
To do this with collisions, we work with collision meshes.
A collision mesh is a simplified version of a rounded mesh
that has a similar shape so that the collision feels consistent
with what you see on the screen.
It's very important to work with collision meshes
and not the rendered mesh when you deal with physics
and collisions for performance reasons.
And as for the ground, we want our character to be able
to progress everywhere in the level
but always stick on the ground.
So we need to know what is the elevation
at any point in the 3D world.
For that, there are several solutions.
What we did here was to again work with the collision meshes.
And we do array intersection, vertical array and meshes,
this I was go us an intersection point,
and this is where we can place our character.
Then whe we move the character with the touchscreen
of the keyboard, we simply move it on the X and Z plane.
Then we do a new intersection where the character is.
This gives us a new intersection point, and we use that point
to adjust the Y position of our character.
We redo that at every frame, and this is how simple it is
to animate our character in the game.
There are several animated elements in the game.
The character is animated when it is idle or working.
The grass and the bamboos are animated
to simulate wind effect.
And as a nice side effect, you can see that the shadows casted
by the bamboos are also automatically animated.
And all these animations are important
because they contribute making your game more live.
And technically, the character
and the bamboos are animated using skinning.
So the artist already built the skeleton and the animations
in the 3D tool and exported that to a DAE file,
so they are absolutely no good to do here for the developer.
And the grass was animated
with a shadow modifier directly in Xcode 7.
Related to animations, particle effects.
So there are several particle effects in the game, and again,
they were all done entirely in Xcode 7, no code here.
The pearl and the flower with some sparkles emitted.
And the enemy throws some fire.
That was done with a -- modulated with a color gradient.
The particle was done with four images of dots more
or less blurred with random sizes and an initial velocity
to simulate the wind effect.
Now, one note about our materials
and how we achieve this great-looking materials
in the game.
So here is our looks like in the game,
but if you just take its geometry and render it
with flat colors, it would look like this,
which is a little bit too simplistic.
So the first thing we did was to set a diffused texture,
and this gives us a base color of our objects.
Then we added a normal map that added very fine details
to our surface without adding many polygons.
So this is quite cheap.
Then we added a light map using the self-illumination property,
which is new in this release.
And the light map adds very fine details
about global illumination to our objects.
You can see in the corner of the stone,
you can see some occlusion, for example,
that is baked in the light map.
Then the last thing we did was a reflective cube map
to reflect an environment and, in particular, the sky.
So the reflective cube map looks like this.
We attenuate its effect using an attenuateer
to attenuate the reflection on the surfaces
that are facing the point of view.
Now, one note about the light map.
It only baked the global illumination
and not the main direct light and not the main shadows
because we want the main lighting
to continue to be fully dynamic.
That way if we want, we can animate the position of the sun
if needed, but most of all, a character can progress
in the game, and its shadows are live and automatically,
you have nothing to do here.
So that's about it for this demo,
and I hand over to Sebastien to talk
about the transition to Metal.
SEBASTIEN METROT: Thank you, Thomas.
So I am Sebastien, and I am working on SceneKit,
and I am going to talk to you about the transition to Metal
as well as a bunch of new features
that we have added this year.
So SceneKit is transitioning to Metal.
Metal, as you know, is very low-level
and low-overhead graphic API that brings better performances
and in much more modern API to the platforms,
and most importantly for you SceneKit users, it just works.
It also brings some new features to SceneKit,
like compute shaders so you can just use a Metal --
or the geometry source or Metal texture as material property.
It also allows us to do automatic batching.
So if you have the geometry used
by multiple objects using the same material,
we can dramatically reduce the number of [Inaudible].
I will show you that with a demo.
So what you see is a simple SceneKit application running
on the OpenGL ES editor.
It is displaying a planet with orbiting asteroids,
and on the OpenGL is eating up much
of the CPU just displaying the asteroids.
As you can see, we cannot add many more asteroids
without maxing CPU and reducing the frame rate.
But if we move to the same application running
on the Metal renderer, we can add many more asteroids,
as you will see in a moment.
Yes. Of course, the CPU starts to be eaten too,
but we can raise the number of asteroids dramatically.
I think it's quite awesome.
So how do you adopt Metal with your SceneKit applications?
Well, first is the default on iOS 9, so you just have
to recompile your application with the new SDK,
and it will use Metal on all the hardware that supports it.
It's also backward compatible,
so if you have an existing application
that you didn't change and didn't recompile,
it will work on OpenGL by default.
And you can opt for using the OpenGL renderer, if you want to,
for example, if you have custom shaders
that are OpenGL specific.
Choosing the renderer is really easy,
if you have a built-in application
with built-in renderer, you just have to use this small button
to change the default.
You can also do it programmatically.
So if you created your scene view at runtime,
you will give it the right option, use Metal
or OpenGL, as you want.
And what's new also is that we have added support
for Metal shaders.
So how do you deal with shaders in SceneKit with the new API?
The first way to use custom shaders with SceneKit is
to use Metal shader modifiers.
Shadow modifyers used to be only in GSL.
Now we support both languages.
If you add on GLSL modifier, it will work as it is on OpenGL,
and it will be translated to Metal.
It works in most situations.
If you write your shadow modifier
with Metal shadow language, it will not work,
it will be ignored, but it will work as Metal.
The second way to use shaders in SceneKit is to use SCNPrograms.
In this case, there is no automatic translation,
so you will do it by hand.
Will you have to translate all your code.
It's supported -- it supports both GLSL
and Metal shading languages, so you can support both
in the same application.
You can compile them offline.
You just have to provide the function names for the shader
for the fragment and vertex shader,
or you can compile it at runtime.
In this case, you just have to give the source code in addition
to the function names.
I will show you now how you do it
by creating a very small shader that would work in SceneKit.
The first operation that you will have to do is
to include our SceneKit header so that you can have access
to some semantics and, for example, the default metrics.
Here we define our custom vertex structure that is
in the correct data in SceneKit thanks
to the semantic definition.
We can also define the use of a metrics that is, for example,
the model view projection.
And we can define our own structure with our own data
for the algorithm we are trying to create.
Then we just have to create the vertex function
by giving the arguments for the structure that we have defined.
Then in the program, we just define the same structures,
we have to use the same types and the same layout
for the structure as we used in the Metal definition.
We created a program, give the vertex
and fragment function names.
And then we can give some data by using the structure
and wrapping it around time stamps.
The last way to use shaders in SceneKit is to use techniques,
techniques were introduced last year.
It's a great way to do post-processing
for all your applications on the role scene, and in this case,
you just have to give the correct function names
for the Metal shaders in your pass definition, and that's it.
It will work just like the OpenGL version.
That was it for Metal.
Let's move on to some new features
that we have added for you this year.
The first one is the integration with Model I/O,
so we have added the support for Model I/O, both for importing
and exporting of your models and scenes.
As you can see, not all the file formats support all the data
that we are supporting in SceneKit, so choose the one
that is most suited for your application.
We have also added scene transition with one line
of code, very easy, you can create transitions
between scenes, and with great animations.
Really easy to use.
We have added also debug options,
so you can visualize your boxes, your physics shapes,
everything at one time for debugging purposes.
We have also added blend modes to material
so you can choose how transparency works
for each node -- for each material in your application.
You can create new effects with that.
And we have also added an audio engine that works
for everything in SceneKit.
We can create 3D sounds, attach them to nodes,
and they will move with the nodes in the scene.
It also works with ambience and music,
so you can digital position audio and loop the sound.
The listener is placed on the camera node, but you can move it
to any node in the scene.
It's really easy.
It's based on the AVAudioEngine, so you just have to annotate
from AV audio node, and you can do your own audio processing
if you want to if you have an existing audio engine.
And last, we have added also a way to created sound action
so you can integrate sound into your action sequences.
That's really easy to do.
And here is how you use the sound API.
First you have to instantiate an audio sound from the file fame.
Then you create a player and attach the player to any node
in the scene, and it will start to play right away and stop
when the sound is finished.
If you want to do ambience or music,
you have to just flip a switch,
like digital positional audio and start looping.
And last, it's really easy to create a play song selection.
It's just one line, and you can run the action or add it
to an existing action sequence.
This API was used in the sample code for all the sound,
and we used the collision mesh as --
that Thomas showed you sooner to detect kind
of ground the character is working on.
For example [scratching sound] when he walks on the grass.
Or when he walks on a rock.
It was really easy to do.
So that's about it.
And I give you back to Thomas for the wrap-up of this session.
THOMAS GOOSSENS: Okay.
So that was an overview of what we added to SceneKit this year.
So a great new Scene Editor that will save you tons of lines
of code and do more for your game insight tools,
which is usually a good idea.
A smooth transition to Metal
which should boost the performance of your game
with nothing to do on your side most of the time.
And some new features that were presented, and we covered some
of them in this session,
but check the documentation for a list.
And an amazing demo app that we are really proud
to share the code with you.
Search for fox on the developer website, and you should find it.
Now for more information, please check the resources
and documentation we have online,
and for general inquiries,
contact our Evangelist, Allan Schaffer.
Some related sessions, Model I/O,
great new frameworks introduced this year,
and check the What's New in SpriteKit session.
They have a really great demo of the scene editors
that go much deeper into details.
So you definitely want to see that one.
And that's about it.
Thanks, and enjoy the conference.
Looking for something specific? Enter a topic above and jump straight to the good stuff.
An error occurred when submitting your query. Please check your Internet connection and try again.