Model I/O provides a seamless infrastructure for accessing and managing 3D models. Its high-performance architecture supports reading and writing popular mesh and model formats, textures, and materials. Get introduced to the Model I/O framework and learn key details about its integration with SceneKit, MetalKit, and GLKit.
NICK PORCINO: Hello, everybody.
Welcome to managing 3D Assets with Model I/O.
I'm Nick Porcino in the image, media, and graphics group.
Today, I'm really excited
to introduce the new Model I/O framework to you.
We are raising the bar on -- or actually we are enabling you
to raise the bar on interactive and realistic graphics.
Up until now, the graphics frameworks and hardware
that you've been dealing with come from an age
when hardware was a lot more limited than it is now.
The thing you have in your pocket is absolutely amazing.
Now we got new frameworks that enable you to get the power
out to your users, but you have to feed that framework
with really good looking stuff.
So in the new world of really high performance
and low overhead graphics APIs, you need a way to get things
that are realistic, realistic means physically derived
and physically based things.
And so there's a lot of research and knowledge that you need
in order to pull that off.
You need to understand the physics of cameras,
the physics of materials, the physics of light
and you can certainly go out and read all of that stuff
and I encourage you to do so.
And implementing each one
of those things requires a fair amount of heavy work,
a fair amount of reading and then a whole lot of integration.
So what we have done with Model I/O is we've gone out,
we have done a whole bunch of research,
we have done a whole bunch of integration,
and we've prepared a unified set of data structures
to describe these things in a consistent way
and an easy to use API.
So as it says, Model I/O is a framework
for handling 3D assets and data.
So at its most basic level, you can use Model I/O to bring
in common file formats and export them.
You can describe in a physically realistic way lighting,
You can get assets and art work from your artists
into Model I/O, do some interesting processes
which we'll get into as the talk progresses.
And there's a lot of tools now that you can get
that are focused
on physically-based rendering and materials.
And Model I/O gives you access to those things
in your own pipelines.
Model I/O is integrated into Xcode 7, and the GameKit APIs,
it's on iOS 9, and OS 10.11.
So in a nut shell, the big green box there is
where Model I/O fits in your framework
or in your application.
We start in the contact creation tool.
Your artist does some great work
and you import it into Model I/O.
Model I/O then does the annoying and tedious and error-prone step
of creating buffers for various frameworks to render quickly.
So those frameworks that we're supporting
out of the box are SceneKit, Metal, and OpenGL.
Now Model I/O doesn't just load files and save them.
It also lets you perform useful operations
that are time consuming or whatever,
that serve to improve the look of your assets,
and you can take an asset.
You can do one of these baking operations which we'll get into,
bring it back into the unified representation and Model I/O
and get it off to your hardware ready buffers.
Also, you can just complete the loop here, export your asset,
out of Model I/O after you did something exotic
that Model I/O provides for you, send it all the way back
to the content creation tool, let the artist noodle
around on it and perhaps add some, you know,
special sweetness or whatever,
and then you can just keep iterating the cycle
until you achieve the look and level of quality
that you need for your app.
So what we're going to talk
about today is what are the features?
What are the data types?
Where does the physical motivation
for those data types come from?
And we're going to talk about support
for various geometrical features and voxels
and some advanced lighting and our baking steps.
So here we go, bread and butter, reading things in.
So import formats, we start with some
of the most commonly used formats.
Alembic is a very high performance format and it comes
from the film industry
and is heavily uses in games now as well.
And it is the most modern of the formats listed up here.
It includes information on animation and material bindings
and all kinds of interesting things.
Polygon is the standard polygon format that you are going to get
out of a lot of academic research.
Triangles are commonly originated and ingested by CAD
and CAM applications
and Wavefront .obj is universally
And for exporting at the moment, you can export
out to your CAD/CAM stuff and [inaudible].
So importing is that easy.
You get an MDLAsset, by initWithURL.
And you can send it back out somewhere with exportAssetToURL.
So just a few words about physical realism
without getting super detailed.
Lights, historically have had like a position, and, you know,
a cone angle and falloff
and some other physically unrealistic parameters
that just are mathematically united to the way hardware
and graphics APIs used to be.
What we are providing here is access to IES profiles.
Now, if you go to the hardware store and find, like,
a light fixture that you really like.
Like, say I really like that light up there
and I found it in the store.
I can go to the manufacturers website after reading the label
on the side and I can find an IES profile file
which is a data file, where they have done measurements all the
way around the light to get the irradiance
from every different angle.
We read that in into a so-called light web, which is a set
of data that's pretty easy to load up into a shader.
So if you want, you can have physically motivated lights
that match real world lights in your shaders.
Now, when you buy a light at the hardware store,
the light isn't specified as an RGB value or whatever.
It's specified as a temperature like,
you know 4,000K or whatever.
So you can specify these heights in degrees Kelvin as well.
We also provide image based lighting which is in play
on the excavator there.
The materials, historically, everything has been Lambertian,
which means fall off according to angle,
and with the Blinn-Phong specular
which means a plastically shiny highlight.
We also provide you
with a baseline physical bidirectional reflection
function, or a BRDF which is what you really need
if you want real world materials.
And once again, that excavator has got physical BDRF on it.
Cameras, historically have been pinhole cameras.
We are describing them all the way from the lens to the sensor
and we are providing you some utilities
for processing environments
from photographs and procedural skies.
Now, you are going to see tools in Xcode to do baking,
and what I'm telling you here on this slide is that the tools
that you have in Xcode
to perform these operations are available to you in Model I/O
through the framework.
So if you want to make your own line --
your own offline pipeline to bake your own assets
from beginning to end, in giant batches or on a farm,
all of those things are available
through the frameworks API.
We have introduced voxels.
So you can take a big old mesh and turn it into giant cache
of indexes that you can associate your own data with.
More on that later.
Once again, it's very straightforward,
create your voxels from an asset and find voxels
in a particular region.
You can do constructive solid geometry
and you can turn the voxels back into a mesh using some type
of smoothing algorithm.
So system integration, Model I/O is integrated directly
into SceneKit and it's utilized in MetalKit and GLKit.
Model I/O is used to do preview in the Finder and in Quick Look.
So you can go find an Alembic .abc cache hit the space bar
and it'll pop up in the Finder and you can tumble your asset
and preview it without even opening up any type
of auditioning environment.
You can edit assets in the Xcode.
The excavator here is loaded in Xcode in this picture.
And Model I/O works in Playgrounds and with Swift.
Down to the nitty-gritty of data types.
So MDLAsset is the thing that you get from a URL
and it's the big overall container
that you will often be working with.
It's an index container, you know, for fast enumeration
and stuff, it has transform hierarchies in it, meshes,
cameras and lights and you can make them using the API
or loading from a URL.
So a typical asset that you might pull out of
like a big old Alembic file, is going to have a whole bunch
of interesting things in it.
This particular example here has got a couple of cameras
and a light, a root transform, the blue box
over on the right there.
And underneath it, are the bits of the excavator
and the orange boxes just indicate that, yes,
you can indicate your material assignments
and things like that as well.
So that's what's in an asset.
It's all the stuff that logically belongs together
and you will get a bunch of assets
and compose them into a scene.
So an asset has allocators in case you need
to manage memory yourself, you can add an allocator.
It has descriptors and which teach about the things
that are inside of the asset.
There's the import and export facilities
and a bunch of children.
And the children are MDLObjects.
MDLObjects themselves can comprise a hierarchy.
Now, a typical scene graph, of course,
has transformation hierarchies.
And so an MDLObject has got a transform component.
We are not implementing the transform in, you know,
the standard way of putting a matrix everywhere.
We actually have a transformation component
and components are very interesting,
because it allows us to make scene graphs
that aren't limited just to the parent of or child
of type relationships.
Instead you can define your own components.
Now, I guess I should have mentioned that the nice thing is
that the API is designed
that you can write your own importers and exporters.
You can write -- if you have your own custom file format,
you can implement that and so back to this,
I've got a custom component which is a TriggerComponent
like a volume your character enters a region and some sort
of action should occur.
You can just make that yourself and define what the behavior is
and what it connects to, the API lets you do that.
Now, a mesh contains one or more vertex buffers.
That's positions and normals the thing that has to go off
to the GPU for rasterization.
Submeshes, to get an idea of what exactly it is,
you might have, like, a character who is going
to drive the excavator
and he might have some optional components like a hard hat,
it shouldn't be optional, but it is.
So in one index buffer, I might have the whole character
without the hat, and in another index buffer,
I might just have all indexes referring
to the original mesh vertex buffers
that have got his hat in it.
So by rendering or not rendering that submesh, he does
or does not have a hat.
Submeshes can share the data in the vertex buffers
so this just allows you to have a single submission
for the hardware.
So the mesh, besides holding vertex and index buffers,
also has utility functions, and there's generators to make all
of your usual things, like boxes and spheres and what not.
So if a mesh that didn't have normals or tangent bases,
or any of those things, you can generate those on demand,
thinking back to that bake and export cycle
that I showed earlier.
And there's, of course the bakers.
Now, a mesh buffer is the thing that has to go to the hardware.
It's got the data in it.
The actual data.
How big the buffer is, how you allocated it.
And you have to describe those vertex buffers.
You have to say what the intention of the buffer is,
like is this a position?
How big is it?
How many bytes does it take and so on and so forth.
So finally the stride from one vertex to the next,
that's what the hardware needs to know.
The same sort of thing for the submesh.
You say what kind of indexes do you have.
Are they 16s or 32s or whatever.
The geometry type.
Is it a triangle strip, or is it a point or a line,
and finally a material.
Materials, as I said are going to be physically motivated
if you use these APIs.
And to tell you what that means,
we have got bidirectional reflectance functions
with ten simple parameters that are designed
by artists to be intuitive.
So one of the more important features is just
to specify whether the material is dialectic
like clay or if it's a metal.
If you set this value all the way to one end, it's dialectic
like clay all the way to the other end,
it will behave like a metal.
Here I have combined the two to put an acrylic clear coat on top
of the metallic base and here I tweaked up one of the parameters
to give a satin finish.
And here's an actual artist-prepared piece
of spaceship with all kinds of different materials on it,
just to give you an idea that a small number
of parameters can give you a wide variety of looks.
So, materials have a name, just like everything
in Model I/O has properties.
You specify the scattering function, whether you want it
to be Lambert Blinn-Phong because you need compatibility
with an older system, and, or physically plausible
if you're going into this brave new world
of physical motivation.
The materials are singly inherited and so what
that means is you might have some kind of a uniform
that only varies by a number or something on each character.
So you can specify a base material and override properties
and subclass materials.
The material properties have names, and they have a semantic
which means how are they to be used, a type,
like is it a floating point value or a color and a value.
And lights have physical parameters
and physical properties and they have geometry.
They have, you know, an extent and a width.
And the light itself emits light in a certain way.
You can specify it with lumens and color temperature.
One thing I'm really excited about overall
in Model I/O is we've got support for color
that can be specified using modern color pipelines.
So if you want to use SRGB, we have a well-specified SRGB.
If you want to use Rec.709
or the new Asus CG color profiles you can ensure
that your color started
in a content production app a certain way
and it got all the way to the end, without going
through odd transformations that might give you some,
you know, color surprises.
So there we have our physically plausible light
and various subclasses, procedure area light,
I should say it procedural description of an area light.
And the photometric light from an IES profile, and light probes
that take reflective maps or irradiance maps
and you can use them to compute spherical harmonics and stuff
like that to compactly represent what's going on.
And our camera is no longer merely a pinhole camera
with an infinite in focus projection.
We describe a camera from one end to the other.
My picture here is supposed to have a lens, shutter
and sensor plain there.
We describe what the lens can see, the properties of the lens,
what kind of distortion, barrel distortion,
or chromatic aberration that sort of thing, the geometry
of the lens and how big is the glass, how long is the barrel.
The exit aperture.
How tightly have you closed the aperture, how big is the sensor
and what is the exposure characteristics of the sensor.
So the end result of all of that is, if you specify your cameras
in this way, if you mount a 35 or say a 50-millimeter lens
with an F1.8 aperture and go check Wikipedia,
what are the characteristics of a lens like that,
what is the field of view, and what's the out of focus light,
highlight size and things like that, the utility functions
on the MDLCamera will agree with what you find in a textbook.
So that's a handy and fun thing.
And I encourage you to incorporate those kinds
of calculations into your shaders and pipelines.
Just a quick example here.
When I first load this thing up in my viewer,
these are the default exposure settings.
Some of the detail is lost, it's a bit washed out.
We can't see a lot of detail on the dark areas of the excavator.
Sorry for the brightness here, but I underexposed this image
and then flashed the sensor in order to bring
out shadow detail, and raise the overall levels while
Just like a photographer might do, if you are playing
around with settings on your own real camera.
We have two ways to create skies.
The first way is to use a procedural sky generator.
It uses physics.
You give it time of day, essentially how high is the sun.
What are the atmospheric conditions.
Is there a lot of back scatter from the ground,
is there a lot of junk in the air.
We calculate exactly what the sky would look
like through some fairly heavy math.
So the other way you can create --
it creates a cube that you can then just, you know, use.
Now, the other way you can create sky for illumination
in this physically based realm is through photography.
You can take a spherical panorama
with your phone or a DSLR.
I made this picture with my iPhone.
Then you can prepare it for rendering using the MDLTexture
and initWithURL API, create a cube map
for reflectance and irradiance.
Irradiance is the incoming light that we can deduce
from the image, so there it's been converted into a cube.
And then, from that, you can compute the irradiance
and what these three strips show is the original image,
the middle line is a texture map showing the irradiance
at a certain convolution or level of blur
and the third one is really fun.
The middle one is a texture and it uses a fair amount of memory.
And the third one is actually spherical harmonic coefficients
and so it's 27 floats that reproduce the look
of the irradiant environment
that previously took several dozen K to represent
in the middle slide or the middle strip.
So putting that all together,
this excavator is fairly well situated in its environment,
in that I photographed and that's, I think,
a pretty exciting result, and I hope you guys can think
of cool things to do with that.
Now, I just want to talk a little bit
about how Model I/O integrates with SceneKit.
There's essentially a one-to-one correspondence
between top level Model I/O elements and SceneKit elements.
MDLAsset corresponds to SceneKit root node
and MDLMesh coordinates with SCNNode with SCNGeometry.
MDLLight with SCNLight camera camera, material material.
Where exact analogs don't exist between a SceneKit parameter
and about a Model I/O parameter, we translate
to get a close approximation.
Now, Model I/O doesn't actually do rendering for you.
You probably gathered that since I mentioned all
of these other APIs that do do rendering.
So in MetalKit, you are much closer to the metal.
The correspondence that you get between Model I/O
and MetalKit is an MDLMesh can be converted
into a model kit array kit of meshes.
Once you have that array of meshes, it's up to you
to write shaders, traverse the mesh, find the materials,
the lighting condition, and do the rendering.
But getting those metal buffers prepared
for you is absolutely transparent.
So with that, I would like to pass the mic to Claudia Roberts
to talk about geometry and voxels.
CLAUDIA ROBERTS: Hello, everyone,
my name is Claudia Roberts and as Nick mentioned,
I will give you an overview of some
of the different ways you can describe
and characterize geometry in Model I/O.
The motivation being to help you create games and applications
that have a more physically plausible look and feel.
To give you all some context of where we are headed,
first I will discuss how Model I/O supports normal smoothing
then go into subdivision surfaces followed
by a discussion on voxels and a quick demo.
Okay. Let's get started.
Normal smoothing is a cool technique that tricks people
into believing that your models have way more geometry
than they actually do.
By default, the vertices
of a polygon all share the same face normal.
And thus all points on the face
of the polygon have the same normal as well.
This creates a perceived crease between adjacent polygons
which is the result of the abrupt change
and vertex normals during the rasterization process.
This sharp contrast in colors can be mitigated
by introducing a shared normal whose value is the average
of the vertex normals that share the same coordinate position.
Now, during the GPU's lighting calculations the normal
at each point on the face
of the polygon will be the interpolation of vertex normals
that are no longer all the same, giving a nice,
smooth shading effect.
Using the MDLMesh API, you can add smoothed out normals
to your object, by calling the addNormalsWithAttributeNamed
method and you can control when normal smoothing is applied
by setting the crease threshold value.
With our spaceship, we see our default mesh
with the flat shading on the left
and the smooth shading on the right.
Next, subdivision surfaces.
Surface subdivision is a common technique
for using low detailed geometry to generate
and render a smooth surface for rendering.
This technique allows you to use a simple polygon control mesh
to create varying levels of detail as needed.
For instance, it would make a lot of sense
to render a character at a low polygon count when further away
and increase the level of detail as the character gets closer
and closer to the camera.
By varying the subdivision level of a model,
you can generate these different meshes without needing
to manually create them all.
In Model I/O, you can create subdivision surfaces
by calling the newSubdividedMesh routine, also found in MDLMesh.
Here at the bottom, we achieve the smooth mesh on the right,
by setting the subdivision level to two,
significantly increasing the number of polygons.
In addition to providing support for various advanced techniques
for polygonal representations of 3D models,
Model I/O also supports volumetric representations.
By representing a model as a close approximation
of how it is actually found in the real world, that is a set
of particles or discreet points in space
with inherent properties such as volume, mass, velocity, color,
the door becomes wide open to a range
of physically realistic techniques,
analysis, and manipulations.
Where as with polygon meshes it's difficult to model
and represent surfaceless phenomena
such as clouds, water, fire.
It becomes much easier with the volume representation.
Now, instead of trying to mingle and twist a rigid shell
of polygons, the model becomes a deformable mass
that can change its properties at any time.
Along those same lines this representation allows
for procedural generation and modeling, meaning it can make
for exciting and novel opportunities and game play.
Think modification and destruction of objects
and terrain on the fly.
Because a voxel model is a more accurate representation
of the real world, it lends itself to being analyzed,
explored, and operated on in a more natural and real way
like slicing and cutting.
This fact is proven to be particularly useful
in the medical imaging field where, lucky for us,
scientists have proven than our skulls are not really comprised
of an empty shell of triangles.
And finally, given that you have a few solid voxelized models,
you can perform Constructive Solid Geometry Boolean
operations on them in order to create a slew
of more interesting and complex models.
In Model I/O, we expose the support
of voxels via the MDLVoxelArray API.
Our implementation represents volume models
as a sparse volume grid, where voxels can be accessed
by using a simple spatial index.
This representation allows for quick neighbor finding
and neighborhood traversal.
In addition to the grid coordinates each voxel contains
a shell level value which indicates how close
or far a voxel is to the surface of the model,
both in the positive exterior direction
and the negative interior direction.
And Model I/O also supports the creation
of closed meshes model cleanup
and conversion back to a polygon mesh.
I will now show you the handful of API calls you will need
in order to get started with voxel and Model I/Os.
So given an initialized MDLVoxelArray you can generate
its voxel data from an MDMMesh object
by calling the setVoxelsForMesh method.
The divisions parameter is used to set the resolution
of your voxel model by specifying the number
of layers your model will be divided
into on the vertical extent.
You can also specify how thick you want the interior
and the exterior walls of your model to be
with the last two parameters.
Once you have your voxel array objects set up,
you can perform various operations on them,
such as intersect, union, and differenceWithVoxel
which perform expected basic Boolean operations.
To actually retrieve your voxel data for processing
and inspection, simply call the getVoxelIndices method
and once your done with your processing convert your voxel
model back to a polygonal representation,
using the meshUsingAllocator routine.
Simple as that.
And now, I will show you voxels in action with a quick demo.
CLAUDIA ROBERTS: So here we have this demo.
It actually took about one hour to create from start
to finish using SceneKit editor.
We simply dragged and dropped in the ground and we did the same
for our red panda which you actually saw yesterday
at the state of the union.
It's just a regular polygon mesh and you can use any mesh
that you want for this.
And then using the MDLVoxelArray API, it only took two lines
of code to turn this mesh into voxels.
And then for each voxel, we creates an SCN box,
and this is what it looks like.
And now that we have our voxels in SceneKit, the exciting thing
about that is we can take advantage
of all the really cool things that SceneKit has to offer.
For instance, with one line of code, we can turn all
of these SCN boxes into SCN spheres.
And just for fun, we will apply SceneKit body
to all the nodes and explode him.
Wee! I am hand it over to Remi now who will talk
about advanced topics in lighting and Xcode support.
REMI PALANDRI: Hey, everyone, and thanks, Claudia.
So hi. As Chris said today, I will be talking about advances
in baking and how Model I/O does all of that for you.
So what exactly is is advanced lighting and baking.
What's the goal here?
The goal is to introduce to your frameworks
and your rendering pipelines
in your games is something called global illumination.
So what is that?
It's not the way of saying, all right I have a point height
and I have a triangle and let's light it
up using dot products and be done with it.
We are going to try to pretend that that scene is real.
We're going to try to simulate how light would actually move
in that scene how light will reflect off the walls
and occlude because it can't go through triangles.
The issue is that it's very expensive.
It's been used a long time in the movies
because you can take a half hour to reframe if you want,
but that doesn't work for us.
If you look at the picture here, you can see, for example,
that the wall here, the red wall on the left irradiates a bit
of red light on that sphere.
The wall isn't really the light per se, but lights as it does
in real life reflection off the wall and irradiated the sphere.
In the back of the sphere, it's a bit dark,
because this sphere occludes some lighting from going there.
It's not a real direct a shadow
but there is still something going on.
The issue is that this is really hard to do in realtime.
So we are going to show you ways to actually get some
of that precomputed before you even launch your game
So that you can get a really realistic rounded look
without having any of the performance drawbacks
and we will balance performance and quality
so you can get the look that you want with very,
very few performance overheads.
So global illumination today will have two different heads.
So we are going to first show you ambient occlusion
and then light maps.
And to introduce ambient occlusion,
I would like to show you an example.
If you look at the spaceship it's the default SceneKit
spaceship, it looks good.
It's a great spaceship.
I love to play the game, but it's a bit flat.
If you look at the wing or the engine, it's not extremely clear
where the wing ends or the engine starts.
If you look at the two fins on the right image,
it's weird because you have the same light as you have
on the front of the ship but the fins should block
out the lights.
If you were to add ambient occlusion it would look
Same shader, but the look is a bit more realistic,
because there is now a shadow between an occlusion
between the wing and the engine.
If I were to add ambient occlusion to the second one,
it would look like this, same thing,
but you can see light that's occluded
and it's a more compelling experience.
It's a better looking picture.
What exactly is ambient occlusion?
What ambient occlusion is is very simply a measure
of geometry occlusion.
What that means is, for my point or my mesh,
how much of the light that arrives
from my world can actually go to my point and how much
of my light is actually blocked by my mesh
and its surrounding meshes.
So it's basically a signal, one, for white,
saying I have absolutely no blocking.
The whole light arrives and/or commuting,
most of my light can't go there.
If we were to look at the signal,
that's what it looks like.
Mostly white, because most light can go there.
But you see some [inaudible] physical data there.
How do we compute that.
We compute that using offline raytracing.
So your mesh gets into a [inaudible]
and we send rays everywhere.
And we calculate how much rays
if I send rays all around my points.
How much rays hit the mesh and how much rays go to the sky box.
Difference between both is my ambient occlusion signal.
So what do we require from you guys?
An input. It's a mesh, my spaceship,
and a set of occlusion meshes.
So here it's only a spaceship.
If I were, for example, to represent
that scene right there, and I wanted to make the ground.
So it would compute ambient occlusion for the ground.
I would also need all the chairs and the people and the floors
and all of that to stop the rays
so that I have a very nicely looking mesh.
So that would be a nice set.
And what do I get as output,
a set of occlusion values just for every point.
What is the occlusion of that point.
So how exactly do we store that?
We have two ways, either vertices or textures.
So if my mesh has lots of vertices
for example a big spaceship with plenty
of triangles we can just store that in vertices.
It works very well because its very cheap it's nearly one float
per vertex and using rasterization for your metal
or pipeline, it's extremely cheap to render,
but for example, that spaceship is actually fairly low triangles
and so we need a texture.
So we actually built for you guys inside Model I/O,
a UV mapper that creates 2D texture and wraps it
around the 3D mesh, and so that corresponds.
And then for every pixel of that texture we have an
If we were to look at what that texture looks like for
that spaceship, it looks like this.
So you can see it's basically the texture wrapped
around the spaceship.
You can see the wings and the engine base and all of that.
How do we compute that?
It's very easy.
It's literally a one liner.
You can look at the top one for example,
here it's vertex.
And so we have two parameters quality and attenuation factor.
If we increase quality, what it will do, it will send more rays
to get a better looking signal but it's going
to take a bit more time to compute.
Because it's baking, it's before you even launch the game,
And if we have a textural bake, then the texture will be bigger
so it will increase a bit your memory cost.
And then that attenuation will simply attenuate the signal
so only the darker parts stay dark.
The really cool thing here is we actually partnered
with the SceneKit team to integrate those features
in both SceneKit but even in the SceneKit editor
that you guys saw at the state of the union.
And I would like to show you that right now.
So it works.
So I just loaded here a little scene
that literally has a spaceship and a sky box around.
The ship is very nicely flying in space.
I added no lights here, which is why is is very flat.
The only thing that we are visualizing right now is
And as you can see the ambient lighting does not take what the
ship looks like into account whatsoever.
It doesn't give the user nice feedback in terms
of where the geometry is, so its very flat.
We're going to change that.
So I'm going to click on my mesh and first,
I will see that we actually have a fair amount of vertexes here
so we will do a vertex bake.
I will bring up the geometry tab.
I'm going to go here and under occlusion bake
and choose the vertex,
those values are perfect, and press bake.
So what's happening here?
For every little vertex of that ship, we will send between 100
to 200 rays around and then it looks like this.
See? Way better!
We had this and now we have got this.
And it makes perfect sense.
If you look at, for example, here, the top deck,
created occlusion on the bottom one,
because the light can't arrive there easily.
If we look at the windows inside here, here the inner port
of the windows have more occlusion than the outer parts.
If we look here.
Let me zoom in.
If we look here at the cannons underneath,
the top of the cannons are really dark
because the whole ship stops the light from arriving there.
If we were to look at the bottom parts,
all white, makes sense, right?
So by just adding one float per vertex, we were able
to give our ambient light away to light up our scenes
and give a better look.
And that's available in the [inaudible] of SceneKit.
So let's go back to the slides.
So that was ambient occlusion.
I will finish today by introducing you
to advanced lighting with light maps.
So what is light maps?
What light maps are is a way to get your diffused lighting
into the game and precomputing how the diffused lighting
affects your textures to not have to do that in realtime.
As you guys know, if you have done some game programming,
lighting is extremely expensive.
If you have ten lights then for every frame you need to compute
that lights' aspect and how that interferes with your game.
That's very expensive.
So what we have here, I just dragged and dropped a plane
and two boxes inside a simulator and put eight lights there
and I computed, before I even launched the game,
using light map baker how those light maps light up my scene
and light up the texture.
And if we were to look at the scene,
that's what it looks like.
It's really realistic.
We've got the shadows and the lights
but this costs me literally one texture fetch.
Usually rendering eight lights especially if you've got shadows
and shadow maps they are very expensive.
That's what the texture looks like.
Very straightforward, you see my boxes
in the middle and my shadows.
The cool thing is this is just a texture fetch, right?
So it supports lots of lights.
I could have 100,000 lights if I wanted
and it would have the exact same runtime costs.
Even the shadows look really cool.
When you do spotlights like this, that are really close
to the ground then the shadow maps you reach kind
of precision issues but with this, for every pixel
on the thing, we send rays and see which ones are in the light
and which ones are not.
So your shadows look really realistic.
Calculate it offline.
And the cool thing is we support super complex slide
that you couldn't even dream of doing at run time.
For example our real lights have for a long time been really hard
to do at runtime because they are really hard to do
with normal point light to triangle illumination processes
but here we are using ray tracing.
So we just send the ray and see
which ones arrive [inaudible] and which ones don't.
We also support the cool IS lights,
that Nick talked to you before.
And that was light maps.
So to close this talk, I would like to summarize a bit.
Today we introduced a great new framework
and we are super happy about it.
First it does the basics imports and exports 3D asset files.
But is actually does so much more for you.
It introduces concepts for physical basis for rendering,
with models, lights, cameras, materials,
and skies that aren't just defined with floats
but are actually based on stuff in real life.
We have integration of system in the frameworks and tools
in Xcode that you guys can play with that and have fun.
For more information, we invite you to look at our documentation
and videos and forums and our technical support
and for any general inquiries, you can contact Allan Schaffer.
Related sessions are "Enhancements
to SceneKit" tomorrow and "What's New
In Metal" on Thursday.
And we would like to see you at Model I/O Lab right now
and tomorrow morning at 9AM.
Thank you all for your time and have a great WWDC.
Looking for something specific? Enter a topic above and jump straight to the good stuff.
An error occurred when submitting your query. Please check your Internet connection and try again.