스트리밍은 대부분의 브라우저와
Developer 앱에서 사용할 수 있습니다.
-
PencilKit 그림 검사, 수정 및 구성하기
앱에서 Apple Pencil을 그리기 및 글쓰기를 위한 유용한 도구를 활용하도록 지원하세요. PencilKit을 사용하면 그림을 구성하는 획, 잉크, 경로 및 점을 자세히 살펴보고, 이러한 특성으로 인식을 사용하는 기능을 구축하고, 입력에 반응하여 그림을 수정할 수 있습니다. 도형과 그림을 동적으로 생성하는 방법을 익히고 PKDrawings 및 PKStrokes와 같은 API에 대해서도 자세히 알아볼 수 있습니다. 이 세션을 최대한 활용하려면 먼저 WWDC19에서 처음으로 PencilKit 프레임워크를 소개한 세션인 ‘PencilKit 소개'와 WWDC20의 ‘PencilKit의 새로운 기능'을 시청하시는 것이 좋습니다.
리소스
관련 비디오
WWDC20
WWDC19
-
다운로드
Hello and welcome to WWDC.
Hi, I'm Will Thimbleby. We're going to take a look inside PencilKit drawings, at what they're made out of and what you can do with them. PencilKit is super easy to adopt, provides beautiful, realistic-looking inks, the best low-latency drawing experience and some great new UI improvements in iOS 14. For more information about these improvements, see the "What's New in PencilKit" talk. In iOS 14, we're also letting you look inside PencilKit's data model: the drawings, the strokes, inks, paths and points. This will enable you to build some great new features in your apps with PencilKit. With access to the data model, you'll be able to inspect the contents of what your users drew, react to what was drawn, manipulate existing drawings or dynamically create new drawings from scratch. To give an example of what's newly possible with PencilKit, let me jump straight into a demo.
This demo is also available as sample code. I've been working on an app to help my seven-year-old son practice his handwriting. At the top of this app is a text field with the words I want him to practice. Below is a synthesized PencilKit drawing of the same text. I could change this text with the keyboard, but this is a Pencil demo. I can use the great new Scribble feature, which, by the way, also uses PencilKit, to change this text to what I want him to practice, perhaps to something more appropriate for a seven-year-old.
As I enter text, the template PencilKit drawing below is constructed from individual letters.
In the top right, I can also choose the size I want him to practice and the difficulty.
Now that I have a template, we're set up to practice handwriting. The next stroke to copy is animated by this red dot that shows how to write the next letter. All I have to do is copy it.
If I write something that's close to the template, my handwriting changes to green and we move on. If I write something badly, the stroke disappears and the animation repeats.
My score is shown in the top right. Let's see how I do.
We did well, so we get some fireworks, a key motivational tool for seven-year-olds. But I've had a few years to practice my handwriting, so why don't we try something a bit trickier? I drew a PencilKit ligature, and I've added that to the app.
Now we can practice our calligraphy.
The app is synthesizing PencilKit drawings from text, animating them and performing recognition on what I wrote. This is just a small example of what you can now do with access to the PencilKit data model. So let's look inside a PencilKit drawing. Here we have a simple drawing of a flower. If we split it up, we can see that the drawing is composed out of many PencilKit strokes. Each stroke represents an individual line that the user drew. These strokes are in the order that the user drew them, so you can see that the outline of the flower was drawn first, then the stalk, leaf, and finally the whole thing was colored in with a marker.
Here we have the data asset that I drew that contains the lowercase letters for the demo I just gave. To be able to generate text, we first want to split this drawing up into individual letters.
Later, these letters are combined to generate the template text that users use to practice. To split this drawing up, we take the lowercase alphabet drawing and get its strokes. We slice this array to get the strokes for each individual letter.
Then we can create new drawings for each letter out of that slice. This is repeated for each letter in the alphabet.
If you want to inspect or modify a drawing, you do this by accessing the drawing's array of strokes. You can also use strokes to create new drawings from scratch.
So, what about strokes? What makes up a PencilKit stroke? For a stroke, the primary feature is the path. This provides the shape of the stroke. You also have an ink, which describes the appearance of the stroke: its color and type. A transform gives the orientation and position of the stroke. Strokes can also have masks, and we'll discuss those later in this talk.
Another useful property of strokes is the renderBounds, and this is a bounding box that encompasses the entirety of the stroke when it is rendered. The renderBounds accounts for the effect of all the stroke properties including the path, ink, transform and mask.
Inks, which describe what a stroke looks like, contain the type of ink and a color. Inks do not have a width. The width of a stroke is variable along the stroke path. The stroke path describes the shape of the stroke and the appearance of that shape as it changes along the path. For example, the stroke path gives you the width of the stroke at any point. A PencilKit stroke path is a uniform cubic B-spline of PencilKit stroke points. Now, that's quite a mouthful. What does that mean? It means that the contents of a path are, in fact, the control points for the B-spline. So if we iterate over the points in a path...
and draw each one in turn...
the resulting points are not actually on the stroke. These points are the B-spline control points and probably not what you want to draw. To get points on the actual path, we need to interpolate the spline.
To interpolate the spline, we access the points using interpolatedPoints strideBy. This provides a sequence of points that we can iterate over like before. Drawing these gives us a series of points on the path. There are a couple of things to notice here. They're on the path, there are more of them and they have uniform spacing, in this case, a distance of 50 points, which is the stride argument passed in to the method.
You might also notice that the spacing of the last point is uneven. This is because the last point on a stroke is always generated regardless of the stride. You can stride by distance, as in this example, time or parametric value. Distance and time are self explanatory. Distance is points in drawing coordinate space, and time is duration in seconds, which depends on how fast the user drew. Parametric value relates to the parametric interpolation of the B-spline. To explain what the parametric value is, let's bring back drawing the control points. This is the same code that we used earlier. If instead of drawing the control points, we iterate over the indices of the path, which go from zero to the control point count, and at each iteration we get the point using interpolatedPoint(at, for the parametric value 0, 1, 2, 3 and so on and draw that...
we get the equivalent points to the control points. But these points are actually on the path. Why is this useful? Let's number the points so you can see what's going on. The parametric value is useful because it is a floating point value. That means you can ask for the interpolated point for any value, including non-integer values between control points like 2.4 or 4.8... and so on. This gives you the flexibility to interpolate the stroke path any way you want. All the interpolation we've seen so far has been interpolating the path with a uniform step. Using the parametric value, PencilKit also provides the ability to step along a path by an arbitrary distance, using parametricValue, offsetBy.
This function allows you to offset a parametric value on a path forwards or backwards any step in time or distance.
One of the places where non-uniform stepping is useful is when animating. The demo I gave earlier uses this ability to animate the red marker dot along the strokes. Each frame, the current marker position on a stroke, is offset by the exact time duration since the last frame.
Non-uniform stepping is necessary because we're not always guaranteed a uniform amount of time between animation frames.
To animate in the demo, first we get the delta time, the time that has elapsed between the current frame and the previous frame.
We use that to offset the current animation parametric value along the path by the same amount of time. This animates along the stroke path with the same velocity as when the user drew it.
Finally, we update the marker position, getting the new location on the path from the new parametric value.
So that's the path. Both the control points and the interpolated points along the path are PencilKit stroke points. These are the atomic building blocks of paths and strokes. They capture both the appearance and touch information of a stroke at a particular location. These points are stored in a lossily compressed format, so any points you create will not capture the values you use with perfect precision. Let's take a closer look at one of these points in a stroke.
A PencilKit stroke point has several appearance attributes.
The first of those is the location of the point.
A point also has a size, which for marker strokes won't be square. A rotation angle, or azimuth.
And finally, the opacity. These attributes combine to describe how a stroke appears at a certain location.
Stroke points also have a couple of properties that are not appearance attributes. Force and altitude match the same values from UITouch when the stroke was drawn. Time offset is the offset in seconds from the creation date of the stroke path that the point belongs to. This provides timing information for how the user drew the stroke.
We'll jump back now to talk about the last property of PencilKit strokes that we haven't covered... and that is masking.
Masked strokes are typically created when the pixel eraser is used to erase only a portion of a stroke. Most strokes are not usually erased, but when they are, the mask is used to clip these strokes in rendering and adjust how the user can interact with them on the canvas. Masks can have holes.
Or they can cut a stroke into multiple pieces. In this example, using the eraser, the stroke has been split into two separate strokes.
These become unique, independent strokes and behave as such to the user and in the API. For example, each of these two new strokes has its own separate transform and mask. The user can select one of the strokes and move it around without affecting the other stroke.
PencilKit strokes are masked, but stroke paths are not. This means that if we take the code we were using earlier to draw a stroke path and use that code to draw a masked stroke...
we're going to get a much longer path than we wanted.
Instead, we want to use the maskedPathRanges property of the stroke. This is an array of parametric value ranges on the stroke path when it is clipped to the mask. Here we iterate over the maskedPathRanges...
and interpolate the points in each of those ranges.
This correctly gives an interpretation of the stroke path in a way that makes sense for a masked stroke.
Strokes can have zero masked ranges. For example, if the user erases all but a fraction of a stroke, and that fraction does not intersect the path spline, then the resulting masked stroke will have zero maskedPathRanges.
Strokes can also have multiple masked ranges. In this case, a stroke with holes in it has four individual ranges.
Recognition is a building block for many great features that you can build with PencilKit. Spline-based recognition can make use of these maskedPathRanges to provide a sensible interpretation of masked strokes, and this is what we do for handwriting recognition in Notes.
When interpreting strokes, you can use maskedPathRanges to get a range of points, interpolate them how you want and use non-appearance attributes like time and force to supplement the shape of the path.
The demo you saw earlier provides a simple example of spline-based recognition. It uses a matching algorithm to compare and score the user on the similarity between what they drew and the template they're trying to copy. If you want to do image-based recognition, use the rendering API on PKDrawing to generate images. PencilKit provides a super easy way to add great Pencil support to your app. Now that you can look inside drawings and access the strokes, inks, paths and points, inspect what the user wrote and drew to build features like the new Scribble experience which uses PencilKit to enable handwriting in text fields across the whole of iPadOS. Modify drawings to create interactive drawing experiences that respond to the user's actions. And create new drawings procedurally, like the sample code does to generate handwriting templates for practicing.
Adding support for Pencil, one of our most expressive input devices, is a great addition to almost any app. PencilKit has always been a fantastic way to add drawing to your app, and now that you can look inside drawings, it's also an incredibly powerful foundation for new Pencil-focused experiences that you want to build.
-
-
찾고 계신 콘텐츠가 있나요? 위에 주제를 입력하고 원하는 내용을 바로 검색해 보세요.
쿼리를 제출하는 중에 오류가 발생했습니다. 인터넷 연결을 확인하고 다시 시도해 주세요.