I am creating an application using PKCanvasview. One function of that app is the ability to trace and retrieve a UITextView that has been addSubviewed to a PKCanvasView. We have confirmed that this functionality works correctly in the simulator and on the actual device on IOS 17.4. This feature did not work correctly on the IOS18 simulator and the actual device. Is this a bug? And if it is normal behavior, is there an alternative?

When a custom tool items set is formed in PKToolPicker with each inking item having non-nil identifier created using PKToolPickerInkingItem(type: ,color:,width:,identifier: ), changing color or width of an inking item (pen, pencil etc) causes an instant crash. I believe it is a bug in PencilKit. It seems that when you change the color or width of an inking item having a identifier in squeeze tool palette, it tries to find a tool item without identifier (the default tool picker has items without identifier) in the tool items set. I guess it cannot find, thus the find function returns either -1 or highest integer number (2^63 -1 ) and it uses this number as index without boundary checking. That's why we observe [__NSArrayM replaceObjectAtIndex:withObject:]: index 9223372036854775807 beyond bounds [0 .. 9] I filed a report on Feedback Assistant with id: FB15519801 too. The corresponding part in crash report is as follows: 0 CoreFoundation 0x183e0908c __exceptionPreprocess + 164 (NSException.m:249) 1 libobjc.A.dylib 0x18110b2e4 objc_exception_throw + 88 (objc-exception.mm:356) 2 CoreFoundation 0x183de4048 -[__NSArrayM replaceObjectAtIndex:withObject:] + 1020 (NSArrayM.m:180) 3 PencilKit 0x1c44f73c8 -[PKToolPicker _setSelectedTool:saveState:updateUI:updateLastSelectedTool:] + 800 (PKToolPicker.m:587) 4 PencilKit 0x1c45a5684 -[PKPencilSqueezeControllerPaletteViewDelegateProxy paletteView:didSelectTool:atIndex:] + 200 (PKPencilSqueezeControllerPaletteViewDelegateProxy.m:227) 5 PencilKit 0x1c460906c -[PKSqueezePaletteView _didSelectTool:atIndex:] + 196 (PKSqueezePaletteView.m:441) 6 PencilKit 0x1c462203c -[PKSqueezePaletteViewExpandedInkingToolLayout _didTapStrokeWeightButton:] + 336 (PKSqueezePaletteViewExpandedInkingToolLayout.m:224) 7 UIKitCore 0x18691edd8 -[UIApplication sendAction:to:from:forEvent:] + 100 (UIApplication.m:5797) 8 UIKitCore 0x18691ecb0 -[UIControl sendAction:to:forEvent:] + 112 (UIControl.m:942) 9 UIKitCore 0x18691eb00 -[UIControl _sendActionsForEvents:withEvent:] + 324 (UIControl.m:1013) 10 UIKitCore 0x187080568 -[UIButton _sendActionsForEvents:withEvent:] + 124 (UIButton.m:4192) 11 UIKitCore 0x187081d7c -[UIControl touchesEnded:withEvent:] + 400 (UIControl.m:692) 12 UIKitCore 0x1868675b0 -[UIWindow _sendTouchesForEvent:] + 852 (UIWindow.m:3313) and the exception reason is *** -[__NSArrayM replaceObjectAtIndex:withObject:]: index 9223372036854775807 beyond bounds [0 .. 9]

There is a significant rendering issue in PencilKit when using an iPad set to "More Space" display mode, combined with an Apple Pencil that supports hover functionality (e.g., Apple Pencil 2). This problem affects all applications that use PencilKit, including Apple's own Notes and Quick Note. The issue results in flickering black borders and subtle jittering while drawing, which is especially noticeable during tasks requiring precise handwriting, such as writing mathematical expressions. Due to the short strokes and frequent lifts and drops of the pencil, the jitter is much more pronounced, leading to visual discomfort and even dizziness after extended use. Steps to Reproduce: Open Settings on your iPad. Navigate to Display & Brightness > Display Zoom > More Space. Switch to the More Space display mode. Open the Notes app or trigger Quick Note from any application by swiping from the bottom-right corner. Start drawing or writing using the Apple Pencil (with hover functionality) in the writing area. Observe the display anomalies as you hover and write: Flickering black borders appear intermittently around the writing area. The strokes show subtle jittering whenever you lift and lower the pencil. This is particularly disruptive when writing short, precise strokes, such as those in mathematical expressions, where the frequent up-and-down motion makes the jitter more apparent. Expected Results: Smooth and stable drawing experience with no visual artifacts or jittering during interactions with the Apple Pencil, regardless of the display zoom settings. Actual Results: Flickering black borders intermittently appear during drawing. Jittering of strokes is noticeable, particularly when lifting and lowering the Apple Pencil for short strokes. This disrupts precise writing tasks, especially in cases like mathematical notation, where frequent pen lifts and short strokes make the jitter much more apparent. This can lead to discomfort (e.g., dizziness) after extended use. System-Wide Impact: This issue affects all apps that utilize PencilKit, including third-party apps such as Doodle Draw (link). Since PencilKit is a core framework, the rendering bug significantly degrades the user experience across multiple productivity and note-taking applications. Similar Reports: There are numerous discussions about this issue online, indicating that many users are experiencing the same problem. Reddit Discussion: https://www.reddit.com/r/ipad/comments/1f042ol/comment/ljvilv6/ Apple Support Thread Additionally, a feedback report (ID: FB15166022) related to this issue with Notes has been previously filed, but the bug remains unresolved. This bug severely impacts the usability of PencilKit-enabled apps, especially during tasks that involve frequent, precise pencil strokes, such as writing mathematical expressions. We kindly request the development team investigate this issue promptly to ensure a smooth, stable experience with the iPad's "More Space" display mode and Apple Pencil hover interaction.

I want to add a pkcanvasview image and change its position and size by dragging. And I want to draw a picture using a pencil on top. The image must be able to change its position at any time, and a picture must be added on top. If I add it to the subview of pkcanvasview, I cannot do what I want. How can I do this?

Attempting to recreate PKDrawing point-by-point using an existing drawing instance results in .pencil strokes being displayed with different (thinner) width than the original. Strokes with all other inks are restored fully visually identically to the original. This can be observed from iOS 15.2 to iOS 18 beta 5. Code used to remake the drawing: let restoredDrawing = PKDrawing(strokes: drawing.strokes.map { stroke in PKStroke( ink: stroke.ink, path: PKStrokePath(controlPoints: (0..<stroke.path.count).map { controlIndex in // This could be replaced with stroke.path.map { point in } let point = Array(stroke.path)[controlIndex] // This does not result in the same presentation as the point let restoredPoint: PKStrokePoint if #available(iOS 17.0, *) { restoredPoint = PKStrokePoint( location: point.location, timeOffset: point.timeOffset, size: point.size, opacity: point.opacity, force: point.force, azimuth: point.azimuth, altitude: point.altitude, secondaryScale: point.secondaryScale ) } else { restoredPoint = PKStrokePoint( location: point.location, timeOffset: point.timeOffset, size: point.size, opacity: point.opacity, force: point.force, azimuth: point.azimuth, altitude: point.altitude ) } // Even this produces correct result: // let restoredPoint = stroke.path.interpolatedPoint(at: CGFloat(controlIndex)) compare(point, restoredPoint) return restoredPoint }, creationDate: stroke.path.creationDate), transform: stroke.transform, mask: stroke.mask ) }) Gist with ViewController with side-by-side to paste into new project: https://gist.github.com/ilevIO/a1dea60ab6cb16047de2b421897d30f1

The sample code project PencilKitCustomToolPicker found at this article Configuring the PencilKit tool picker here fails to compile with the following errors in Xcode 16 beta 6

I’m using PDFPageOverlayViewProvider with pdfview. I want to control the visibility of the overlay view using a button. However, the view updates only when it disappears and reappears. I would like the changes to be reflected immediately. How can I achieve this? struct PDFKitView: View { let bookId: UUID let bookTitle: String @State private var currentPage = "1" @State private var isSideTab = false @State private var selectedNote: [SelectedNote] = [] var body: some View { let pdfDocument = openPDF(at: bookId.uuidString) ZStack(alignment: .trailing) { HStack(spacing: 0) { PDFKitRepresentableView( bookId: bookId, selectedNote: $selectedNote, currentPage: $currentPage, pdfDocument: pdfDocument ) if isSideTab { SideView() .transition(.move(edge: .trailing)) .zIndex(1) .frame(maxWidth: 260) } } .padding(.top, 73) } .onAppear { getAllNote(bookId: bookId) } .customNavigationBar(back: true) { Text("\(currentPage)/\(pdfDocument.pageCount)") .pretendard(.CaptionRegular) .foregroundStyle(Color.Text.primary) } TrailingView: { Button(action: { withAnimation { isSideTab.toggle() } }) { Image(systemName: SFSymbol.squareStack3dDownForwardFill.icon) .sfPro(.IconMedium) .foregroundStyle(Color.Text.primary) } } } @ViewBuilder func SideView() -> some View { VStack(spacing: 16) { HStack(spacing: 4) { Image(systemName: SFSymbol.squareStack3dDownForwardFill.icon) .sfPro(.IconSmall) .foregroundStyle(Color.Text.primary) Text(Literals.sideTabTitle) .pretendard(.P2Bold) .foregroundStyle(Color.Text.primary) Spacer() } .padding(16) .background { Color.Background.white } ScrollView { ForEach($selectedNote, id: \.noteId) { note in NoteCellView(note: note) } } .background { Color.Fill.white } } .background { Color.Background.blueGray } } @ViewBuilder func NoteCellView(note: Binding<SelectedNote>) -> some View { HStack(alignment: .top, spacing: 16) { Image(.writingNote) .resizable() .scaledToFit() .frame(width: 42, height: 60) VStack(alignment: .leading, spacing: 8) { Text(note.wrappedValue.noteId == bookId.uuidString ? "345" : "123") .foregroundStyle(Color.Text.secondary) .padding(.horizontal, 8) .background { Rectangle() .strokeBorder(Color.Layout.secondary) } Text(bookTitle) .lineLimit(2) Toggle("", isOn: note.selected) .labelsHidden() .tint(Color.Fill.activePrimary) } } .padding(EdgeInsets(top: 20, leading: 16, bottom: 16, trailing: 16)) } } struct PDFKitRepresentableView: UIViewRepresentable { let bookId: UUID @Binding var selectedNote: [SelectedNote] @Binding var currentPage: String let pdfDocument: PDFDocument let pdfView = PDFView() let toolPicker = PKToolPicker() func makeUIView(context: Context) -> PDFView { pdfView.displayMode = .singlePageContinuous pdfView.usePageViewController(false) pdfView.displayDirection = .vertical pdfView.pageOverlayViewProvider = context.coordinator pdfView.autoScales = true pdfDocument.delegate = context.coordinator pdfView.document = pdfDocument return pdfView } func updateUIView(_ uiView: PDFView, context: Context) { if let localNote = selectedNote.first(where: {$0.noteId == bookId.uuidString}), !localNote.selected { toolPicker.setVisible(false, forFirstResponder: uiView) } else { toolPicker.setVisible(true, forFirstResponder: uiView) } uiView.becomeFirstResponder() } func makeCoordinator() -> CanvasProvider { return CanvasProvider(parent: self) } } final class CanvasProvider: NSObject, PDFPageOverlayViewProvider, PDFDocumentDelegate { var localNotes = [PDFPage: PKCanvasView]() var passNotes = [PDFPage: Image]() let parent: PDFKitRepresentableView init(parent: PDFKitRepresentableView) { self.parent = parent super.init() getDrawingDatas( bookId: parent.bookId.uuidString, selectedNote: parent.selectedNote, document: parent.pdfDocument ) } func pdfView(_ view: PDFView, overlayViewFor page: PDFPage) -> UIView? { var coverView: PKCanvasView? = PKCanvasView() if let view = localNotes[page], parent.selectedNote.first(where: { $0.noteId == parent.bookId.uuidString })?.selected ?? false { view.backgroundColor = .clear view.isOpaque = true view.drawingPolicy = .anyInput view.delegate = self parent.toolPicker.addObserver(view) coverView = view (page as? CanvasPDFPage)?.canvasView = view } else { coverView = nil } for subView in view.documentView?.subviews ?? [] { if subView.theClassName == "PDFPageView" { subView.isUserInteractionEnabled = true } } return coverView } func pdfView(_ pdfView: PDFView, willDisplayOverlayView overlayView: UIView, for page: PDFPage) { } func pdfView(_ pdfView: PDFView, willEndDisplayingOverlayView overlayView: UIView, for page: PDFPage) { } }

Does anybody knows how to implement draw on pdf function for iOS? Approach is the same as Files app. Telegram and Whatsapp using the same technique.

Hey, so I am working on a note taking part of my app and I have been working on allowing different inputs. I got the PKCanvas to work on the preview and on the simulator when it is the only thing on the window group. But when I use this View as part of the tool selection in my app, when the user selects the Canvas view it loads but scribbles don't work, only in the preview in Xcode. I don't know why this behavior is happening. The flow in which the PKCanvasView appears is, user taps NavLink to see their notes, in the NotesView they tap to show the toolbar, toolbar has 4 types of input, text, images, audio, scribbles (all other inputs work accordingly), user taps scribbles the view loads correctly but doesn't allow them to draw. I have set the drawingPolicy to anyInput

Dear All, I am currently facing a challenge with updating the contentScaleFactor of my PKCanvasView, which is embedded within a custom UIScrollView. I have configured the PKCanvasView to have both its maximum and minimum zoomScale set to 1.0, ensuring that users can only zoom into the UIScrollView and not directly into the PKCanvasView. Upon the completion of a zoom action in my UIScrollView, a notification is published and received by the PKCanvasView. This triggers a function intended to update the contentScaleFactor. However, this process is not functioning as expected. Despite consulting with several engineers at WWDC24 and exploring multiple potential solutions, the issue remains unresolved. Interestingly, when zooming into a PDF in the Documents app on iOS 17, the strokes are re-rendered perfectly, while in previous iOS versions, this bug persists in the Documents app. I would greatly appreciate any insights or solutions from those who might have encountered and resolved this issue. Thank you in advance for your assistance. Best regards, Timon

When using PencilKit and the default PKCanvasView, we get a pop-up menu with options like 'Select All | Insert Space' when we long tap on the canvas. What is the proper way to remove this menu completely? Thank you for your help.

I'm attempting to retrieve the values of the Pencil Pro using the new PencilHoverPose, however according to the documentation it appears these can only be accessed while in a hover state (0 to ~0.5" from the screen). Is there a way to get the orientation or roll values of the Apple Pencil Pro outside of a hover? I'm attempting to use the pencil as an input device, necessitating it be farther from the screen than 0.5" Below is a simple view I'm using to display the available values: import SwiftUI import UIKit struct PencilSqueezeView: View { @State private var squeezeText: String = "Squeeze the Apple Pencil Pro" @State private var hoverDistance: CGFloat = 0.0 @State private var anchor: UnitPoint = .center @State private var location: CGPoint = .zero @State private var altitude: Angle = .degrees(0) @State private var azimuth: Angle = .degrees(0) @State private var roll: Angle = .degrees(0) var body: some View { VStack { Text(squeezeText) .font(.largeTitle) .padding() Text("Hover Distance: \(String(format: "%.2f", hoverDistance))") .font(.title2) .padding() Text("Anchor: \(anchor)") .font(.title2) .padding() Text("Location: \(location.x), \(location.y)") .font(.title2) .padding() Text("Altitude: \(altitude.degrees)°") .font(.title2) .padding() Text("Azimuth: \(azimuth.degrees)°") .font(.title2) .padding() Text("Roll: \(roll.degrees)°") .font(.title2) .padding() Spacer() } .onPencilSqueeze { phase in handlePencilSqueeze(phase: phase) if case let .ended(value) = phase { if let hoverPose = value.hoverPose { updateHoverPose(hoverPose) } else { print("hoverPose is nil") } } } .padding() } private func handlePencilSqueeze(phase: PencilSqueezeGesturePhase) { switch phase { case .active: squeezeText = "Squeeze began" case .ended: squeezeText = "Squeeze ended" case .failed: squeezeText = "Squeeze failed" } } private func updateHoverPose(_ hoverPose: PencilHoverPose) { hoverDistance = hoverPose.zDistance anchor = hoverPose.anchor location = hoverPose.location altitude = hoverPose.altitude azimuth = hoverPose.azimuth roll = hoverPose.roll print("zDistance: \(hoverPose.zDistance)") print("anchor: \(hoverPose.anchor)") print("location: \(hoverPose.location)") print("altitude: \(hoverPose.altitude)") print("azimuth: \(hoverPose.azimuth)") print("roll: \(hoverPose.roll)") } }

I wonder if an Apple engineer could confirm: will the Apple Pencil Pro squeeze functionality be detectable in the current API, or will this be a future iPadOS extension to gesture recognizers / UIKit? I’d like to start playing with the functionality if it’s detected behind an existing event though. (Long press?)

When PKCanvasView is first drawn, existing drawing objects disappear. PKDrawing, which previously had drawings saved with PKCanvasView, was saved as separate data or file. After that, while creating a new PKCanvasView, I loaded the saved PKDrawing and reflected it in the new PKCanvasView. Below is the code. canvasView!.drawing = draw! Previously saved text or lines will be displayed normally. However, when I draw with the pen for the first time (when I touch the screen with the pen), the old writing or lines disappear. And after I leave a line or text, if I pan across the screen, the old text or line appears again. If this phenomenon occurs and you touch the screen again with the fan, this phenomenon will no longer occur. This phenomenon occurs the first time when a new PKCanvasView is declared and the previously saved PKDrawing is reflected. Could you please help me with why this phenomenon occurs and how to improve it?

Hi, I am trying to make a simple note taking app that users can draw something on pdfview with apple pencil. (I used PDFKit, PencilKit for the code.) I followed the instruction code of WWDC22's "What's new in PDFKit." - "overlayProvider" (so you can see the code at the video.) I was able to draw something each view of pdf page. But the issue was, the resolution of overlayview or subview of pdfview is low. As far as I know, the pkcanvasview draws vertor-based drawings. So I never thought the image or the lines I draw will be that blurry. Is this buggy or is this the normal thing? (+ I added a uibutton as subview of pdfview and the button also looks blurry.) I even tried to scale up the all the subviews when the subviews' layout is done, using contentScaleFactor. PKCanvasView inherits UIScrollView, so I enlarged the frame of pkcanvas view and fixed the scale to below 1.0. If the pkcanvasview looks blurry and that is because somewhat zoomed in wrong way, zooming out should be the solution. But, didn't work. Still blurry. and any other stuff like changing frame or size. So, anyone having same problem with me, or anyone can give me any solution. Please help me. I wish this is bug thing that can be fixed in any moment. -> This image is little bit zoomed in. but the drawing is blurry. and this is the normal pkcanvasview drawing, just subview of view(of VC).

Hi, I have an app using PencilKit that works on VisionOS 1.0. It means, a user can pick an inking tool from PKToolPicker and draw on PKCanvasView. The app is now on available on Vision Pro AppStore as well. However, when I test the app on VisionOS 1.1 RC Simulator, I can pick an inking tool but when I try to draw on the canvas, it just scrolls and no drawing appears on the PKCanvasView. I also noticed that the VisionOS 1.0 Simulator has the FreeForm app where you can draw with PencilKit but the VisionOS 1.1 RC Simulator does not have the FreeForm app. Is this a known issue? Will it be fixed before the release or is there a change in API so I can update the app accordingly? Thanks Zafer

Hello! I've been teaching myself Swift and wanted to challenge myself by creating a drawing app. I want to add a feature that allows the user to share the drawing on their canvas, but I'm having some difficulty. I tried using a ShareLink, but it's asking that I conform in to Transferable. How do I do that? import SwiftUI import PencilKit struct ContentView: View { @State private var canvasView = PKCanvasView() var body: some View { Text("Let's draw!") GeometryReader { geometry in VStack { Spacer() HStack { Spacer() PencilKitView() .frame(width: geometry.size.width * 1, height: geometry.size.height * 0.7) Spacer() } Spacer() } } //share button Button("share with a friend") { let portionRect = CGRect(x: 0, y: 0, width: 100, height: 100) let scale: CGFloat = 1.0 let image = canvasView.drawing.image(from: portionRect, scale: scale) ShareLink( item: image, preview: SharePreview( "Share Preview", image: image ) ) } } } struct PencilKitView: UIViewRepresentable { typealias UIViewType = PKCanvasView let toolPicker = PKToolPicker() func makeUIView(context: Context) -> PKCanvasView { let pencilKitCanvasView = PKCanvasView() pencilKitCanvasView.drawingPolicy = PKCanvasViewDrawingPolicy.anyInput toolPicker.addObserver(pencilKitCanvasView) toolPicker.setVisible(true, forFirstResponder: pencilKitCanvasView) pencilKitCanvasView.becomeFirstResponder() return pencilKitCanvasView } func updateUIView(_ uiView: PKCanvasView, context: Context) { } } #Preview { ContentView() }

In 2 days we have observed in Crashlytics over 50 crashes related to PKDrawing.image with EXC_BAD_ACCESS KERN_INVALID_ADDRESS 0x0000000000000210 So far all on iPads with A10, A12 and A13; 100% on iOS 17 (17.1.1, 17.2, 17.3) (while in others the percent of iOS 17 was around 60-80%). Context: Images of the varying frame and scale resulting in screen resolution are being generated sequentially (in a background serial queue called almost one after another when requested), updating one CALayer.contents (and only after this update on Main Thread the next generation is allowed). One zoomable PKCanvasView is present on screen. The crashing line in code: let image = drawing.image(from: frame, scale: renderScale).cgImage The questions: Is there anything that can be done apart from throttling generation? Can the circumstances of the crash be determined – are there any indications accessible in code before calling PKDrawing.image that app might crash? The traces: 0 AGXMetalA12 AGX::BlitContext<AGX::G11::Encoders, AGX::G11::Classes, AGX::G11::ObjClasses>::copyTextureToBuffer(IOGPUMetalResource const*, unsigned long, unsigned long, unsigned long, AGXA12FamilyTexture*, unsigned int, unsigned int, MTLOrigin, MTLSize, unsigned long) + 96 9 PencilKit PKDrawing.image(from:scale:) + 28 0 AGXMetalA13 <redacted> + 96 9 PencilKit PKDrawing.image(from:scale:) + 28 0 AGXMetalA10 <redacted> + 72 9 PencilKit $s9PencilKit9PKDrawingV5image4from5scaleSo7UIImageCSo6CGRectV_12CoreGraphics7CGFloatVtF + 24

Is there any chance to modify the strokes of PKDrawing while drawing in on-process ? since I notice in canvasViewDrawingDidChange is called after drawing stroke is finished My objective is to get realtime feedback modification of PKStroke Thank you in advance

I've noticed in FreeForm app drawing stroke is very nice, like vector object, is there any chance to get the same result of drawing stroke quality like in the FreeForm app ?