Is `.appEntityIdentifier` + `Transferable` the intended way to let Siri send an on-screen image to another app? (iOS 27)

Question

Created 10h

Replies 2

Boosts 3

Participants 3

I'm trying to make a third-party app's on-screen image available to Siri / Apple Intelligence so the user can say something like "send this to <app>" and have the image handed off. And I'd like to confirm whether I'm using the intended mechanism or whether this particular case just isn't supported yet.

What I'm trying to do

My app shows a single image (it owns no photo library, the image is an in-app render). I want the user to be able to reference it as "this" and have Siri move it to another app.

What I implemented (following Making onscreen content available to Siri and Apple Intelligence and WWDC26 240/343)

A plain AppEntity with a stable id, conforming to Transferable, annotated on the image view with the iOS 27 .appEntityIdentifier(_:) modifier:

struct OnScreenImageEntity: AppEntity, Transferable {
    static let typeDisplayRepresentation = TypeDisplayRepresentation(name: "On-Screen Image")
    static let defaultQuery = Query()
    let id: String

    static var transferRepresentation: some TransferRepresentation {
        DataRepresentation(exportedContentType: .plainText) { … } // distinctive text
        DataRepresentation(exportedContentType: .png)       { … } // the image
    }

    struct Query: EntityQuery {
        func entities(for ids: [String]) async throws -> [OnScreenImageEntity] {
            logger.notice("entities(for:) CALLED \(ids)")   // <- never fires
            return ids.filter { store.isCurrent($0) }.map(OnScreenImageEntity.init)
        }
    }
}

// on the view:
Image(uiImage: image)
    .appEntityIdentifier(EntityIdentifier(for: OnScreenImageEntity.self, identifier: id))

What I observe (iOS 27 Beta 3, device)

"Describe this image" / "Create a note for this" appear to use an automatic screenshot — the responses reference my app's UI chrome, and EntityQuery.entities(for:) is never called, so my entity's Transferable is not involved at all.
"Send this to <contact>" doesn't attach the image; Siri says something like "I can't attach the image directly from your screen," which again sounds like a screen grab. entities(for:) is not called.
The only flow that resolves my entity is the ChatGPT hand-off: asking Siri about the on-screen content calls entities(for:) (several times). But even there it stalls — Siri responds "could not clarify what you mean by 'about this'… document, image, or topic?", and no Transferable representation is ever read (my export closures never run). So the entity resolves, yet the content is never actually transferred.
The older NSUserActivity.appEntityIdentifier route was never consumed at all; only the iOS 27 .appEntityIdentifier(_:) view modifier produced any resolution, and only in the ChatGPT flow above.

I also understand from WWDC26 240/343 that the cross-app content move is built on IntentValueRepresentation (entity ↔ a system value type like IntentPerson), which the recipient imports via IntentValueQuery / IntentValueRepresentation(importing:). Since there doesn't seem to be a system value type for an arbitrary image (and IntentFile isn't one), I'm not sure an image entity can participate in that transfer at all.

My questions

Is .appEntityIdentifier(_:) the intended way to expose a single on-screen item in iOS 27? It only resolves my entity in the ChatGPT hand-off, and even then the Transferable is never read — under what conditions is the resolved entity's Transferable actually consumed?
Are on-screen content requests ("describe this", "create a note for this") ever meant to use an app-provided AppEntity + Transferable, or are they always served by screen capture? (For me they never call entities(for:).)
Is there a supported path for Siri to send an image from a third-party app's on-screen content to another app? If the transfer requires IntentValueRepresentation and there's no image system value, is a Transferable image representation ever consumed for this — and if so, how is it triggered?

If this is simply not supported for image content yet, that's useful to know too, I just want to make sure I'm not missing a required piece (a schema conformance, an eligibility flag, a different annotation API, etc.).

I have a minimal sample project that reproduces this (single annotated image, no photo library, logs to a subsystem so you can see entities(for:) never being called) and filed it as Feedback FB23813341 — happy to share details.

Thanks!

Answer 1

DTS Engineer OP

Apple

6h

Thanks for the very detailed post and the bug at the end. This is great for many developers to check the issues you have posted.

There is so much you have provided including issues making onscreen content available to Siri and Apple Intelligence and providing contextual cues to Apple Intelligence and Siri.

All the 4 sample I’m just familiar with photo app and messaging app where the entity through an NSUserActivity, not just a standalone .appEntityIdentifier(_:) view annotation.

Because you submitted a focused sample and the same explanation on a good written bug, I’ll let the team take over all the issues.

You can see the status of your feedback in Feedback Assistant. There, you can track if the report is still being investigated, has a potential identifiable fix, or has been resolved in another way. The status appears beside the label "Resolution." We're unable to share any updates on specific reports on the forums.

For more details on when you'll see updates to your report, please see What to expect after submission.

Albert  WWDR

Answer 2

J0hn OP

2h

So, this actually works for me...

If I right click on a gallery item on iPad and type "send this to <myself>" it will grab all items on screen and put them into a message.

Here is what I have in my UICollectionView:

public func collectionView(
        _ collectionView: UICollectionView,
        appEntityIdentifierForItemAt indexPath: IndexPath
    ) -> EntityIdentifier? {
        guard let item = dataSource?.itemIdentifier(for: indexPath) else { return nil }
            guard let fileIdentifier = try? FileEntityIdentifier.file(url: <URL>) else { return nil }
            return EntityIdentifier(for: <ENTITY>.self, identifier: fileIdentifier)
    }

Note using the "FileEntityIdentifier" with the file's URL.

Also, the <ENTITY> that gets provided happens to conform to @AppEntity(schema: .files.file)

My Transfer representations are just the following:

 public static var transferRepresentation: some TransferRepresentation {
        FileRepresentation(
            contentType: .image,
            exporting: { entity in
                print(" INT Exporting File Entity as File Representation")
                guard let url = try await entity.id.fileURL else {
                    throw Errors.unableToRetrieveURL
                }
                
                return SentTransferredFile(url)
            },
            importing: { received in
                let attributes = try? FileManager.default.attributesOfItem(atPath: received.file.path())
                let creationDate = attributes?[.creationDate] as? Date
                let modificationDate = attributes?[.modificationDate] as? Date
                
                print("INT Attempting import of Image Entity from File Representation")
                return <ENTITY>(
                    id: try FileEntityIdentifier.file(url: received.file),
                    creationDate: creationDate,
                    fileModificationDate: modificationDate,
                    name: received.file.lastPathComponent
                )
            })
    }