Is Data.withUnsafeMutableBytes(_:) safe to rely on long-term for C++ interop?

Hi, in our project we use Network.framework, with NWListener and NWConnection, to send and receive data over UDP. We use the receiveMessage(completion:) API, which delivers the data as a Data object. We assign this Data to a property of one of our classes, which we continue to work with after the completion handler returns. We pass that property's bytes to our C++ code (through Swift–C++ interop) using Data.withUnsafeMutableBytes(_:). The C++ code modifies the bytes in place, and we then continue working with the modified Data in Swift. Since nothing else refers to the Data we received, the bytes are modified in place and no copy is made. However, if another Data value shared the same buffer, just calling withUnsafeMutableBytes(_:) would create a copy, whether or not we actually modified the bytes. Our understanding is that this is because the method is declared mutating, so the call itself is treated as a write and copy-on-write makes the buffer unique before the pointer is handed out. We also understand that ResultType : ~Copyable (along with the typed throws(E)) only relaxes the requirements on the closure's return type and doesn't affect this behaviour. We understand this approach is unsafe and that the pointer "should not be stored and used outside of the lifetime of the call to the closure", so we will take care of that. Since the typed-pointer variants of APIs such as Data.withUnsafeBytes were deprecated in Swift 5, we would also like to confirm that withUnsafeMutableBytes(_:), the variant that takes an UnsafeMutableRawBufferPointer, is the intended long-term API and isn't expected to be deprecated in the same way.

Answered by DTS Engineer in 910589022
Our understanding is that this is because the method is declared mutating

Yes, but that’s not really the main point. Rather, the withUnsafeMutableBytes(_:) call has to pass a mutable pointer to its closure and it can’t tell whether that closure is or isn’t going to mutate the bytes. Thus it must necessarily ensure that the bytes are mutable, and hence it triggers the CoW.

we would also like to confirm that withUnsafeMutableBytes(_:) … is the intended long-term API and isn't expected to be deprecated in the same way.

I don’t think it’s going away any time soon, but it’s fundamentally unsafe and thus not something that we encourage folks to use. Swift is moving towards a world where this sort of low-level access is both efficient and safe [1]. It’s doing this via the Span type (and its variants, like MutableSpan, RawSpan, and so on). Notably, these bridge to the similarly named C++ types, which gives you efficient and safe [2] integration to C++.

So, if you’re looking to the future then my advice would be:

  • Adopt span types in your C++ code. You can start out by making these thin wrappers around the legacy pointer-based C++ code, after which you can incrementally adopt this at lower-layers of the codebase.
  • Alternatively, you can use annotations to tell the Swift compiler to import methods that take a pointer and a count twice, once as per usual and again as if it were declared to use a span.
  • Use that to integrate with Swift.

To learn more about this:

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] I’m using safe in the sense commonly used by Swift APIs, namely that it’s memory safe.

[2] Well, the Swift side is safe. The C++ side is harder to make safe, although even that’s becoming feasible these days.

Our understanding is that this is because the method is declared mutating

Yes, but that’s not really the main point. Rather, the withUnsafeMutableBytes(_:) call has to pass a mutable pointer to its closure and it can’t tell whether that closure is or isn’t going to mutate the bytes. Thus it must necessarily ensure that the bytes are mutable, and hence it triggers the CoW.

we would also like to confirm that withUnsafeMutableBytes(_:) … is the intended long-term API and isn't expected to be deprecated in the same way.

I don’t think it’s going away any time soon, but it’s fundamentally unsafe and thus not something that we encourage folks to use. Swift is moving towards a world where this sort of low-level access is both efficient and safe [1]. It’s doing this via the Span type (and its variants, like MutableSpan, RawSpan, and so on). Notably, these bridge to the similarly named C++ types, which gives you efficient and safe [2] integration to C++.

So, if you’re looking to the future then my advice would be:

  • Adopt span types in your C++ code. You can start out by making these thin wrappers around the legacy pointer-based C++ code, after which you can incrementally adopt this at lower-layers of the codebase.
  • Alternatively, you can use annotations to tell the Swift compiler to import methods that take a pointer and a count twice, once as per usual and again as if it were declared to use a span.
  • Use that to integrate with Swift.

To learn more about this:

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] I’m using safe in the sense commonly used by Swift APIs, namely that it’s memory safe.

[2] Well, the Swift side is safe. The C++ side is harder to make safe, although even that’s becoming feasible these days.

@DTS Engineer, Following your suggestion, I tried the span-based approach: C++ functions taking std::span<const uint8_t> / std::span<uint8_t> marked __noescape, called from Swift with data.span / data.mutableSpan (Swift 6.2.3, SafeInteropWrappers). In my use case, the bytes delivered by NWConnection.receiveMessage are passed this way to C++ for processing.

My understanding is:

  • Passing data.span gives C++ a view of the original bytes with no copy, even if the storage is shared. data.mutableSpan copies shared storage first, the same as withUnsafeMutableBytes.
  • In Swift, the span cannot outlive the Data. On the C++ side, however, the callee receives a plain std::span, and __noescape is only a promise by the C++ code that the compiler does not enforce. So if C++ keeps the pointer (in a member, a global, or work queued to another thread) and uses it after the call returns, it may find it dangling.

Since the safety on the C++ side depends entirely on how the C++ code is written, could you share the guidelines for doing this correctly?

  • What are the recommended practices, and the patterns to avoid, for C++ code that receives a span from Swift?
  • If C++ needs the bytes beyond the call, what is the recommended way to handle it?
Written by vishalsehgal in 911090022

My understanding is …

That sounds about right.

Written by vishalsehgal in 911090022

Since the safety on the C++ side depends entirely on how the C++ code is written, could you share the guidelines for doing this correctly?

Not really. The span facility is a standard part of C++; there’s nothing Apple-specific about it. Thus, you can take avantage of of the numerous C++ resources out there: books, websites, LLMs, and so on. I don’t use C++ a lot [1] so I don’t have any concrete recommendations.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] I read a lot of C++, but most of the time I write C++ it’s to exercise Swift / C++ interoperability (-:

@DTS Engineer, One more point to validate. In our POC, when a Data had no other reference than a single variable, passing data.mutableSpan to a C++ function taking std::span<uint8_t> did not create a copy: the address C++ received was the same as the Data's storage, and the write was seen in place.

Is it guaranteed that mutableSpan does not copy when the Data's storage is uniquely referenced, and copies only when the storage is shared?