Workarounds for lack of parameterized protocols?

I'm constantly running into problems with protocols that have type members (i.e. associated types of protocols). One observation I made while finding workarounds, is that the lack of type parameterization forces me to expose implementation details that I do not want to expose. I find this highly annoying, especially given that one of the use cases of protocols, is to abstract over the concrete implementation of data types. I am looking for some advice on best practices when it comes to using such "generic protocols."


Here is a hypothetical example outlining the problem. Assume we want to define a protocol for a one place buffer. I believe this is what a typical Swift 2 programmer would write:


protocol BufferType {
  typealias Element
  func getValue() -> Element?
  mutating func setValue(value: Element)
}


Here is a simple generic implementation of this procotol:


struct OnePlaceBuffer<T>: BufferType {
  var value: T? = nil
  func getValue() -> T? {
    return value
  }
  mutating func setValue(value: T) {
    self.value = value
  }
}


With this implementation, it's straightforward to create instances of type BufferType:


var buffer = OnePlaceBuffer<Int>()


Now let's assume we would like to implement a simple wrapper that implements BufferType and forwards all methods to another object implementing the 'BufferType' protocol. This was my first attempt:


struct Buffer<T>: BufferType {
  private var delegatee: BufferType
  func getValue() -> T? {
    return delegatee.getValue()
  }
  mutating func setValue(value: T) {
    delegatee.setValue(value)
  }
}


This is, unfortunately, not legal Swift, because 'BufferType' is a protocol with an associated type requirement. Those protocols can't be used as property types (like in line 2). So, this clearly won't work.


I finally managed to implement a buffer wrapper by exposing the type of the delegatee as a type parameter of the wrapper. Here's the code:


struct Buffer2<T, B: BufferType where B.Element == T>: BufferType {
  private var delegatee: B
  func getValue() -> T? {
    return delegatee.getValue()
  }
  mutating func setValue(value: T) {
    delegatee.setValue(value)
  }
}


This is obviously very ugly, because now, the wrapper isn't hiding the type of the delegatee object anymore. Furthermore, my buffer wrapper has two type parameters making it even more difficult to handle. I eventually found a workaround by using closures (i.e. not wrapping another object, but just storing references to the other objects methods):

struct Buffer3<T>: BufferType {
  private var _getValue: () -> T?
  private var _setValue: (T) -> Void
  init(getValue: () -> T?, setValue: (T) -> Void) {
    self._getValue = getValue
    self._setValue = setValue
  }
  func getValue() -> T? {
    return _getValue()
  }
  func setValue(value: T) {
    _setValue(value)
  }
}


For creating a wrapper, a programmer would now have to write code like this:


var wrapper = Buffer3(getValue: buffer.getValue, setValue: buffer.setValue)


Interestingly, this does not compile. For reasons that are unclear to me, the Swift compiler tells me that "partial application of 'mutating' method is not allowed". This is referring to 'buffer.setValue'. Obviously, there's a trivial workaround by wrapping the whole thing in another closure:


var wrapper = Buffer3(getValue: buffer.getValue, setValue: { v in buffer.setValue(v) })


I find this solution totally messy. Is there really nothing better?


A second example (I'm not going into detail this time) is related to functions returning objects of such generic protocols. For instance, assume we would want to model a Set type in Swift (you won't find one in the standard library for good reasons). My first attempt at this looked like this:


protocol SetType {
  typealias Element
  func intersect<S: SetType where S.Element == Element>(other: S) -> SetType
}


Again, the compiler complained about me using 'SetType' in a place where it wasn't allowed (the result type of 'intersect'). So, I rewrote the whole thing like this:


protocol SetType2 {
  typealias Element
  func intersect<S: SetType, R: SetType where S.Element == Element, R.Element == Element>(other: S) -> R
}


This does compile, but I have not found any way to implement this protocol. This is because type parameter 'R' comes in from the client, but I would like to decide in the implemention of 'intersect' what 'SetType' implementation to return. This is where I am stuck now. Any ideas? Any advice? Am I really forced to define a concrete (non protocol) return type whenever I'm dealing with protocols that have associated type requirements?


== Matthias

ObjectHub said:

var wrapper = Buffer3(getValue: buffer.getValue, setValue: { v in buffer.setValue(v) })

I find this solution totally messy. Is there really nothing better?


How about this:

protocol OnePlaceBufferType { // I've slightly altered the names (this was BufferType).
    typealias Element
    func getValue() -> Element?
    mutating func setValue(value: Element)
}

struct SomeOnePlaceBuffer<T>: OnePlaceBufferType { // (was OnePlaceBuffer).
    var value: T? = nil
    func getValue() -> T? { return value }
    mutating func setValue(value: T) { self.value = value }
}

struct BufferWrapper<T, U: OnePlaceBufferType where U.Element == T>: OnePlaceBufferType { // (was Buffer)
    private var wrappedBuffer: U
    init(_ wrappedBuffer: U) { self.wrappedBuffer = wrappedBuffer } // <-- This is the only thing I really did, it will enable compiler to infer U.
    func getValue() -> T? { return wrappedBuffer.getValue() }
    mutating func setValue(value: T) { wrappedBuffer.setValue(value) }
}

var buffer = SomeOnePlaceBuffer<Int>()
var wrapper = BufferWrapper(buffer) // <-- You can now simply do this.


As for your second example, that about modeling the Set type, I guess you would have to decide whether you want:

protocol SetType {
    typealias Element
    func intersect<S: SetType where S.Element == Element>(other: S) -> Self
}

or

protocol SetType {
    typealias Element
    func intersect<S: SetType where S.Element == Element>(other: S) -> S
}


or if you want to create and return some base class or some struct conforming to SetType.


Think about what it would mean If that intersect method actually could return the SetType protocol itself (as you first tried/wanted) then what should be returned, should it just be some random diffuse "entity" that conforms to SetType? How should that be constructed?


You could perhaps find these related threads interesting:

https://forums.developer.apple.com/thread/4741

https://forums.developer.apple.com/thread/6901

Thanks for your proposals, Jens!


If I see this right, your 'BufferWrapper' struct is a refinement of my 'Buffer2' struct. The problem I have with a solution that requires a second type parameter is that 1) it exposes the delegatee implementation type, and 2) introduces type incompatibilities between wrappers for delegatees with different implementation types. For instance, assume you want to do this:


var buffer1 = SomeOnePlaceBuffer<Int>()
var buffer2 = AnotherOnePlaceBuffer<Int>()
var wrapper1 = BufferWrapper(buffer1)       // wrapper1: BufferWrapper<Int, SomeOnePLaceBuffer<Int>>
var wrapper2 = BufferWrapper(buffer2)       // wrapper2: BufferWrapper<Int, AnotherOnePlaceBuffer<Int>>
var several = [wrapper1, wrapper2]          // won't compile because this is a heterogeneous array


This simply doesn't work with this approach. The implementation type of the delegatee needs to be hidden so that it will be possible to stuff wrappers for buffer1 and buffer2 into an array.


Your other proposal for my SetType challenge which uses Self types looks interesting. I didn't consider this yet. I'll have a look if that works for me. I will let you know if I run into problems. Thanks!

The reasons for why the following does not work:

var several: [OnePlaceBufferType] = [wrapper1, wrapper2]          // won't compile because this is a heterogeneous array

but instead results in the compile time error:

Protocol 'OnePlaceBufferType' can only be used as a generic constraint because it has Self or associated type requirements


is explained in one of the threads I mentioned above, ie this:

https://forums.developer.apple.com/thread/4741


you'll also find some hints on suggested workarounds there.


Also see this:

https://forums.developer.apple.com/thread/7185

The solution here is probably something like the solution for storing heterogeneous SequenceTypes, i.e. wrap them all in an AnySequence to hide the underlying type.

Now that you mention 'AnySequence', I am wondering how it's actually implemented? 'AnySequence' is really just a wrapper and all the implementation challenges discussed in this thread apply to it fully.


I saw folks from Apple on this forum. Maybe someone can enlighten us how 'AnySequence' is implemented?


Thanks,

Matthias

Thanks for the pointer to thread https://forums.developer.apple.com/thread/4741 with the subject "Can you create a Set of a single protocol?". This is indeed another symptom of exactly the same problem. Seeing all these discussions, I don't understand why Swift is being promoted as a protocol-oriented programming language. It seems that protocols are so limited as generic abstraction mechanisms so that for interesting use cases, concrete class and struct types need to be used.


== Matthias

I've puzzled over the AnySequence implementation for awhile, and I think I've reverse engineered how Apple does it. There is also, I believe, a much simpler way to implement it. First, it's pretty easy to create a type-eraser with closures. This is for Sequences, but it applies to many cases:


struct AnySequence<Element> : SequenceType {
  let generator: () -> AnyGenerator<Element>
  init<S : SequenceType where S.Generator.Element == Element>(_ base: S) {
     generator = { anyGenerator(base.generate()) }
  }
  init<G : GeneratorType where G.Element == Element>(_ makeUnderlyingGenerator: () -> G) {
     generator = { anyGenerator(makeUnderlyingGenerator()) }
  }
  func generate() -> AnyGenerator<Element> {
     return generator()
  }
}


Just include a closure property for each method you care about. (Swift doesn't support a closure property providing a protocol method, so you can't just name "generator" as "generate" and skip the duplication.)


But from my looking at AnySequence, this isn't how Apple implements it. I believe this is how it's written (I've included a "My" prefix to make it easy to compare debugger output to demonstrate that it has the same structure as stdlib's):


class _MyAnySequenceBox {
  func generate() -> AnyObject {
     fatalError()
  }
}
final class _MySequenceBox<Seq: SequenceType>: _MyAnySequenceBox {
  let _base: Seq
  init(_ base: Seq) { _base = base }
  override func generate() -> AnyObject {
     return anyGenerator(_base.generate())
  }
}

struct _MyClosureBasedSequence<G: GeneratorType>: SequenceType {
  let _makeUnderlyingGenerator: () -> G
  init(_ makeUnderlyingGenerator: () -> G) {
     _makeUnderlyingGenerator = makeUnderlyingGenerator
  }
  func generate() -> G {
     return _makeUnderlyingGenerator()
  }
}

struct MyAnySequence<Element> : SequenceType {
  let _box: _MyAnySequenceBox

  init<S : SequenceType where S.Generator.Element == Element>(_ base: S) {
     _box = _MySequenceBox(base)
  }
  init<G : GeneratorType where G.Element == Element>(_ makeUnderlyingGenerator: () -> G) {
     _box = _MySequenceBox(_MyClosureBasedSequence(makeUnderlyingGenerator))
  }

  func generate() -> AnyGenerator<Element> {
     return _box.generate() as! AnyGenerator<Element>
  }
}


This probably takes less memory than the equivalent closure-basd solution (especially if there are a larger number of methods). AnySequenceBox could have been a protocol, but that would (I believe) make AnySequence much larger in memory. This suggests that memory usage was a major concern in the design?


I've gone into a bit more detail on using type erasure with protocol associated types at http://robnapier.net/erasure

Rob, thanks a lot for reverse-engineering Apple's `AnySequence` implementation. This is very insightful. :-) It's basically combining Jens' idea of making the delegatee type a class parameter with an additional wrapper (around the box) that hides this extra parameter. And since there's no way to define a field of the parametric box type, the untyped superclass is used and the result is down-cast in `generate`. What a hack! Even this workaround is bypassing the type system.


If this is really what the code for the Any* wrappers looks like, then I hope the folks at Apple realized that there's something fundamentally wrong with the current limitations they have put in place for protocols.


BTW, you can write your own `AnySequence` implementation slightly shorter avoiding the repetition of the `anyGenerator(...)` call:


struct AnySequence<Element> : SequenceType {
  let generator: () -> AnyGenerator<Element>
  init<G : GeneratorType where G.Element == Element>(_ makeUnderlyingGenerator: () -> G) {
    generator = { anyGenerator(makeUnderlyingGenerator()) }
  }
  init<S : SequenceType where S.Generator.Element == Element>(_ base: S) {
    self.init(base.generate)
  }
  func generate() -> AnyGenerator<Element> {
    return generator()
  }
}

@Rob, Really good insite into how Apple do this - thanks.


I just wish you could have generic protocols that were types - life would be so much easier.

@Rob, Are still planning on publishing your part 2 blog re. erasure? Hope so!

Hi Rob Napier,

Regarding the Animal-GrassEater-example in your blog post. I think the most obvious solution is something like this:

// An Animal can eat
protocol Animal {
    typealias Food
    func feed(food: Food)
}

// Kinds of Food
struct Grass {}
struct Worm {}

struct Cow: Animal {
    func feed(food: Grass) { print("moo") }
}

struct Goat: Animal {
    func feed(food: Grass) { print("bah") }
}

struct Bird: Animal {
    func feed(food: Worm) { print("chirp") }
}

// --- solution begin ---
protocol GrassEater { func feed(food: Grass) }
extension Cow : GrassEater {}
extension Goat : GrassEater {}
let grassEaters: [GrassEater] = [Cow(), Goat()]
// --- solution end ---

for animal in grassEaters { animal.feed(Grass()) }

How would you do an AnyEnum?


This crashes the compliler:


protocol Animal: RawRepresentable {}

extension Animal {
    typealias RawValue = String
    var value: String { return rawValue }
}

enum LandAnimal: String, Animal {
    case Dog = "dog"
}

enum SeaAnimal: String, Animal {
    case Fish = "fish"
}

class PrintAnimal {
    static func printName<T: Animal>(animal: T) {
        let name: String = animal.rawValue
        print("\(name)")
    }
}
Workarounds for lack of parameterized protocols?
 
 
Q