Best workaround for the lack of generic protocols

Let's say I have a protocol "ValueProvider", which is just something that provides a value:


protocol ValueProvider
{
     typealias ValueType
     func valueInContext(context: ValueProviderContext) -> ValueType
}


I have various structs that implement this protocol, such as:


struct Constant<T>: ValueProvider
{
     var value: T
     init(value: T)
     {
          self.value = value
     }
     func value(context: ValueProviderContext) -> T
     {
          return value
     }
}


and many other such (generic) structs.



I have another struct, Thing, that has properties which are value provides of particular type. Ideally, I would like to express it like this:


struct Thing
{
     var position: ValueProvider<CGPoint>
     var name: ValueProvider<String>
     ...
}



Unfortunately I cannot do this, because ValueProvider is a protocol (which cannot be generic in current version of Swift), and ValueProvider<String> is illegal as a property type.


It looks to me I basically have two options:


(1) Make a new generic ThingProperty<T> enum, use that as the type of the "position" and "name" properties. The enum will have a value provider as its associated value. I don't like this solution because it forces me to change my model only because the type system cannot express what I want, although the runtime shouldn't have any problem with what I want. It feels like changing my model just to make the typechecker happy, which is absurd.


(2) Instead of "ValueProvider<CGPoint>", I can just use "ValueProvider" as the type of the position (and name) properties. This works and allows me to keep the model simple, but I loose the benefits of static typing to a large degree - the system doesn't know that the "position" and "model" properties can only hold CGPoint and String value providers, respectively. As a consequence, when I call valueInContext method on those properties, type checker wouldn't know the correct type of the return value.


What's the "correct" approach to take here? Am I missing something?

Answered by ObjectHub in 18054022

Mike, you do point out a really problematic shortcoming of Swift's current type system. I've written about it in an earlier thread with the title "Workarounds for lack of parameterized protocols?".


The folks from Apple working on the standard library were facing this problem all the time as well. They invented a design pattern as a workaround. I'll explain it using your example but using Apple's naming conventions.


Let's assume you want to define a protocol like ValueProviderType which abstracts over a type. You do this in Swift via an associated type:


protocol ValueProviderType {
  typealias Value
  func valueAtTime(time: Time) -> Value
}


This protocol isn't very useful as a type. You can basically only use it as a type constraint, but not standalone. This limitation can be worked around (with a hack) by defining a struct AnyValueProvider which wraps any implementation of ValueProviderType. This struct hides the concrete implementation of the value provider while providing a usable standalone type. Here's a way to implement AnyValueProvider:


struct AnyValueProvider<T>: ValueProviderType {
  let _valueAtTime: (Time) -> T
  init<V: ValueProviderType where V.Value == T>(_ delegatee: V) {
    _valueAtTime = delegatee.valueAtTime
  }
  func valueAtTime(time: Time) -> T {
    return _valueAtTime(time)
  }
}


I have not found a good general way to implement such wrappers, but the approach above using closures should be good enough in most cases (this is discussed in the thread "Workarounds for lack of parameterized protocols?").


Now with AnyValueProvider, you finally have a useful type which you have to use whenever you want to refer to a variable, a parameter, etc. of a concrete value provider. Here's an adaptation of your code:


typealias Time = Int
typealias Color = String
struct Car {
  var color: AnyValueProvider<Color>? = nil
  var position: AnyValueProvider<Double>? = nil
  init () {}
  func draw(time: Time) { }
}


Your existing value provider implementations don't need to be changed. They can stay:


struct ConstantValue<T>: ValueProviderType {
  var value: T
  init(_ value: T) {
    self.value = value
  }
  func valueAtTime(time: Time) -> T {
    return value
  }
}
struct TimeDependentValue<T>: ValueProviderType {
  var min: T
  var max: T
  init(_ min: T, _ max: T) {
    self.min = min
    self.max = max
  }
  func valueAtTime(time: Time) -> T {
    return min  // just for the example
  }
}


Whenever you need to create an instance of AnyValueProvider, you can simply pass your real value provider implementation to its constructor:


var carA = Car()
carA.position = AnyValueProvider(ConstantValue<Double>(123))
carA.color = AnyValueProvider(ConstantValue<Color>("red"))
var carB = Car()
carB.position = AnyValueProvider(TimeDependentValue<Double>(0, 123))
carB.color = AnyValueProvider(TimeDependentValue<Color>("red", "green"))


Does this explain how this pattern works? If you want more inspiration on how Apple uses this pattern, you can check out the various Any* classes provided by Apple in the standard library.


And if there's anyone who has a good idea how to implement those Any* wrappers, please let me know. I'm still very interested in this.

I ran into the same problem (and I am also looking for a better solution).


What I did as a work around is create separate protocols for each data type. So I have a "StringValueProvider" and a "IntValueProvider". This is annoying, and it only works because I have 4 data types that I need to work with. If you only have a few data types, it may work as a workaround for you as well.

Regarding this ... thing:

struct Thing
{
     var position: ValueProvider<CGPoint>
     var name: ValueProvider<String>
     ...
}


If I understand your requirements correctly, you want eg position to be something of any type T that conforms to the ValueProvider protocol and whose ValueProvider.Value is CGPoint? (there could of course be lots of such types) Will that type T be known at compile time?


If so, should position be able to change into a value of some other type than T at runtime, or will it always stay the same type for a particular instance of Thing? If, for example some Thing is initialized with position being of the type MidPointValueProvider, should it then remain a MidPointValueProvider or could it change into say a LineIntersectionPointValueProvider?


What I mean is, could you use something like this:

struct Thing<P, N where
    P: ValueProvider, P.ValueType == CGPoint,
    N: ValueProvider, N.ValueType == String
    >
{
    var position: P
    var name: N
}

?


I guess this could be a case where it is easy to get confused about what will work dynamically and/or statically, especially if coming from Objective C where people are used to letting the runtime do almost everything (dynamically), wheras in Swift we have a lot more ways to do things statically, which will of course not always be suitable for doing things dynamically and vice versa. And then you should probably search for and watch the talk entitled "Zero-Cost Abstractions" by Airspeed Velocity.

I need to have this dynamic, i.e. what concrete type of value provider is there can change at runtime.


Ideally, I would write ValueProvider<Double> as the type of the property and this will mean that at runtime, only value providers that provide Double values will be assigned to that property. But I do not know which particular value providers would that be, and they may change at runtime (provided they are all Double value providers).

So it needs to be dynamic, yet you want the generic (well, "associated type possessing") ValueProvider protocol to be at the heart of it?


Isn't this something that you should simply be doing dynamically, without using too much compile time / static features of the lnaguage (like eg generics)?


I can't see any point in "faking" compiler knowledge where there can't possibly be any. If the compiler can't possibly know, you'll of course just have to do things dynamically (something that people have always been doing in eg Objective C, even for problems that would have been better solved statically).


So isn't this where you should stop and think about the following words from the notes of the Airspeed Velocity talk I referred to above?


"The answer they get back is usually not very satisfying, because generics belong to the compile time. They’re designed to be used with static types, and mixing static generics with dynamic types at runtime is rather like mixing steak and ice cream. Steak is great. Ice cream is great. Steak with ice cream, well, it’s bit unpleasant."

This is a misunderstanding. Let me explain.


Lets say you have 2 structs, Constant and Variable:


protocol ValueProvider { ... }


struct Constant<T>: ValueProvider { ... }

struct Variable<T>: ValueProvider { ... }


Now, in my Thing struct, I want to have properties like this:


struct Thing

{

var rotation: ValueProvider<Double>

}


I don't know whether the rotation property will be Constant<Double> or Variable<Double>, but I want to be able to express that it will use Double, because I can guarantee that, even at compile time. However, Swift doesn't allow me to use ValueProvider<Double>, because it lacks generic protocols.


In short, I want to express the compile-time knowledge I have statically (that the structs conforming tothe ValueProvider protocol here should deal with Doubles), but I want to leave the exact type of the value provider itself (Constant<Double> or Variable<Double>) unspecified till runtime (because I don't know that beforehand).

That is excactly the problem I currently have with swift. Coming from java I like to use many interfaces when designing my class architecture. In java those can be generic and can be stored in collections and declared as type of a variable.

Espacially as swift promotes "protocol oriented programming" I am very confused what the (practical or theoretical) problem with generic protocols would be, eg why they are not implemented.


Every time I try to port one of my java projects to swift I stumble uppon this issue and for me it really seams like a show stopper as it forces me to change my whole class design WITHOUT any comprehensible reason.

Well changing from the horrible Java-mindset would be first on my list 😉


Generic protocols sound strange. Protocols have no type so how could they be generic type?

What would you suggest as the alternative solution for the posed question? I (and many others) are looking for a canonical, elegant, Swift-like way of solving this problem. Your "Changing from the horrible Java-mindset" suggestion isn't particularly helpful, can you please be more specific?

Would it be possible for you to turn your example scenario into something even simpler but also more concrete and self contained? That way maybe people here will find it easier to suggest more concrete solutions (in the form of a short program).

Yeah my problem exactly. I don't get what the example should accomplish.


It is happaning with a lot of threads since the start of the new forum - People throw some strange non working code in the thread and expects people to understand what it is supposed to do. It's kinda frustrating, we had always such nice and productive talk in the old Swift subforum.

OK, I'll try to explain this better.


Imagine you have a Car struct. The car can draw itself at a particular time.


You have array of cars in your model. Some of those cars should be green. Some of them should be red.

Some of them should move as time passes. Some of them should stay in the same place as time passes.

Some of them should turn from green to red as time passes. Some of them should stay the same color.


struct Car {
     var color: ValueProvider<Color>
     var position: ValueProvider<Double>

     func draw(time: Time) { ... }
}

protocol ValueProvider<T> {
     func valueAtTime(time: Time) -> T
}

struct ConstantValue<T>: ValueProvider<T> {
     var value: T
     init(value: T) {
          self.value = value
     }
     func valueAtTime(time: Time) -> T {
          return value
     }
}

struct TimeDependentValue<T>: ValueProvider<T> {
     var min: T
     var max: T

     init(min: T, max: T) {
          self.min = min
          self.max = max
     }

     func valueAtTime(time: Time) -> T {
          return interpolatedValue(min, max, time)
     }
}

Then, you can create clocks like this:


var carA = Car()
carA.position = ConstantValue<Double>(123)
carA.color = ConstantValue<Color>(red)

var carB = Car()
carB.position = TimeDependentValue<Double>(0, 123)
carB.color = TimeDependentValue<Color>(red, green)


When I put these two cars in an array and iterate the array

periodically, drawing the cars, carA will always have the same color

and won't move. CarB, on the other hand, would change its color from red to green

and move from position 0 to position 123.


Crucially, I need to be able to do this at runtime:


carA.position = TimeDependentValue<Double>(23, 45)


Here, I changed carA's position from a constant value to an animated, time dependent value.


However, due to the lack of generic protocols, I cannot do this:


struct Car {
     var color: ValueProvider<Color>
}


(ValueProvider is a protocol).


That means that I simply have no way of expressing that the color property of the car should be any ValueProvider, but that ValueProvider also has to provide Colors.


I hope this makes more sense now.

Sadly not really. What I understand is that you want to fight the type system with some strange dynamic type magic 😕


Protocols are there to define and add behaviour - not type system magic.

I think the replies you're getting are a little bit harsh, but there are some larger issues to think about:


1. At least in your original example, you wanted protocols that differed only in the type on which they were based. In spite of recent enhancements, protocols are really about behavior, and it's at best questionable whether your various ValueProviders have different behavior when they just operate on different types. In a way, it's exactly the point of Swift protocols that the type represented by your ValueType typealias isn't part of the ValueProvider behavior.


2. There's an issue about the difference between generics (as in Swift), versus templates (as in C++). I think this has been discussed in the forums several times in the past. In a way, you're asking for a ValueProvider template, not a ValueProvider protocol.


3. As well as trying to define conformance, in your scenario you are also trying to implement something like inheritance for ValueProvider. It may be that the correct solution is to make your value provider implementations be classes, not structs, and to use actual inheritance.

Please read the code again. The code I wrote doesn't express anything that couldn't (in principle) be determined at compile time.


Heck, I can even express this in Swift itself if I just use classes instead of structs and use a generic superclass instead of the generic protocol!

But I want value semantics.


Please note that C++, C# and Java all support generic protocols / interfaces. This can also be expressed in Haskell type system. I, jonhull, laszlokorte and many other people are searching for a nice alternative to this in Swift.

Thanks Quincey :-)


(1) Ok, I can understand this angle of reasoning. Jonhull's suggestion of declaring separate ValueProvideDouble, ValueProviderInt, ValueProviderString protocols would then be the most Swift-like solution. Hopefully we can all agree that declaring a bunch of protocols like that looks like code smell and a potential candidate for language-level solution or deeper refactoring of the code.


(2) This might be the case, although I do not have experience with C++ templates. However, as I wrote above, C# and Java have equivalents of generic protocols/interfaces.


(3) As you correctly pointed out, this can be done in Swift using classes. Instead of the generic protocol, I can use a generic superclass (which Swift supports) and it will work just as I want it to.


Unfortunately, I also need value semantics, which is why I'm trying to do this with structs.


Also, there is a subtle issue - I don't really need the implementation inheritance. The superclass would have empty implementation, which is usually a pretty clear indication that you should use protocol instead of superclass.

I tried playing with several approaches in a playground, and didn't come up with anything that works. You can certainly define a generic method in the ValueProvider protocol, but in the implementation, getting other struct members to share the generic type seemed insoluble ("can't convert value of type T to T").


One thing I didn't raise before is that if you have a typealias in the ValueProvider definition, you can (currently?) use the protocol only as a constraint, not as a type. This is going to get in your way anyway, generics aside.


It seems weird that you want value semantics for the value providers. Under what circumstances do you assign an existing (that is, not just explicitly created) value provider to another variable, where your intent is to make a copy? Other than that, how much do the semantics matter? Your use case seems like a gray area where neither value nor reference semantics does exactly what you want.


Finally, it's not clear at the moment whether Swift's current protocol conceptualization is intended to be mid-way between other language offerings, or is mid-course in a longer journey to something else. I wouldn't like to set my programming patterns in stone until I know the answer to that. Whatever the answer will be, for now all you can do is file your bug report showing your use cases, and live with what Swift allows, until it changes.

The important thing here is: Swift is NOT C++, C#, Java or Haskel - It is Swift, so solutions and patterns will be different.


Why are so many people trying to make Swift like other languages instead of using it as it is?

Your problem might be that value semantics, protocols and generics are not what you seem to think they are (judging by the way you use the terms and programming constructs). And this is not because Swift tries to put some dramatically different meaning into these notions compared to other languages, because it doesn't, not that different anyway.


First of all, there are "generic protocols" in Swift, it's just protocols with associated types. Just like your ValueProvider protocol.


So your problem is not the "lack of generic protocols" but rather this (error message): "Protocol 'ValueProvider' can only be used as a generic constraint because it has Self or associated type requirements"


If your ValueProvider protocol hadn't had its ValueType typealias, ie if it hadn't been generic, then, as you already know, position and color would have been possible to declare as ValueProviders (but that would not solve your problem in an elegant way).


Secondly, what you really are trying to achieve in your latest more concrete example scenario, if I understand you correctly, is that you want a bunch of cars that can change position and color in a way that should be controllable at runtime.


Probably you also want this to be implemented via your ValueProvider protocol, although I'm still not at all sure if this is a requirement or not, and I see nothing in the scenario that would make this the obvious way to structure the code. This is what I meant by self-contained scenario. There is nothing in your example scenario that lets us understand whether or why it is important or not to use eg ValueProviders and / or value semantics or any other implementation detail.

If someone here wrote a small and elegant Swift program that presented you with a bunch of cars that could move and change color, and their motion and color changing could be controlled at runtime. Would that be sufficient to solve your underlying problem? Even if the solution/program didn't include ValueProviders and didn't use (enough or any) value semantics?

And if you really do need value semantics, then I guess you will have to be a bit more specific as to eg why and in what parts of the scenario this is important. But it would of course be better to devise the scenario in such a way as to make such requirements implicit / self evident.

Accepted Answer

Mike, you do point out a really problematic shortcoming of Swift's current type system. I've written about it in an earlier thread with the title "Workarounds for lack of parameterized protocols?".


The folks from Apple working on the standard library were facing this problem all the time as well. They invented a design pattern as a workaround. I'll explain it using your example but using Apple's naming conventions.


Let's assume you want to define a protocol like ValueProviderType which abstracts over a type. You do this in Swift via an associated type:


protocol ValueProviderType {
  typealias Value
  func valueAtTime(time: Time) -> Value
}


This protocol isn't very useful as a type. You can basically only use it as a type constraint, but not standalone. This limitation can be worked around (with a hack) by defining a struct AnyValueProvider which wraps any implementation of ValueProviderType. This struct hides the concrete implementation of the value provider while providing a usable standalone type. Here's a way to implement AnyValueProvider:


struct AnyValueProvider<T>: ValueProviderType {
  let _valueAtTime: (Time) -> T
  init<V: ValueProviderType where V.Value == T>(_ delegatee: V) {
    _valueAtTime = delegatee.valueAtTime
  }
  func valueAtTime(time: Time) -> T {
    return _valueAtTime(time)
  }
}


I have not found a good general way to implement such wrappers, but the approach above using closures should be good enough in most cases (this is discussed in the thread "Workarounds for lack of parameterized protocols?").


Now with AnyValueProvider, you finally have a useful type which you have to use whenever you want to refer to a variable, a parameter, etc. of a concrete value provider. Here's an adaptation of your code:


typealias Time = Int
typealias Color = String
struct Car {
  var color: AnyValueProvider<Color>? = nil
  var position: AnyValueProvider<Double>? = nil
  init () {}
  func draw(time: Time) { }
}


Your existing value provider implementations don't need to be changed. They can stay:


struct ConstantValue<T>: ValueProviderType {
  var value: T
  init(_ value: T) {
    self.value = value
  }
  func valueAtTime(time: Time) -> T {
    return value
  }
}
struct TimeDependentValue<T>: ValueProviderType {
  var min: T
  var max: T
  init(_ min: T, _ max: T) {
    self.min = min
    self.max = max
  }
  func valueAtTime(time: Time) -> T {
    return min  // just for the example
  }
}


Whenever you need to create an instance of AnyValueProvider, you can simply pass your real value provider implementation to its constructor:


var carA = Car()
carA.position = AnyValueProvider(ConstantValue<Double>(123))
carA.color = AnyValueProvider(ConstantValue<Color>("red"))
var carB = Car()
carB.position = AnyValueProvider(TimeDependentValue<Double>(0, 123))
carB.color = AnyValueProvider(TimeDependentValue<Color>("red", "green"))


Does this explain how this pattern works? If you want more inspiration on how Apple uses this pattern, you can check out the various Any* classes provided by Apple in the standard library.


And if there's anyone who has a good idea how to implement those Any* wrappers, please let me know. I'm still very interested in this.

Thanks ObjectHub, finally an answer that sheds more light on this :-)

Jens, I'm not a beginner intentionally trying to find an obscure way to make cars move on screen :-)


The examples above are just extreme oversimplifications of what we're really doing.


We have a giant (hundreds of thousands lines of code) app written in Objective-C with a very complex model. We're doing a preliminary "research" in the direction of re-making the model in Swift, by making it immutable (or even better, use value semantics) for various reasons (concurrency, robust undo support, document history trees, etc.).


What I described above is semantically what we really need to do. You'll have to trust me on this; if I were to explain in detail why exactly is that what we need, I would need to write a book. It would be of no use to anybody.


With Objective-C, it is very inconvenient to make large, complex immutable models. The language wasn't really cut out to do that, so our current model is mutable. With Swift's value semantics, what we wanted to do for several years now can finally be done in relatively pain-free way.


If we do this with classes instead of structs and enums, we have no reason to use Swift at all - all the benefits we're looking for would vanish.

I don't think you're asking for anything 'magical' or 'against the Swift way', so some of the response you've gotten is somewhat unfortunate. I'm glad you got a serious answer though.


I think part of the issue is the problem framing. You're asking for protocol generics, but in the context of Swift, I think it would be more accurate to describe what you want as protocol type constraints. To paint a picture:


protocol<ValueProvider where ValueType == Double> // fully explicit
ValueProvider<ValueType=Double> // shorthand


So, basically, you want to create an anonymous protocol for all implementers of ValueProvider that meet an additional constraint on the associated type. To contrast this with what protocol generics might mean, consider:


struct ValueProviderType: ValueProvider<Int>, ValueProvider<String> { ... }


By having a type parameter, you make it so that for each possible argument to the type parameter you have a distinct protocol. You could then implement more than one of these resulting protocols for a given type. Rust is an example of a language that has both these features (constrained protocol types and generic protocols), and its trait (i.e. protocol) system can be fairly complicated as a result—which isn't necessarily a bad thing, but it's a definite tradeoff, especially since it makes the type system turing complete.


Java doesn't let you do what I suggested above with generic interfaces, of course, but Java basically uses interface type parameters as associated types. It just happens to require you to always constrain them to a specific type—even if that's another type parameter introduced solely for that purpose—and uses type parameter syntax to do this. This makes it essentially equivalent to what my "protocol type constraints" above would allow you to do in Swift.

Thank you Steve, your reply is much appreciated. I'm really happy that finally, some replies that understand the true nature of the problem/question are starting to appear. What you suggest as protocol type constraints would indeed perfectly solve my problem. So would generic protocols, as you outlined.


Is any of this planned for the future? Does anybody know a timeframe estimate? Is jonhull's suggestion the best temporary workaround?

Thank you very much. That seems like a usable workaround.

Having such a general mechanically applyable solution tells me that there is no underlying semantical problem with what MikeA, myself and others want to achieve but just a syntactical.

We would just need a syntax for describing anonymous protocols as SteveMcQwark describes in his answer below.

Resulting in (for example)

var color : ValueProviderType<where Value=Double> = MyValueProvider(5.0)

being "expanded" into

struct AnonymAnyValueProvider<T>: ValueProviderType { /* automatically generated delegation */ }
var color : AnonymAnyValueProvider<Double> = AnonymAnyValueProvider(MyValueProvider(5.0));


Since many people seem having trouble with this issue and as you say even the stdlib suffers from it, I am wondering why this is not what is done by swift.

I know this is an old thread … but i am interested in the latest thinking on this.

Best workaround for the lack of generic protocols
 
 
Q