How to avoid Data Races in deinit

Analysing my code with the new exciting Thread Sanitizer tool from Xcode 8, in certain scenarios I'm experiencing unexpected data races in `deinit` of classes which protect the access to their properties with a synchronization primitive (a dispatch queue) .


Given a class, with a "sync queue" and a synchronized member value:


enum MyEnum {
    case first(_: String)
    case second(_: [String])
}
public class Foo {

    private let _syncQueue = DispatchQueue(label: "foo-sync-queue", attributes: .serial)
    private var _value: MyEnum = .first("")

    public init() {   
    }

    deinit {
    }

    public func modify(value: String) {
        _syncQueue.async {
            self._value = .first(value)
        }
    }
}


What's not immediately obvious is the fact, that the deinit method must access the enum in order to figure out how to deallocate the underlying value.


Suppose, method `modify` will be executed on Thread A. This causes the actual write operation to be executed on a thread, say M, which is synchronized with the `syncQueue`.


It may happen that Thread A equals Thread M, but that is random.


Now, suppose a reference to variable of type `Foo` exists in Thread B. It happens that this is the last strong reference to the variable. Now, when this variable ceases to exist, the `deinit` method of the variable will be called. This deinit reads from the memory store for member `_value`. This read operation is not synchronized with the sync queue. Since the executing thread happens to be not equal the thread M, where the previous write operation has been performed, a data race occurs.


This data race is not easy to fix. I dare to state, it is even impossible in certain use cases. For example:


Suppose there is a function which returns a new value of type Foo:


func createFoo() -> Foo



And then:


createFoo().modify(value: "Hello Data Race")



The problem here is, that the live-time of the returned temporary is largely uncontrollable by the programmer. This prevents the programmer to determine _when_ the variable foo should be deallocated. And to be honest, a programmer shouldn't care about these details anyway!

In practice, this creates a very reliable data race.


This use case is not artificial in the functional paradigm. Actually, I'm having this issue, and I have no idea how to fix it - except there would be help with a corresponding language construct (e.g. let us implement the deinit function) - which does not currently exists.


Any ideas are appreciated.

Replies

Can you provide a short code sample that reproduces the Thread Sanitizer warning?


I understand your analysis of the data race, but the deinit function is special in the sense that it's only called when a single reference to the object exist. In this case, I'm not sure if it's really a data race, because the object should be synchronized (all writes need to be finished) before the other references are released, as part of swift_release.

Can't the deinit also use _syncQueue to access the value?

Thanks for the response. I fear, it's not that easy to provide a short sample that demonstrates the issue - but I'm struggling to find a suitable setup. I will send a bug report when I find a simple example where the issue can be reproduced.


But honestly, I still would like to have a definite answer whether the compiler generated deinitialization code is (should be) thread-safe, that is, does not cause data races when accessing (destroying) the member variables which have been modified within an arbitrary thread during the life-time of the object.


Currently, I'm using the unit tests (>150) in a library for analysing the problem. And while Thread Sanitizer may quite reliable find a data race at a certain test when running all tests, it does not (always) find it when running the single test. It seems, it depends on side effects, like the number of active threads etc. (Here, we see that this tool is a run time analyser - aka Sanitizer, not a static analyser).


The library code is also heavily asynchronous and may use several threads (dispatch queues). It may happen, that a new unit test will be startet on the main thread, while there is still some other thread (queue) alive from the previous test, for example, library code executing after executing an `expectation1.fulfill()` which is called in a client provided completion handler. The "epilog" of the completion handler may still exeute on a background thread, while the main thread continues and the test finishes and a next one is startet. I do not consider this a bug in the unit test setup - the library should deal with this situation gracefully - but it is clear, that executing code from different tests may overlap.


Thread Sanitizer detectes quite a number of issues. But, regarding my own code, I'm quite confident that it is thread-safe (perhpas with the exception of `deinit` - which is still unclear). For example, Thread Sanitizer also detects a data race for a block object in `block_destroy_helper`, allocated for a dispatch source timer, as a result of calling `cancel` for the dispatch source when called within the closure. That is, releasing a closure leads to a data race. This is worrying.


> I understand your analysis of the data race, but the deinit function is special in the sense that it's only called when a single reference to the object exist. In this case, I'm not sure if it's really a data race, because the object should be synchronized (all writes need to be finished) before the other references are released, as part of swift_release.


Yes, deinit will be called only when there is only one reference, and when it is about to be deallocated. However, there are many flavors of memory barriers, and when there are not the correct ones, a data race may occur. Notice also, that it's not `deinit`, but rather the `deinitialization` code generated by the compiler where the data race occurs and which is in turn called after deinit. In constrast, from the perspective of the compiler vendor, it should be clear and clearly specified, whether the deinitialization code should be free from data races that may occur in any circumstances, or whether it's the responsibility of the programmer to insert synchronization primitives (but how?).


Since the deinitialization is performed by compiler generated code, how should a programmer add synchronization - if we had to. Do I? In order to fix this data race, one could think to use an optional instead a value, and just assign it `nil` wrapped into syncchronization promitives within `deinit`, and we should be fine!? No. The deinitialization code still needs to access the optional and would cause the same data race.


So, unless `deinit` does not require any synchronization, and what we see is simply a bug in the compiler generated deinitialization code, the only approach that may work is to use a subclasso of `ManagedBuffer` where the former member variables become the Elemen(s)t of the MangedBuffer. The deinit then has to explicitly destroy the elements, which we then could make synchronized.

Yes it can, but it would be futile.


For example, since the compiler generated deinitialization code causes a data race when accessing an enum, we might think: OK, lets wrap it into a class, and let Foo use an optional. In `deinit` we use our syncQueue and assign `nil` to the wrapped enum. Fine? No! The compiler generated deinitialization code would still need to access the optional (in order to figure out how to destroy it) - and without synchronization this is a data race.

Did you figure out a way to fix this issue?