BERT-SQuAD: Question Answering not working anymore on iOS15.4

Good Morning,

I'm not sure whether I am alone but the BERTSQUAD seems to not work anymore (https://developer.apple.com/machine-learning/models/#text) since iOS 15.4 update.

I tried different configurations and the basic example model and it does not work at all.

Do you have also the issue? If yes, is there a workaround to make it work with the iOS update?

Thank you in advance for your help

Replies

Hi Alexis, is the model crashing while running, or is it running error free but giving incorrect answers?

I'm seeing a similar problem. The app just says "Searching" but never returns an answer. There's a log printed which could be a clue:

2022-07-03 08:42:45.859844-0500 FindingAnswers[32757:1495990] [default] failed to map shmem for window 0

Updated to add: the sample code is working, just unbelievably slowly.

It turns out the culprit is the use of a [Substring: Int] dictionary in loadVocabulary. For some reason, constructing a dictionary with Substring keys takes many orders of magnitude longer than using String keys.

Switching the implementation to [String: Int] solves the problem. When I have time, I'll file a bug for slow performance with Dictionaries using Substrings as keys.

Thanks for the feedback.

After checking the loadVocabulary() method, the Dictionary(uniqueKeysWithValues: zip(words, values) is never initialized and the thread runs forever.

The 2 arguments -> word: [String.Subsequence] and value: Range<Int> seems correct after printing them. Thus, the issue might come from the Dictionary(uniqueKeysWithValues: ) initialization.

I'll give a try to see how long it takes to build the dictionary.

Alexis

It looks like Apple fixed the bug in Dictionary(uniqueKeysWithValues: ) at some point during this beta cycle. If you try now, the sample code should be working again!