Inconsistent caseInsensitiveCompare behavior

Tried switching to localizedCaseInsensitiveCompare, though that has similar issues

NSString *a = @"\u1F28\u0399abc"; // ἨΙabc
NSString *b = @"\u1F2E\u0399zzz"; // ἮΙzzz

[a localizedCaseInsensitiveCompare:b] == NSOrderedDescending; // YES
[b localizedCaseInsensitiveCompare:a] == NSOrderedDescending; // YES

NSString *c = @"\u0397\u0313\u0345abc"; // ᾘabc
NSString *d = @"\u1F28\u0399abc";       // ἨΙabc

[c localizedCaseInsensitiveCompare:d] == NSOrderedAscending; // YES
[d localizedCaseInsensitiveCompare:c] == NSOrderedSame;      // YES

Really getting confused here...

Feedback Assistant: FB24975737

Yeah, string ordering is hard in the general case. I think it’s reasonable to file a bug about the behaviour you’re seeing, so thanks for FB24975737. As to what you can do about it right now, I want to ask about this:

This causes inconsistent indexing in a sorted array resulting in obscure crashes of my app.

What are you creating this array for? Is it a temporary thing that only exists in memory and it used, say, to present a sort list of items to the user? Or are you using it for something more persistent? Or something that’s internal, so not being displayed to the user.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

What are you creating this array for? Is it a temporary thing that only exists in memory and it used, say, to present a sort list of items to the user? Or are you using it for something more persistent? Or something that’s internal, so not being displayed to the user.

After sorting, the elements are divided into sections and then displayed to the user. The section membership is unstable now which causes the UI framework to throw.

I think I found a workaround for now in this category on NSString:

- (NSComparisonResult)fb24975737LocalizedCaseInsensitiveCompare:(NSString *)other {
    NSLocale *locale = NSLocale.currentLocale;
    // Fold each entire string first, then collate without case-insensitive options.
    NSString *left = [self stringByFoldingWithOptions:NSCaseInsensitiveSearch locale:locale];
    NSString *right = [other stringByFoldingWithOptions:NSCaseInsensitiveSearch locale:locale];
    return [left compare:right options:0 range:NSMakeRange(0, left.length) locale:locale];
}

With this code I cannot reproduce the inconsistencies anymore. Looks like Foundation's combined case-insensitive comparison is trying to outsmart itself.

I think I'd have the exact same issue if my app was fed strings like this.

Also would be particularly scared of using

- (NSUInteger)indexOfObject:(ObjectType)obj inSortedRange:(NSRange)r options:(NSBinarySearchingOptions)opts usingComparator:(NSComparator) 

Thanks for the background info.

With this code I cannot reproduce the inconsistencies anymore

Cool.

While playing around with this I also noticed that normalising the strings (using -decomposedStringWithCanonicalMapping) also improved things. However, I only did limited testing, so it’s possible that it only works in the specific cases you highlighted.

Still, the normalisation test is useful because, by definition, Foundation’s locale-aware APIs (like -localizedStandardCompare:, which is my go-to tool for this sort of thing) are supposed to be normalisation-insensitive. So if normalisation changes things then that it’s clearly Foundation’s bug to fix.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I tried that while debugging earlier, applying NFC normalization didn't help, NFD looked like another promising workaround (fixed the cases I had at hand). I settled on the folding approach as that felt more robust, given that NFC still failed.

So if normalisation changes things then that it’s clearly Foundation’s bug to fix.

Otherwise not? I mean, a framework that can't reliably compare two strings, how are developers supposed to build upon such a framework?