DateFormatter, fixed-format dates and iOS Simulator

I’ve been trying to debug a seemingly random issue that involves dates, timezones, formatters and locales. I wanted to make sure I got my basics right.

This I a test that passes “locally” as well as on a second computer but failed on a third. It should come down to this code once I strip away all the noise.

    func testExample() throws {
        //formatter to create date from string
        let DateFormatterSecondsFromGmT = DateFormatter()
        //notice how the order is different to the other formatter
        DateFormatterSecondsFromGmT.dateFormat = "dd/MM/yyyy HH:mm" //format first
        DateFormatterSecondsFromGmT.timeZone = TimeZone(secondsFromGMT: 0)
        DateFormatterSecondsFromGmT.locale = Locale(identifier: "en_GB") //locale second

        let string = "08/03/2023 12:10"
        //date the test failed (approx.): 21 Apr 2023, 11:20:32
        let date = DateFormatterSecondsFromGmT.date(from: string) ?? Date()

        //formatter to create string from date
        let DateFormatterUTC = DateFormatter()
        DateFormatterUTC.timeZone = TimeZone(identifier: "UTC")
        DateFormatterUTC.locale = Locale(identifier: "en_GB")
        DateFormatterUTC.dateFormat = "HH:mm"


        let actual = DateFormatterUTC.string(from: date) //On the computer that failed, this date was at 17:10 (i.e. off by 5 hours)
        let expected = "12:10"

        //One case where actual was 17:10 and expected 12:10.
        XCTAssertEqual(actual, expected)
    }

According to the Technical Q&A QA1480 NSDateFormatter and Internet Dates, when using a fixed-format date "you should first set the locale of the date formatter to something appropriate for your fixed format. In most cases the best locale to choose is 'en_US_POSIX'".

Based on the TN, "On iOS, the user can override the default AM/PM versus 24-hour time setting, which causes NSDateFormatter to rewrite the format string you set". Emphasis mine.

AFAIK the DateFormatter does not log a warning so I am not sure exactly what is the underlying cause for the test to fail.

Also, It’s not immediately obvious to me looking at the code what could be going wrong, given that both formatters have a fixed locale (and for what is worth timezone) which leads me to believe the DateFormatter did not rewrite the format string set.

On that note. I understand that setting the TimeZone to the DateFormatter will take that into account when formatting a Date to a String. Does setting the Timezone also play a role when converting a String to a Date? If so, how?

Ideally, if possible, would be good for me to understand how the test ended up failing. In other words, how can I reproduce the failure which should then give me a clue into what is wrong and coming up with a solution to the root cause.

Accepted Reply

The short answer here is that there is no safe way to use DateFormatter with a fixed dateFormat string without pinning the locale to en_US_POSIX. You can spend a bunch of time trying to understand the intricacies of how DateFormatter behaves but what does that buy you? The are two possible conclusions:

  • You are able to find a specific set of parameters that seem to work today.

  • You aren’t.

The second conclusion isn’t a problem. You’ll go straight to en_US_POSIX and all will be well [1].

The problem is with the first conclusion. You might be tempted to use those parameters in production code but, if you do that, you’re exposing yourself to future binary compatibility problems. For example, while the en_GB locale currently implies the Gregorian calendar, that could change in the future [2]. If that happens, your code will start reporting dates that are off by roughly 500 years. And while this is not a very realistic possibility, weird stuff like this happens all the time [3] and the only way to protect yourself is to use en_US_POSIX.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Or one of its alternatives, like NSISO8601DateFormatter or the Swift VerbatimFormatStyle.

[2] In a post-Brexit world, anything is possible, right?

[3] A more realistic example would be in the other direction, date rendering, where an Arabic-speaking country might change from western to eastern Arabic numerals, or vice versa.

  • Thank you eskimo! Much love.

    Personally, my main motivation in understanding why it fails is to be able to confidently say, here is the fix. Having said that, I hear what you are saying. When using fix-format date, you really really should use en_US_POSIX. I'll happily take that. The wording of the TN isn't as strong/clear which lead me to doubt.

Add a Comment

Replies

The short answer here is that there is no safe way to use DateFormatter with a fixed dateFormat string without pinning the locale to en_US_POSIX. You can spend a bunch of time trying to understand the intricacies of how DateFormatter behaves but what does that buy you? The are two possible conclusions:

  • You are able to find a specific set of parameters that seem to work today.

  • You aren’t.

The second conclusion isn’t a problem. You’ll go straight to en_US_POSIX and all will be well [1].

The problem is with the first conclusion. You might be tempted to use those parameters in production code but, if you do that, you’re exposing yourself to future binary compatibility problems. For example, while the en_GB locale currently implies the Gregorian calendar, that could change in the future [2]. If that happens, your code will start reporting dates that are off by roughly 500 years. And while this is not a very realistic possibility, weird stuff like this happens all the time [3] and the only way to protect yourself is to use en_US_POSIX.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Or one of its alternatives, like NSISO8601DateFormatter or the Swift VerbatimFormatStyle.

[2] In a post-Brexit world, anything is possible, right?

[3] A more realistic example would be in the other direction, date rendering, where an Arabic-speaking country might change from western to eastern Arabic numerals, or vice versa.

  • Thank you eskimo! Much love.

    Personally, my main motivation in understanding why it fails is to be able to confidently say, here is the fix. Having said that, I hear what you are saying. When using fix-format date, you really really should use en_US_POSIX. I'll happily take that. The wording of the TN isn't as strong/clear which lead me to doubt.

Add a Comment

I now had a chance to make the suggested code changes and test this again. Unfortunately, this hasn't solved the issue. Here is the new code:


extension Date {

    /// Creates a `user-visible` date string with the given `template` for the given `locale` using the given `dateFormatter`. Effectively, this gives you 3 benefits:
    ///
    /// * You don't need to compute/guess the appropriate date format for every locale
    /// * You are guaranteed to always format a date that is localised to the user's locale
    /// * You respect user's preference on how to format a date
    ///
    /// The default arguments allow you to to create a `user-visible` date string that is localised to the user's current locale.
    ///
    /// Alternatively, use this method to create the correct date string for the given locale. e.g.
    ///
    /// Given a `template` of `yMMMMd` to create a `user-visible` date string that includes the year, full name of the month and day of the month for a `en_GB` locale, the string returned for a date representing April 1st in 1974 will be in the format of `d MMMM y` (1 April 1974) . While for a `en_US` locale will be in the format of `MMMM d, y` (April 1, 1974)
    ///
    /// - Parameters:
    ///     - parameter template: A string containing date format patterns (such as “MM” or “h”).
    ///     - parameter locale: The locale . `Locale.current` by default.
    ///     - parameter dateFormatter: The `DateFormatter` to generate the date string from this date.
    /// - Returns: a `user-visible` string formatted for the given locale. That means that the returned string may not contain exactly those components given in template, but may—for example—have locale-specific adjustments applied.
    ///
    /// - Note: This is the recommended API to use when creating `user-visible` date strings over `.toString(inFormat:)` or  `DateConverter.toString(date:format)`
    /// - Postcondition: any previously set `dateFormat` for the given `DateFormatter` will be replaced by a date format respecting the given `template` and `locale`.
    /// - SeeAlso: http://www.unicode.org/reports/tr35/tr35-31/tr35-dates.html#Date_Format_Patterns
    func string(template: String, locale: Locale = Locale.current, using dateFormatter: DateFormatter) -> String {
        dateFormatter.dateFormat = DateFormatter.dateFormat(fromTemplate: template, options: 0, locale: locale)
        return dateFormatter.string(from: self)
    }
}

extension DateFormatter {
    
     static let gregorianUTCTimezone: DateFormatter = make(calendar: Calendar(identifier: .gregorian), timeZone: TimeZone(identifier: "UTC")!)

    /// Returns a `DateFormatter` that can be used to parse and generate **fixed-format dates** in any date format.
    ///
    /// This date formatter will create dates on the Gregorian Calendar. By default, it does not take into account timezone information.
    ///
    /// This date formatter will behave **consistently** for all users, across devices and simulators with a different locale, calendar and user settings on date & time.
    ///
    /// - Parameters:
    ///     - parameter calendar: the calendar the date formatter should parse and generate dates in. `.gregorian` by default.
    ///     - parameter timeZone: the timezone the date formatter should parse and generate dates in. `nil` by default.
    /// - SeeAlso: https://developer.apple.com/library/archive/qa/qa1480/_index.html
    static func enUSPOSIXDateFormatter(calendar: Calendar = Calendar(identifier: .gregorian), timeZone: TimeZone? = nil) -> DateFormatter {
        let formatter = DateFormatter()
        formatter.locale = Locale(identifier: "en_US_POSIX")
        formatter.calendar = calendar
        formatter.timeZone = timeZone
        return formatter
    }
}


    /// Returns a date for the given string using the given `DateFormatter`. Use the default arguments to parse **fixed-format** (string) dates in a reliable way.
    ///
    /// You should **only** use instances returned by `.enUSPOSIXDateFormatter()` to parse and generate dates for any given `dateFormat`.
    ///
    /// Should you wish to provide a different `dateFormat`, e.g. one that includes `TimeZone` information, use `.enUSPOSIXDateFormatter()` to create a
    /// `en_US_POSIX` date formatter with a `TimeZone`.
    ///
    /// - Parameters:
    ///     - parameter string: The date string used with this should be of the fixed.
    ///     - parameter dateFormat:the format the given `string` must be in. *dd/MM/yyyy HH:mm* by default.
    ///     - parameter dateFormatter: the `DateFormatter` to parse the given date string. `DateFormatter.enUSPOSIXDateFormatter` by default.
    /// - Returns: a date object for the given date `string` as represented by the given `dateFormat`
    /// - SeeAlso: `DateFormatter.enUSPOSIXDateFormatter`
    static func date(string: String, dateFormat: String = "dd/MM/yyyy HH:mm", dateFormatter: DateFormatter = .enUSPOSIXDateFormatter()) -> Date {
        dateFormatter.dateFormat = dateFormat
        guard let date = dateFormatter.date(from: string) else {
            let error = DateFormatError.UnexpectedFormat(actual: string, expected: dateFormatter.dateFormat)
            preconditionFailure(String(reflecting: error))
        }
        
        return date
    }

Using the above code as shown below, I see the same issue as described in the original post.


func testExample() throws {
    let date = DateFormatter.date(string: "08/03/2023 12:10")

    // On the computer that failed, this date was at 17:10 (i.e. off by 5 hours)
    let actual = date.string(template: "HHmm", locale: Locale(identifier: "en_GB"), using: .gregorianUTCTimezone)
    let expected = "12:10"

    // One case where actual was 17:10 and expected 12:10.
    XCTAssertEqual(actual, expected)
}

I feel like either I have misunderstood something or have made a mistake in the code I am blind to.

I've tried to write a test to further isolate the issue to the formatter.

    func testGivenFixedFormatDateAssertComponents() throws {
        var calendar: Calendar = Calendar(identifier: .gregorian)
        calendar.timeZone = TimeZone(identifier: "UTC")!
        let dateComponents = DateComponents(calendar: calendar,
                                            year: 2023,
                                            month: 3,
                                            day: 8,
                                            hour: 12,
                                            minute: 10)

        let given = "08/03/2023 12:10"
        let actual = try DateFormatter.date(string: given)
        
        let expected = try XCTUnwrap(dateComponents.date)
        
        XCTAssertEqual(actual, expected) ///XCTAssertEqual failed: ("2023-03-08 17:10:00 +0000") is not equal to ("2023-03-08 12:10:00 +0000")
    }

I've noticed that the enUSPOSIXDateFormatter does not specify the TimeZone. Looking into the documentation I see that it says "If unspecified, the system time zone is used."

I believe this must be the issue even though I'm pretty sure I remember coming across a different documentation that said "if unspecified, GMT will be used." I can't recall where I saw that.

  • That was indeed the issue. The "The time zone used to create and parse date representations. When unspecified, GMT is used." is true for the ISO8601DateFormatter so I must have mixed the two in my head while working on this.

Add a Comment