Swift Charts: chartScrollTargetBehavior fails to snap to a day unit

I am working on a scrollable chart that displays days on the horizontal axis. As the user scrolls, I always want them to be able to snap to a specific day. I implemented the following steps described in this WWDC23 session to achieve this.

1. I have set the chartScrollTargetBehavior to .valueAligned(matching: DateComponents(hour: 0))
2. I have set the x value unit on the BarMark to Calendar.Component.day

I ended up with the chart code that looks like this:

Chart(dates, id: \.self) { date in
    BarMark(
        x: .value("Date", date, unit: Calendar.Component.day),
        y: .value("Number", 1)
    )
    .annotation {
        Text(date.formatted(.dateTime.day()))
            .font(.caption2)
    }
}
.chartXAxis {
    AxisMarks(format: .dateTime.day())
}
.chartScrollableAxes(.horizontal)
.chartScrollTargetBehavior(.valueAligned(matching: DateComponents(hour: 0)))
.chartXVisibleDomain(length: fifteenDays)
.chartScrollPosition(x: $selection)

However, this fails to work reliably. There is often a situation where the chart scroll position lands on, for instance, Oct 20, 11:56 PM, but the chart snaps to Oct 21.

I attempted to solve this problem by introducing an intermediate binding between a state value and a chart selection. This binding aims to normalize the selection always to be the first moment of any given date. But this hasn't been successful.

private var selectionBinding: Binding<Date> {
    Binding {
        Calendar.current.startOfDay(for: selection)
    } set: { newValue in
        self.selection = Calendar.current.startOfDay(for: newValue)
    }
}

It's also worth mentioning that this issue also exists in Apple's sample project on Swift Charts.

How would you approach solving this? How can I find a way to make the chart scroll position blind to time values and only recognize whole days?

Here's the minimal reproducible example project for your reference.