According to the MKMapview, all annotations should be added to the map:
https://developer.apple.com/documentation/mapkit/mkmapview
When configuring your map interface, be sure to add all of your annotation objects right away. The map view uses the coordinate data in each annotation object to determine when the corresponding annotation view needs to appear onscreen. When an annotation moves onscreen, the map view asks its delegate to create a corresponding annotation view. If your app has different types of annotations, it can define different annotation view classes to represent each type.
But this has a very low performance when the map is zoomed out and all annotations are visible (although market already filters out most of the annotations and only displays a few). As fas as I've seen, the suggestion from the documentation doesn't scale properly when the number of annotations is very large.
Grouping doesn't help, neither using reusable annotations.
Is the only option here, handling manually the add/remove annotations as the map rect changes?