Hello, @interferenc!
Thank you for the detailed post. The numbers are very helpful.
On the memory spike:
The scaling of 4-5 times the memory increase for 2 times the item count is definitely something worth looking at.
Are you able to file a bug report with Feedback Assistant and attach this sample project you provided?
Once complete, reply with the Feedback number so I can make sure this report is routed to the relevant engineering team.
As a workaround, rather than calling selectItem(at:animated:scrollPosition:) for every index path, maintain a Set<IndexPath> or an isAllSelected: Bool flag in your data source. Then in cellForItemAt, apply the visual selected state manually.
This avoids UICollectionView's internal selection machinery for 10k-20k of items at once, since it appears that it might not be designed to support that volume.
Creating a Feedback report will create direct dialog with the engineering team directly and they can even respond to you there. I'm curious to investigate this further with that team.
Thank you. I hope this helps.
Travis