Actually seems that the above described solution works with the latest iOS and macOS, so should take my words back :)

"The TLDR is: ..." - very good summary, I have the exact same behaviour. Been trying and testing for days and no luck getting a public, a private and a shared database to work with the new sharing API. Same setup (without the new sharing API) works fine on iOS 14.5 and Xcode 12.5.1. In addition, in Xcode 13 beta 3 (latest build ending with j), in the simulator, if I use a test Apple ID (not my personal), I have to do "device" > "erase all contents and settings..." every day, because the simulator just stops working with iCloud and asks to review the terms (which I have done at icloud.com). I also get an occasional error "appleid... stopped running... reopen". I use macOS Big Sur 11.4. Very disappointing, please fix these bugs asap.