I should have mentioned that this is happening after using the "Restore Purchases" button in my app. This calls the restoreCompletedTransactions() method on the default SKPaymentQueue. I then wait for paymentQueueRestoreCompletedTransactionsFinished() and send the receipt to my server for validation. Afterward I go through the items in transactions where the transactionState is restored or purchased and call finishTransaction() for each one.
I did some testing, and it seems this is an important step in what's happening. If I validate the same receipt without restoring anything, I get consistent results. The transaction in latest_receipt_info is still not the most recent transaction—but it is the same transaction each time.
When I discovered this, I thought I might be able to solve the problem by sorting the transactions by date before calling finishTransaction(). This doesn't seem to make a difference. It's possible I'm overlooking something in my code but it seems like simply calling restoreCompletedTransactions() jumbles the transactions and results in a random one being considered the "latest" according to the verifyReceipt endpoint.
This post on Stack Overflow - https://stackoverflow.com/q/42312927/813247 is the closest I've found to anyone discussing similar behavior. I hadn't noticed it previously but I'm seeing the same thing—restoreCompletedTransactions() does change the transaction IDs. Seems surprising to me, but it's not a problem. Then I noticed that original_purchase_date is also changing—it's the current date and time! The purchase_date is the actual date of the purchase. That's backwards from what I would expect. But this at least makes sense now: according to this date it is the latest transaction. It seems like it's re-running all the transactions in an effectively random order and updating the purchase date as it goes. This still doesn't make sense to me, but I at least feel like I understand what's happening.
The sole answer to that Stack Overflow post suggests using SKReceiptRefreshRequest instead of restoreCompletedTransactions(). My understanding was that these do not necessarily do the same thing, and restoreCompletedTransactions() should be done to make sure transactions appear on a new device, or after deleting and reinstalling the app. SKReceiptRefreshRequest is what you should use if the receipt file is invalid or missing. These seem like pretty distinct use cases that are backed up by the current documentation. There's also a reply here from someone at Apple - https://developer.apple.com/forums/thread/127923 suggesting the app won't pass app review if I rely on SKReceiptRefreshRequest for this.
I think at this point I'm going to:
Assume latest_receipt_info may contain more than one transaction in non-chronological order. I'll loop through each one and look for the latest expires_date_ms. This addresses my first two questions and seems like the safest approach.
Assume that latest_receipt_info may not include the most recent purchase and check receipt.in_app as well. It's extra work, but due to the way purchases are restored it seems necessary to show my customers when their subscription expired.
For my bonus questions I'm just going to assume I'm looking in the right place for the grace period and that pending_renewal_info should only contain a single item in my case. It shouldn't be hard to adjust these later as necessary.
I feel pretty comfortable with this now, but if anyone has relevant knowledge I'd love to hear it.