I'm nervous to just revoke those and start over new,
Yes, revoking your Distribution Signing Certificate could disrupt existing Provisioning Profiles and services that rely on this certificate. Note the message when trying to revoke a certificate:
" Revoking this certificate will invalidate it and any related services or provisioning profiles that use this certificate may be affected."
Instead, what you can do is create a new CSR and create a new Distribution Signing Identity on your machine. When I refer to a signing identity here I am referring to the private key that is generated when you create a CSR from the Keychain and the Distribution Certificate that is issued from the Developer Portal and downloaded to your Keychain. From there you should be able to create a new provisioning profile with your new Signing Identity to update your app. My recommendation would be to push your update through TestFlight first to make sure you've recreated your profile 1:1 like your old one and have a chance to iron anything you need to out before it hits production.
Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com