Question regarding to Service extension handling.

Hi All,

Recently I received customer report that my service extension suddenly stop working, and it worked fine for a while.

I reviewed the Customer report, and found Service extension stop at in the middle of nowhere, and it does not wake up afterwards. one of the device has service extension log, but I can't parse it. once service extension get into this state, rebooting device / terminating app does not help, only remove/re-install will make it work again according to customer's report.

Here are my question:

  1. How to detect Service extension stop working? it is hard to reproduce on my side.
  2. Can anyone help me to parse the crash and identify the root cause?

Thanks in advance.

{"app_name":"ServiceExtension","timestamp":"2022-10-11 01:01:33.00 -0500","app_version":"3.12.0","slice_uuid":"d232b48f-7906-30e1-8b47-19075e8b88bf","build_version":"3.12.0.3536","platform":2,"bundleID":"com.vocera.vcs.ServiceExtension","share_with_app_devs":0,"is_first_party":1,"bug_type":"309","os_version":"iPhone OS 16.0.2 (20A380)","roots_installed":0,"name":"ServiceExtension","incident_id":"92623797-3571-43EC-9F76-0A597E7D96DA"}
{
  "uptime" : 460000,
  "procRole" : "Unspecified",
  "version" : 2,
  "userID" : 501,
  "deployVersion" : 210,
  "modelCode" : "iPhone15,2",
  "coalitionID" : 9631,
  "osVersion" : {
    "isEmbedded" : true,
    "train" : "iPhone OS 16.0.2",
    "releaseType" : "User",
    "build" : "20A380"
  },
  "captureTime" : "2022-10-11 01:01:33.3725 -0500",
  "incident" : "92623797-3571-43EC-9F76-0A597E7D96DA",
  "pid" : 23733,
  "cpuType" : "ARM-64",
  "roots_installed" : 0,
  "bug_type" : "309",
  "procLaunch" : "2022-10-10 20:12:27.8132 -0500",
  "procStartAbsTime" : 10817558550868,
  "procExitAbsTime" : 11150569912891,
  "procName" : "ServiceExtension",
  "procPath" : "\/private\/var\/containers\/Bundle\/Application\/95F1FCD4-AB74-4DFE-B0B5-D94BBFF6BCF0\/VCS.app\/PlugIns\/ServiceExtension.appex\/ServiceExtension",
  "bundleInfo" : {"CFBundleShortVersionString":"3.12.0","CFBundleVersion":"3.12.0.3536","CFBundleIdentifier":"com.vocera.vcs.ServiceExtension"},
  "parentProc" : "launchd",
  "parentPid" : 1,
  "coalitionName" : "com.vocera.vcs.ServiceExtension",
  "crashReporterKey" : "9ae94a7bf4ad3fea4749f56701f9e72add6d8c94",
  "basebandVersion" : "1.00.05",
  "exception" : {"codes":"0x0000000000000000, 0x0000000000000000","rawCodes":[0,0],"type":"EXC_CRASH","signal":"SIGABRT"},
  "asi" : {"libsystem_c.dylib":["abort() called"]},
  "lastExceptionBacktrace" : [{"imageOffset":41544,"symbol":"__exceptionPreprocess","symbolLocation":164,"imageIndex":6},{"imageOffset":96872,"symbol":"objc_exception_throw","symbolLocation":60,"imageIndex":4},{"imageOffset":1763028,"symbol":"-[__NSCFString characterAtIndex:].cold.1","symbolLocation":0,"imageIndex":6},{"imageOffset":1284788,"symbol":"-[__NSCFAttributedString _tryRetain]","symbolLocation":0,"imageIndex":6},{"imageOffset":42948,"symbol":"__NSSET_IS_CALLING_OUT_TO_A_BLOCK__","symbolLocation":0,"imageIndex":6},{"imageOffset":60980,"imageIndex":14},{"imageOffset":54496,"imageIndex":14},{"imageOffset":227268,"symbol":"__CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__","symbolLocation":148,"imageIndex":6},{"imageOffset":911196,"symbol":"___CFXRegistrationPost_block_invoke","symbolLocation":88,"imageIndex":6},{"imageOffset":793140,"symbol":"_CFXRegistrationPost","symbolLocation":440,"imageIndex":6},{"imageOffset":310728,"symbol":"_CFXNotificationPost","symbolLocation":704,"imageIndex":6},{"imageOffset":378076,"symbol":"-[NSNotificationCenter postNotificationName:object:userInfo:]","symbolLocation":92,"imageIndex":7},{"imageOffset":119344,"imageIndex":14},{"imageOffset":9396,"symbol":"_dispatch_call_block_and_release","symbolLocation":32,"imageIndex":5},{"imageOffset":16348,"symbol":"_dispatch_client_callout","symbolLocation":20,"imageIndex":5},{"imageOffset":75764,"symbol":"_dispatch_main_queue_drain","symbolLocation":928,"imageIndex":5},{"imageOffset":74820,"symbol":"_dispatch_main_queue_callback_4CF","symbolLocation":44,"imageIndex":5},{"imageOffset":633352,"symbol":"__CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__","symbolLocation":16,"imageIndex":6},{"imageOffset":508776,"symbol":"__CFRunLoopRun","symbolLocation":2036,"imageIndex":6},{"imageOffset":528868,"symbol":"CFRunLoopRunSpecific","symbolLocation":612,"imageIndex":6},{"imageOffset":268312,"symbol":"-[NSRunLoop(NSRunLoop) runMode:beforeDate:]","symbolLocation":212,"imageIndex":7},{"imageOffset":267948,"symbol":"-[NSRunLoop(NSRunLoop) run]","symbolLocation":64,"imageIndex":7},{"imageOffset":105620,"symbol":"_xpc_objc_main","symbolLocation":496,"imageIndex":8},{"imageOffset":114164,"symbol":"xpc_main","symbolLocation":156,"imageIndex":8},{"imageOffset":561100,"symbol":"+[NSXPCListener serviceListener]","symbolLocation":0,"imageIndex":7},{"imageOffset":106316,"imageIndex":9},{"imageOffset":26420,"imageIndex":9},{"imageOffset":23112,"imageIndex":9},{"imageOffset":136616,"symbol":"EXExtensionMain","symbolLocation":252,"imageIndex":10},{"imageOffset":840668,"symbol":"NSExtensionMain","symbolLocation":240,"imageIndex":7},{"imageOffset":88392,"symbol":"start","symbolLocation":2504,"imageIndex":11}],
  "faultingThread" : 0,
  "threads" : [{"triggered":true,"id":5215709,"threadState":{"x":[{"value":0},{"value":0},{"value":0},{"value":0},{"value":8771473653},{"value":6092250992},{"value":110},{"value":18446744069414584832},{"value":3971888757583221645},{"value":3971888748797721165},{"value":512},{"value":11},{"value":11},{"value":0},{"value":0},{"value":0},{"value":328},{"value":8932354496,"symbolLocation":0,"symbol":"_main_thread"},{"value":0},{"value":6},{"value":259},{"value":8932354720,"symbolLocation":224,"symbol":"_main_thread"},{"value":0},{"value":276},{"value":0},{"value":8932354720,"symbolLocation":224,"symbol":"_main_thread"},{"value":4391439488},{"value":15},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":8772235692},"cpsr":{"value":1073745920},"fp":{"value":6092250848},"sp":{"value":6092250816},"esr":{"value":1442840704,"description":" Address size fault"},"pc":{"value":8501453312,"matchesCrashFrame":1},"far":{"value":8888529264}},"queue":"com.apple.main-thread","frames":[{"imageOffset":29184,"symbol":"__pthread_kill","symbolLocation":8,"imageIndex":0},{"imageOffset":29100,"symbol":"pthread_kill","symbolLocation":268,"imageIndex":1},{"imageOffset":134304,"symbol":"abort","symbolLocation":180,"imageIndex":2},{"imageOffset":72588,"symbol":"abort_message","symbolLocation":132,"imageIndex":3},{"imageOffset":6784,"symbol":"demangling_terminate_handler()","symbolLocation":336,"imageIndex":3},{"imageOffset":122604,"symbol":"_objc_terminate()","symbolLocation":144,"imageIndex":4},{"imageOffset":69416,"symbol":"std::__terminate(void (*)())","symbolLocation":20,"imageIndex":3},{"imageOffset":69316,"symbol":"std::terminate()","symbolLocation":56,"imageIndex":3},{"imageOffset":16368,"symbol":"_dispatch_client_callout","symbolLocation":40,"imageIndex":5},{"imageOffset":75764,"symbol":"_dispatch_main_queue_drain","symbolLocation":928,"imageIndex":5},{"imageOffset":74820,"symbol":"_dispatch_main_queue_callback_4CF","symbolLocation":44,"imageIndex":5},{"imageOffset":633352,"symbol":"__CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__","symbolLocation":16,"imageIndex":6},{"imageOffset":508776,"symbol":"__CFRunLoopRun","symbolLocation":2036,"imageIndex":6},{"imageOffset":528868,"symbol":"CFRunLoopRunSpecific","symbolLocation":612,"imageIndex":6},{"imageOffset":268312,"symbol":"-[NSRunLoop(NSRunLoop) runMode:beforeDate:]","symbolLocation":212,"imageIndex":7},{"imageOffset":267948,"symbol":"-[NSRunLoop(NSRunLoop) run]","symbolLocation":64,"imageIndex":7},{"imageOffset":105620,"symbol":"_xpc_objc_main","symbolLocation":496,"imageIndex":8},{"imageOffset":114164,"symbol":"xpc_main","symbolLocation":156,"imageIndex":8},{"imageOffset":561100,"symbol":"-[NSXPCListener resume]","symbolLocation":312,"imageIndex":7},{"imageOffset":106316,"imageIndex":9},{"imageOffset":26420,"imageIndex":9},{"imageOffset":23112,"imageIndex":9},{"imageOffset":136616,"symbol":"EXExtensionMain","symbolLocation":252,"imageIndex":10},{"imageOffset":840668,"symbol":"NSExtensionMain","symbolLocation":240,"imageIndex":7},{"imageOffset":88392,"symbol":"start","symbolLocation":2504,"imageIndex":11}]},{"id":5215723,"name":"com.apple.NSURLConnectionLoader","frames":[{"imageOffset":2888,"symbol":"mach_msg2_trap","symbolLocation":8,"imageIndex":0},{"imageOffset":77832,"symbol":"mach_msg2_internal","symbolLocation":80,"imageIndex":0},{"imageOffset":78408,"symbol":"mach_msg_overwrite","symbolLocation":388,"imageIndex":0},{"imageOffset":4236,"symbol":"mach_msg","symbolLocation":24,"imageIndex":0},{"imageOffset":503296,"symbol":"__CFRunLoopServiceMachPort","symbolLocation":160,"imageIndex":6},{"imageOffset":507972,"symbol":"__CFRunLoopRun","symbolLocation":1232,"imageIndex":6},{"imageOffset":528868,"symbol":"CFRunLoopRunSpecific","symbolLocation":612,"imageIndex":6},{"imageOffset":2444204,"imageIndex":12},{"imageOffset":371944,"symbol":"__NSThread__start__","symbolLocation":716,"imageIndex":7},{"imageOffset":5836,"symbol":"_pthread_start","symbolLocation":148,"imageIndex":1},{"imageOffset":2980,"symbol":"thread_start","symbolLocation":8,"imageIndex":1}]},{"id":5391799,"frames":[{"imageOffset":2960,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":1}]},{"id":5392232,"queue":"com.apple.CFNetwork.CacheDB-write","frames":[{"imageOffset":10276,"symbol":"guarded_pwrite_np","symbolLocation":8,"imageIndex":0},{"imageOffset":749496,"imageIndex":13},{"imageOffset":447532,"imageIndex":13},{"imageOffset":784664,"imageIndex":13},{"imageOffset":445636,"imageIndex":13},{"imageOffset":361824,"imageIndex":13},{"imageOffset":146676,"imageIndex":13},{"imageOffset":337424,"imageIndex":13},{"imageOffset":278296,"symbol":"sqlite3_step","symbolLocation":1300,"imageIndex":13},{"imageOffset":38032,"symbol":"sqlite3_exec","symbolLocation":368,"imageIndex":13},{"imageOffset":278956,"imageIndex":12},{"imageOffset":847224,"imageIndex":12},{"imageOffset":78100,"symbol":"_dispatch_block_async_invoke2","symbolLocation":148,"imageIndex":5},{"imageOffset":16348,"symbol":"_dispatch_client_callout","symbolLocation":20,"imageIndex":5},{"imageOffset":46740,"symbol":"_dispatch_lane_serial_drain","symbolLocation":672,"imageIndex":5},{"imageOffset":49684,"symbol":"_dispatch_lane_invoke","symbolLocation":436,"imageIndex":5},{"imageOffset":93712,"symbol":"_dispatch_workloop_worker_thread","symbolLocation":652,"imageIndex":5},{"imageOffset":3576,"symbol":"_pthread_wqthread","symbolLocation":288,"imageIndex":1},{"imageOffset":2968,"symbol":"start_wqthread","symbolLocation":8,"imageIndex":1}]},{"id":5392233,"frames":[{"imageOffset":2960,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":1}]},{"id":5392363,"frames":[{"imageOffset":2960,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":1}]},{"id":5392364,"frames":[{"imageOffset":2960,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":1}]}],
  "usedImages" : [
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 8501424128,
    "size" : 225276,
    "uuid" : "a5d3b725-78c3-3e19-a765-cceb22355093",
    "path" : "\/usr\/lib\/system\/libsystem_kernel.dylib",
    "name" : "libsystem_kernel.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 8772206592,
    "size" : 49152,
    "uuid" : "b89b9a5b-55d9-3e84-b6d3-c3da93c1cd39",
    "path" : "\/usr\/lib\/system\/libsystem_pthread.dylib",
    "name" : "libsystem_pthread.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7609995264,
    "size" : 524288,
    "uuid" : "82c0f938-e277-3165-a614-70151c651798",
    "path" : "\/usr\/lib\/system\/libsystem_c.dylib",
    "name" : "libsystem_c.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 8771387392,
    "size" : 98300,
    "uuid" : "e0dbf945-9cc5-305d-b096-abc1cda9c8ca",
    "path" : "\/usr\/lib\/libc++abi.dylib",
    "name" : "libc++abi.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7373979648,
    "size" : 277568,
    "uuid" : "b3a78098-c0fb-3dcd-b1ac-0712762510db",
    "path" : "\/usr\/lib\/libobjc.A.dylib",
    "name" : "libobjc.A.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7609704448,
    "size" : 290816,
    "uuid" : "c663d847-b94f-3fb0-9254-32edbc55315e",
    "path" : "\/usr\/lib\/system\/libdispatch.dylib",
    "name" : "libdispatch.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7487537152,
    "size" : 4083712,
    "uuid" : "42c5c917-0447-3995-b50f-de4d132c2435",
    "path" : "\/System\/Library\/Frameworks\/CoreFoundation.framework\/CoreFoundation",
    "name" : "CoreFoundation"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7391346688,
    "size" : 9744384,
    "uuid" : "aa92cd58-561a-3414-92f4-b4120298b39a",
    "path" : "\/System\/Library\/Frameworks\/Foundation.framework\/Foundation",
    "name" : "Foundation"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 8772485120,
    "size" : 262144,
    "uuid" : "30ec91de-da90-35e1-8403-6402e6559bae",
    "path" : "\/usr\/lib\/system\/libxpc.dylib",
    "name" : "libxpc.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 8107638784,
    "size" : 225280,
    "uuid" : "8f6980fa-6991-3658-a230-f7f0ab5fd801",
    "path" : "\/System\/Library\/PrivateFrameworks\/PlugInKit.framework\/PlugInKit",
    "name" : "PlugInKit"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7709122560,
    "size" : 401408,
    "uuid" : "7af812bd-ed46-3096-9f87-9f7dad295e0a",
    "path" : "\/System\/Library\/Frameworks\/ExtensionFoundation.framework\/ExtensionFoundation",
    "name" : "ExtensionFoundation"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7994589184,
    "size" : 536144,
    "uuid" : "341bbf64-6034-357e-8aa6-e1e4b988e03c",
    "path" : "\/usr\/lib\/dyld",
    "name" : "dyld"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7505866752,
    "size" : 3956736,
    "uuid" : "35605de3-3723-335a-83d9-6f35f2989935",
    "path" : "\/System\/Library\/Frameworks\/CFNetwork.framework\/CFNetwork",
    "name" : "CFNetwork"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 8084262912,
    "size" : 1699832,
    "uuid" : "110cd7ec-982d-3425-932d-c9dddd8a571a",
    "path" : "\/usr\/lib\/libsqlite3.dylib",
    "name" : "libsqlite3.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64",
    "base" : 4374626304,
    "size" : 1589248,
    "uuid" : "d232b48f-7906-30e1-8b47-19075e8b88bf",
    "path" : "\/private\/var\/containers\/Bundle\/Application\/95F1FCD4-AB74-4DFE-B0B5-D94BBFF6BCF0\/VCS.app\/PlugIns\/ServiceExtension.appex\/ServiceExtension",
    "name" : "ServiceExtension"
  }
],
  "sharedCache" : {
  "base" : 7373225984,
  "size" : 2891382784,
  "uuid" : "878a8ca2-7334-3bc2-a4c8-ff6069d46cc1"
},
  "vmSummary" : "ReadOnly portion of Libraries: Total=689.6M resident=0K(0%) swapped_out_or_unallocated=689.6M(100%)\nWritable regions: Total=70.7M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=70.7M(100%)\n\n                                VIRTUAL   REGION \nREGION TYPE                        SIZE    COUNT (non-coalesced) \n===========                     =======  ======= \nActivity Tracing                   256K        1 \nDispatch continuations            6144K        1 \nKernel Alloc Once                   32K        1 \nMALLOC                            59.5M       33 \nMALLOC guard page                  128K        8 \nSQLite page cache                  192K        3 \nSTACK GUARD                        112K        7 \nStack                             4272K        7 \n__AUTH                            1474K      245 \n__AUTH_CONST                      14.8M      395 \n__CTF                               756        1 \n__DATA                            3782K      383 \n__DATA_CONST                      16.8M      398 \n__DATA_DIRTY                      1393K      324 \n__FONT_DATA                          4K        1 \n__LINKEDIT                       302.5M        2 \n__OBJC_CONST                      2900K      235 \n__OBJC_RO                         78.1M        1 \n__OBJC_RW                         2415K        1 \n__TEXT                           387.2M      408 \ndyld private memory                256K        1 \nlibnetwork                         640K       16 \nmapped file                       30.7M        8 \nshared memory                       48K        3 \n===========                     =======  ======= \nTOTAL                            913.0M     2483 \n",
  "legacyInfo" : {
  "threadTriggered" : {
    "queue" : "com.apple.main-thread"
  }
},
  "trialInfo" : {
  "rollouts" : [
    {
      "rolloutId" : "62fe74515312cd4599bd3c80",
      "factorPackIds" : {
        "MYRIAD_BOOSTS" : "62fe74805312cd4599bd3c81"
      },
      "deploymentId" : 240000006
    },
    {
      "rolloutId" : "60da5e84ab0ca017dace9abf",
      "factorPackIds" : {

      },
      "deploymentId" : 240000008
    }
  ],
  "experiments" : [
    {
      "treatmentId" : "a2c66376-6910-4142-b340-3476dd2ce1bf",
      "experimentId" : "621923057b59736391be1571",
      "deploymentId" : 400000011
    },
    {
      "treatmentId" : "07a84544-f1ae-44db-9549-dc729e749e1f",
      "experimentId" : "6148ab04480ea0575d7a537c",
      "deploymentId" : 400000025
    }
  ]
}
}

Answer by myself, We found it was not Notification service extension issue, it was on our push sever stop pushing

Question regarding to Service extension handling.
 
 
Q