more 0x8BADF00D issue on iOS 15.4

After iOS15.4 was released, we noticed the issue. It seems a system bug. It's waiting for the lock for a long time(dead lock).

Hardware Model:      iPhone13,3
Process:             app [9492]
Path:                /private/var/containers/Bundle/Application/***/app.app/app
Identifier:          com.aaa.app
AppStoreTools:       13C90b
AppVariant:          1:iPhone13,3:15
Code Type:           ARM-64 (Native)
Role:                Foreground
Parent Process:      launchd [1]
Coalition:           com.aaa.app [956]

Date/Time:           2022-03-17 18:04:46.2811 +0800
Launch Time:         2022-03-17 17:35:18.7389 +0800
OS Version:          iPhone OS 15.4 (19E241)
Release Type:        User
Baseband Version:    2.53.01
Report Version:      104

Exception Type:  EXC_CRASH (SIGKILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Termination Reason: FRONTBOARD 2343432205 
<RBSTerminateContext| domain:10 code:0x8BADF00D explanation:scene-update watchdog transgression: application<com.aaa.app>:9492 exhausted real (wall clock) time allowance of 10.00 seconds
ProcessVisibility: Background
ProcessState: Running
WatchdogEvent: scene-update
WatchdogVisibility: Background
WatchdogCPUStatistics: (
"Elapsed total CPU time (seconds): 20.970 (user 13.070, system 7.900), 35% CPU",
"Elapsed application CPU time (seconds): 0.829, 1% CPU"
) reportType:CrashLog maxTerminationResistance:Interactive>

Triggered by Thread:  0


Kernel Triage:
VM - pmap_enter failed with resource shortage
VM - pmap_enter failed with resource shortage


Thread 0 name:
Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x00000001cddd1178 __ulock_wait + 8
1   libsystem_platform.dylib      	0x0000000207a399ac _os_unfair_lock_lock_slow + 172 (lock.c:578)
2   JavaScriptCore                	0x00000001a26f1c38 pas_page_sharing_pool_take_least_recently_used + 800 (lock_private.h:684)
3   JavaScriptCore                	0x00000001a26f2cf4 pas_physical_page_sharing_pool_take + 248 (pas_page_sharing_pool.c:716)
4   JavaScriptCore                	0x00000001a26ca2ec bmalloc_medium_bitfit_page_config_specialized_allocator_try_allocate + 260 (pas_page_sharing_pool.c:862)
5   JavaScriptCore                	0x00000001a26cf48c bmalloc_heap_config_specialized_local_allocator_try_allocate_slow + 328 (pas_local_allocator_inlines.h:1595)
6   JavaScriptCore                	0x00000001a26ba39c bmalloc_allocate_impl_casual_case + 888 (pas_local_allocator_inlines.h:1796)
7   JavaScriptCore                	0x00000001a19d6fb8 ***::RobinHoodHashTable<***::RefPtr<***::UniquedStringImpl, ***::RawPtrTraits<***::UniquedStringImpl>, ***::DefaultRefDerefTraits<***::UniquedStringImpl> >, ***::KeyValuePair<***::RefPtr<***::Uniqu... + 52 (FastMalloc.h:219)
8   JavaScriptCore                	0x00000001a19c6a2c ***::HashTableAddResult<***::HashTableIterator<***::RobinHoodHashTable<***::RefPtr<***::UniquedStringImpl, ***::RawPtrTraits<***::UniquedStringImpl>, ***::DefaultRefDerefTraits<***::UniquedStringIm... + 140 (RobinHoodHashTable.h:729)
9   JavaScriptCore                	0x00000001a19c3dac JSC::BytecodeIntrinsicRegistry::BytecodeIntrinsicRegistry(JSC::VM&) + 33204 (BytecodeIntrinsicRegistry.cpp:59)
10  JavaScriptCore                	0x00000001a23931fc JSC::VM::VM(JSC::VM::VMType, JSC::HeapType, ***::RunLoop*, bool*) + 11444 (unique_ptr.h:728)
11  JavaScriptCore                	0x00000001a2395fd8 JSC::VM::createContextGroup(JSC::HeapType) + 52 (VM.cpp:241)
12  JavaScriptCore                	0x00000001a1811dc0 JSContextGroupCreate + 32 (JSContextRef.cpp:70)
13  JavaScriptCore                	0x00000001a14aa2a4 -[JSVirtualMachine init] + 24 (JSVirtualMachine.mm:94)
14  JavaScriptCore                	0x00000001a14aa304 -[JSContext init] + 32 (JSContext.mm:71)
15  WebKit                        	0x00000001a5058b94 API::SharedJSContext::ensureContext() + 56 (APISerializedScriptValueCocoa.mm:51)
16  WebKit                        	0x00000001a5058ad4 API::SerializedScriptValue::deserialize(WebCore::SerializedScriptValue&, OpaqueJSValue const**) + 44 (APISerializedScriptValueCocoa.mm:78)
17  WebKit                        	0x00000001a50d7180 ScriptMessageHandlerDelegate::didPostMessage(WebKit::WebPageProxy&, WebKit::FrameInfoData&&, API::ContentWorld&, WebCore::SerializedScriptValue&) + 136 (WKUserContentController.mm:152)
18  WebKit                        	0x00000001a5424da4 WebKit::WebUserContentControllerProxy::didPostMessage(***::ObjectIdentifier<WebKit::WebPageProxyIdentifierType>, WebKit::FrameInfoData&&, unsigned long long, ***::Span<unsigned char const, 18446744... + 732 (WebUserContentControllerProxy.cpp:346)
19  WebKit                        	0x00000001a5708dcc WebKit::WebUserContentControllerProxy::didReceiveMessage(IPC::Connection&, IPC::Decoder&) + 2716 (HandleMessage.h:139)
20  WebKit                        	0x00000001a5159b60 IPC::MessageReceiverMap::dispatchMessage(IPC::Connection&, IPC::Decoder&) + 264 (MessageReceiverMap.cpp:129)
21  WebKit                        	0x00000001a5381dac WebKit::WebProcessProxy::didReceiveMessage(IPC::Connection&, IPC::Decoder&) + 40 (AuxiliaryProcessProxy.cpp:247)
22  WebKit                        	0x00000001a514e2ac IPC::Connection::dispatchMessage(std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >) + 788 (Connection.cpp:1080)
23  WebKit                        	0x00000001a514d960 IPC::Connection::dispatchIncomingMessages() + 508 (Connection.cpp:1229)
24  JavaScriptCore                	0x00000001a265de10 ***::RunLoop::performWork() + 200 (Function.h:82)
25  JavaScriptCore                	0x00000001a265ebac ***::RunLoop::performWork(void*) + 36 (RunLoopCF.cpp:46)
26  CoreFoundation                	0x000000019632ff04 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 28 (CFRunLoop.c:1972)
27  CoreFoundation                	0x0000000196340c90 __CFRunLoopDoSource0 + 208 (CFRunLoop.c:2016)
28  CoreFoundation                	0x000000019627a184 __CFRunLoopDoSources0 + 268 (CFRunLoop.c:2053)
29  CoreFoundation                	0x000000019627fb4c __CFRunLoopRun + 828 (CFRunLoop.c:2951)
30  CoreFoundation                	0x00000001962936b8 CFRunLoopRunSpecific + 600 (CFRunLoop.c:3268)
31  GraphicsServices              	0x00000001b232d374 GSEventRunModal + 164 (GSEvent.c:2200)
32  UIKitCore                     	0x0000000198bf8e88 -[UIApplication _run] + 1100 (UIApplication.m:3511)
33  UIKitCore                     	0x000000019897a5ec UIApplicationMain + 364 (UIApplication.m:5064)
34  app                     	0x0000000104cd277c main + 388 (main.mm:38)
35  dyld                          	0x0000000109919ce4 start + 520 (dyldMain.cpp:879)
Up vote post of codefei Down vote post of codefei
3.5k views
Posted by
codefei
Reply
Add a Comment

Accepted Reply

I had reason to dig into this issue in another context and it seems that this is an issue we’re tracking (r. 89020902). AFAICT this should be fixed in the iOS 15.5b1 (19F5047e) that we’re currently seeding. If you still see the problem on that release, or anything later, please let us know.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Post marked as solved
Add a Comment

Replies

same issue+1

it seems this commit enabled libpas, didn't see the same backtrace before.pas_page_sharing_pool_take_least_recently_used

anyone knows which iOS version merged this commit?

Posted by
zltunes
Post not yet marked as solved Up vote reply of zltunes Down vote reply of zltunes
Add a Comment

It's waiting for the lock for a long time(dead lock).

Please post a full crash report for this. See my Posting a Crash Report post for advice on how to do that.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Add a Comment

Please post a full crash report for this.

Recently we have taken a lot of time to analyze the issue of our app. But this issue only happens on iOS15.4.

With the upgrade of iOS15.4, we're getting more and more crashes reports. We both think it should be a serious system bug.

Refer to the full crash report in the attachment ('...' means I removed some stack of our image).

Does anyone else has met the same issue?

full_crash.log
Posted by
codefei
Up vote reply of codefei Down vote reply of codefei

  • We don't understand why only iOS 15.4 wait for the lock timeout.

    —
    codefei
Add a Comment

Our app also encountered a similar problem. Do you have a solution?

Posted by
aaacsc
Up vote reply of aaacsc Down vote reply of aaacsc
Add a Comment

Same here!!!!!

Posted by
jocelynnnn
Up vote reply of jocelynnnn Down vote reply of jocelynnnn
Add a Comment

Refer to the full crash report in the attachment

Thanks for that.

'...' means I removed some stack of our image

Unfortunately that’s a problem. Posting a Crash Report has advice on how to redact stuff, and I crafted that advice carefully. One weapon in my crash-analysis arsenal is a set of Apple internal tools that does… well… lots of helpful stuff. Those tools need a full Apple crash report. If you redact it in ad hoc ways, I can’t be sure I’ll get valid results.

Please repost your crash report, redacted as per my advice, and I’ll take another look.

Alternatively, if you’re OK with sharing the crash report privately, feel free to email it to me (my address is in my signature). Make sure to include a reference to this thread; I get a lot of email (-:

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Add a Comment

The main thread is waiting for the _pthread_list_lock that is held by thread 82. And Thread 82 seems paused by the JavaScript scavenger thread 2, thread 2 also called pthread_mach_thread_np. In this situation, the _pthread_list_lock will not unlock. The app will be force killed by the watchdog.

By the way, thread 82 is created by the gcd queue. If we do not use the gcd queue will avoid the issue.

Does apple notice the serious bug? Some implementation changes on iOS15.4 may lead to the issue, such as the JavaScript scavenger? Hope apple fix the system issue quickly.

Posted by
codefei
Up vote reply of codefei Down vote reply of codefei
Add a Comment

I had reason to dig into this issue in another context and it seems that this is an issue we’re tracking (r. 89020902). AFAICT this should be fixed in the iOS 15.5b1 (19F5047e) that we’re currently seeding. If you still see the problem on that release, or anything later, please let us know.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Post marked as solved
Add a Comment

@eskimo could you please take a look at a similar issue, possibly related? https://developer.apple.com/forums/edit/question/704718021

Posted by
RuiB
Up vote reply of RuiB Down vote reply of RuiB
Add a Comment

How to avoid this on iOS15.4, we still have a lot user.

Posted by
fengnianji
Up vote reply of fengnianji Down vote reply of fengnianji
Add a Comment

You should advise those users to upgrade to the latest version of iOS 15. This should help with this problem, but it also includes a bunch of important security fixes.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Add a Comment