Post not yet marked as solved
This is a follow up to feedback FB9144718, which we also discussed at a WWDC21 "Performance, power, and stability" lab session.
Issue Summary
The actual issue we are facing is that our XPC service is not running as fast as we would expect it to be, especially not on Intel machines; somewhat better on M1 machines but still not really good.
After a lot of profilling with instruments, it finally turned out that the problem is caused by our processing getting regularly stopped as our processing thread is being preempted and put on hold for sometimes a tramendous amount of time (up to over 32 ms have been monitored). Even it is preempted for just a couple of ms most of the time, this is still a lot considering that the actual work it would otherwise perform is only in the range of microseconds.
The reason why this is happening is probably caused by the fact that we don't use the XPC service just for processing application messages through the XPC protocol, which we do as well, but also retrieve requests through a mach port from another process.
This causes our thread priority to be dropped down to 4 (see highlighted log line) and that's the reason why we get preempted for so long. The reason why it's not equally dramatic on M1 is that we are not preempted there, instead we are forced to run on the high efficiency cores instead of the high performance ones.
Ideas from the Lab
Other than completely restructuring our entire implementation which is eventually going to happen in the future anyway for Big Sur and newer, we still have to maintain this structure as long as we need to also support pre-Big Sur macOS version, so it would be great to have a less dramatic fix.
Two suggestions were made at the lab:
Change the RunLoopType in the XPC Plist from dispatch_main to NSRunLoop. We tried that but that didn't made any difference.
Add a key ProcessType with the value Interactive to the XPC Plist. This key is not documented for XPC services, only for launchd daemons but we were told it should actually work for XPC services as well. We tried that as well, both, top level as well as adding it to the XPC sub-key but that didn't make a difference either.
Another Idea That Didn't Work
Now that second suggestion made me look up that key in the man page for launchd.plist and what I found there was pretty interesting. Apparently there is a ProcessType value documented as
Adaptive
Adaptive jobs move between the Background and Interactive classifications based on activity over XPC connections. See xpc_transaction_begin(3) for details.
This seems to be our problem. Our XPC service is considered inactive when it processes messages over the mach port. Looking up the documentation of xpc_transaction_begin(3) tells me:
Services may extend the default behavior using xpc_transaction_begin() and xpc_transaction_end(), which increment and decrement the transaction count respectively. This may be necessary for services that send periodic messages to their clients, not in direct reply to a received message.
Using these two messages also frees us from the requirement to enable/disable sudden termination our own as it will automatically be controlled by these two functions as well. Yet even using these two functions to indicate activity doesn't prevent us from being preempted at regular intervals as our priority still drops to priority level 4 while we are still in the middle of processing (haven't called xpc_transaction_end() yet) . We seem to use it correctly though as it correctly disables sudden termination on our behalf as long as our XPC service remain in the active state (it will only receive mach messages for processing while in that state) and also gets re-enabled when we leave the active state again.
Final Thoughts
Also on the man page of xpc_transaction_begin() is written:
The XPC runtime will also automatically manage the service's priority based on where a message came from. If an app sends a message to the service, the act of sending that message will boost the destination service's priority and resource limits so that it can more quickly fill the request. If, however, a service gets a message from a background process, the service stays at a lower priority so as not to interfere with work initiated as a direct result of user interaction.
It looks like this is not working the way we use the XPC service at the moment. Our mach port messages either come from a System Extension (Big Sur and up) or from a root daemon started by launchd (Catalina and below, ProcessType is Interactive and nice value is -10) but apparently these messages cannot boost our XPC service and so it will stay on low prio.
Post not yet marked as solved
Hello,
I want to build a developer tool and I have the following setup in mind. The tool consists of
a macOS app with a GUI
a background service (XPC service?) that performs the "heavy lifting"
and a framework ("Client.framework") that can be used by the developer to build customisations based on the background service. These customisations are supposed to be run as a command line tool, in the swift REPL or as a Playground.
Both the macOS GUI app and the Client.framework are kind of clients of the background service.
I did some experiments with an XPC service hosted by the macos app. During these experiements I was not able to connect to the XPC from the Client.framework (tried in the swift REPL, as a command line tool and a Playground). As soon as I try to open the connection to the XPC, the connection's invalidationHandler is called. Accesing the XPC from within the macOS GUI app ("host" app) works fine.
How can I implement this setup? Is an XPC service the right choice for my background service? Do I need to configure the XPC in a specific way to make it accessible from a framework used in the swift REPL, a command line tool or an Xcode Playground?
Thanks!
steven
Post not yet marked as solved
My standard "native" Mac app uses Catalyst helper. Helper is registered as Login Item via SMLoginItemSetEnabled. I tried to establish XPC communication between these process but I failed.
Is it possible to configure XPC connection between these two apps (mac host and catalyst helper)?
As far as I can understand it is possible to create XPC connection in Catalyst via passing listener endpoint. However how to send this endpoint between helper and host app? NSXPCListener.init(machServiceName:) is not available for Catalyst app.
Post not yet marked as solved
I have an XPC connection that I want to know if I should trust using the following code. Does this look like a workable secure approach?
func connectionIsValid(connection: NSXPCConnection) -> Bool {
let checker = CodesignChecker()
var localCertificates: [SecCertificate] = []
var remoteCertificates: [SecCertificate] = []
let pid = connection.processIdentifier
do {
localCertificates = try checker.getCertificatesSelf()
remoteCertificates = try checker.getCertificates(forPID: pid)
} catch let error as CodesignCheckerError {
NSLog(CodesignCheckerError.handle(error: error))
} catch let error {
NSLog("Something unexpected happened: \(error.localizedDescription)")
}
NSLog("Local certificates: \(localCertificates)")
NSLog("Remote certificates: \(remoteCertificates)")
let remoteApp = NSRunningApplication.init(processIdentifier: pid)
if remoteApp != nil && !remoteCertificates.isEmpty {
let policy = SecPolicyCreateBasicX509()
var optionalTrust: SecTrust?
let status = SecTrustCreateWithCertificates(remoteCertificates as AnyObject,
policy,
&optionalTrust)
guard status == errSecSuccess else {
NSLog("failed evaluating trust")
return false
}
let trust = optionalTrust!
var secResult = SecTrustResultType.invalid
SecTrustGetTrustResult(trust, &secResult)
if(secResult == .proceed || secResult == .unspecified) {
let names = remoteCertificates.map { commonName(cert:$0) }
let validCert1 = ["Apple Development: john.doe(at)example.com (XY12XY12X)", "Apple Worldwide Developer Relations Certification Authority", "Apple Root CA"]
if(names == validCert1) {
NSLog("Found a valid client (fingerprint #1)")
return true
}
let validCert2 = ["Developer ID Application: John Doe (XY13XY13X)", "Developer ID Certification Authority", "Apple Root CA"]
if(names == validCert2) {
NSLog("Found a valid client (fingerprint #2)")
return true
}
return false
} else {
NSLog("Got invalid secResult: \(secResult.rawValue)")
}
return false
}
func commonName(cert: SecCertificate) -> String {
var commonName: CFString?
SecCertificateCopyCommonName(cert, &commonName)
return commonName as String? ?? ""
}
As the description of XPC says that it can be used for inter process communication mechanism, what exactly the inter process means?
Can it be used to create shared memory between any type of processes (ex. Swift Process (App) and any other language process) or it is for strictly App to App (swift to swift) communication?
Post not yet marked as solved
IMPORTANT This post is now retired in favour of official documentation, namely TN3113 Testing and Debugging XPC Code With an Anonymous Listener. I’m leaving the original post here just for the record.
Testing and debugging XPC code can be tricky because there are two processes involved. Imagine you might have an app with an embedded XPC Service. To debug this you have to run two instances of the debugger, one connected to the app and another to the service, and then bounce between them. There is, however, an easier way, a way that allows you to test and debug all of your XPC code in a single process.
The trick is to set up an anonymous XPC listener. To start, tweak your XPC listener abstraction to accept an NSXPCListener instance. For example, say you have a MyListener class like this:
class MyListener {
init() {
self.listener = NSXPCListener.service()
}
let listener: NSXPCListener
… more code …
}
Change the initialiser to look like this:
init(listener: NSXPCListener = .service()) {
self.listener = listener
}
This uses the XPC Service’s listener by default, but allows you to override that by passing a value to the listener parameter.
Now, in your test environment, call the anonymousListener method to create a anonymous listener and pass that to your listener abstraction:
let myListener = MyListener(listener: .anonymous())
On the client side, tweak your XPC connection abstraction to accept an NSXPCConnection instance. For example, say you have a MyConnection class like this:
class MyConnection {
init() {
self.connection = NSXPCConnection(serviceName: "com.example.MyService")
}
let connection: NSXPCConnection
}
Change the initialiser to look like this:
init(connection: NSXPCConnection = .init(serviceName: "com.example.MyService")) {
self.connection = connection
}
This sets up a connection to the XPC Service’s listener by default, but allows you to override that by passing a value to the connection parameter.
Finally, in your test environment, use the init(listenerEndpoint:) initialiser to create a connection to your anonymous listener:
let connection = NSXPCConnection(listenerEndpoint: myListener.listener.endpoint)
let myConnection = MyConnection(connection: connection)
You now have a connection connected to your listener, both running in the same process. This makes it much easier to debug your XPC code. It’s also perfect for unit tests.
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
Hi all,
According to the reference link,
Packaging a Daemon with a Provisioning Profile - https://developer.apple.com/forums/thread/129596
I changed our launchd daemon to run as an .app.
When generating a Provisioning Profile, I added our Development computer (including UUID) into 'Devices'.
But when I ran .app/Contents/MacOS/FamRTServicebig on our macOS Big Sur test environment, this error occurred:
embedded provisioning profile not valid: file:///Library/Application%20Support/test/bin/FamRTServicebig.app/Contents/embedded.provisionprofile
error: Error Domain=CPProfileManager Code=-212 "Provisioning profile does not allow this device." UserInfo={NSLocalizedDescription=Provisioning profile does not allow this device.}
Questions:
Could you let me know how to resolve this error?
Is it necessary to install the Provisioning profile in the test device as well?
Thanks in advance for your help.
I have a xpc module (wrote in Swift using NSXPCConnection), it works well with test application (simple swift test app), my plan is to put this module under XPCService of a DAL plugIn, since the plugin written in C++, I created an Objective-C dylib which response for talking to the XPC, and this dylib using NSXConnection to activate the XPC module(as the Swift test App), it exports C API to the
plugin.
Somehow when application using the plugin, the plugin call the API of the dylib to activate the XPC, but the XPC won't get loaded (from Activity Monitor).
Any idea to get this working? I cannot put XPCService folder inside the Obj-C dylib instead of inside the plugin bundle, and I don't want load the xpc as daemon, want it be loaded as plugin demanded.
Adding Obj-C into current C plugin is pretty headache, is there any C++ version NSXPConnection for this kind of situation?
=======this is the code in the dylib for activating the connection ======
- (int) initXPC
{
connection = [[NSXPCConnection alloc] initWithServiceName:@"com....."];
connection.remoteObjectInterface = [NSXPCInterface interfaceWithProtocol:@protocol(MyProtocol)];
[connection resume];
self->started = false;
NSLog(@"😂 XPC connection setup, result:%@", ((connection == nil) ? @"Failed":@"Succeed!") );
return connection != nil ;
}
Thanks
Steven
Post not yet marked as solved
Can one restrict XPC communications to only occur between processes that share the same code signing identity?
On OS X, I'm looking for a command line which would show the known path of a bundleIdentifier from Launch Services point of view.If there're several bundles with same id, my app keep looking for a trashed one, or copied, and it's hard to troubleshoot without a command like "launchctl .... mybundleId"In swift, I've found: let launchdPathToHelper = NSWorkspace.sharedWorkspace().absolutePathForAppBundleWithIdentifier(bundleIdHelper)This helps me and gives me the answer, but sure there's a command line for that.Thanks!
Post not yet marked as solved
When an app is running on a Mac, it can add a helper app to its bundle, in the Contents/Library/LoginItems.
This app can launch this helper app by running SMLoginItemSetEnabled.
This helper app is then always open by macOS after restarts.
What bothers me is that there is no visibility on all these apps. They do not appear in any of the LaunchDaemons/LaunchAgents directories, nor in the System Preferences/Users/LoginItem list.
So I have a few questions :
are strictly all apps started in such a way XPC services ?
how can I list all such helper apps? Running "launchctl list" seems to list all processes, not simply all such processes, in spite of the help description "Lists information about services". This description therefore appears incorrect to me, am I missing something?
How can I prevent these apps from being launched at login/startup? Is "launch unloads" the right way to go about it?
Thanks
Post not yet marked as solved
No matter what I try to do to quit/kill my XPC service while its processing a message, my client only ever gets XPC_ERROR_CONNECTION_INTERRUPTED and never XPC_ERROR_TERMINATION_IMMINENT. To be specific, I tried all these things:
Quitting the service while it's idle (SIGTERM)
Force-quitting the service while it's idle (SIGKILL)
Quitting the service while it's in the middle of processing a message (SIGTERM)
Force-quitting the service while it's in the middle of processing a message (SIGKILL)
Making the service quit itself in the middle of processing a message (exit(EXIT_SUCCESS))
Making the service cancel the connection in the middle of processing a message (xpc_connection_cancel(connection))
Hoping to find some examples of its usage, I searched around GitHub, and haven't really found anyone do much at all in response to this message. Some parts look like they come up from a common ancestor (probably some guide or template): https://github.com/search?q=%22The+client+process+on+the+other+end+of+the+connection+has+either%22&type=Code
Even Apple's own WebKit2 doesn't seem to do much with it: https://github.com/apple-opensource/WebKit2/search?q=XPC_ERROR_TERMINATION_IMMINENT
When does XPC_ERROR_TERMINATION_IMMINENT ever come into play?
Post not yet marked as solved
Hi all, i am kind of new in apple, and I am developing an app that fetch realtime data from a market, and display in the graphical macOS X app.
The thing is, is was thinking in separate the app in pieces, to prevent crashes from the network, and I was thinking create a XPC service that manage the network connection to the realtime data service, but reading, looks like the xpc can be shutdown by the system if resources are need. There is a way to prevent this?
XPC service should not be the natural way to implement this? or there is another way
Thanks in advance
Hi!I have a launchd agent that requires Screen Recording permissions in Catalina.Below is the simple example:#!/bin/bash
/usr/sbin/screencapture $FILEIf the script is run from a terminal, permissions are required for Terminal.appBut if the script is launched from launchd, the User prompt doesn't appear.I found some workarounds to get it:1.#!/bin/sh
/usr/sbin/screencapture $FILE2.#!/bin/bash
exec /usr/sbin/screencapture $FILENow in Catalina, Sh is just a wrapper (man sh), which "re-execing" scripts in one of the supported shells.In the first case, permissions are requested for Sh.In the second case, "exec" causes User prompt for Bash.If the script runs another script that launches screen capture, then "exec" helps only if it used in the first script, and the second script uses a different shell:#!/bin/bash
exec secondScript.sh
...
#!/bin/zsh
/usr/sbin/screencapture $FILEOther options for changing the shebang or "exec" position do not cause User prompt.I also have a launchd agent in Mojave that requires Accessibility permissions. It contains a startup sequence of scripts and the target app. Permissions are required for the first script (by name, not for shell), and TCC.db contains csreq of this script, which means that only this script has permissions.In Catalina, the permissions required by this script are saved in TCC.db as permissions for shell, which means that ANY shell script can now use them.My questions are:Why only Catalina's /bin/sh or "exec" in the first script causes User prompt?Is there a proper way to get permissions without a workaround?Is there a way to get permissions only for my script (as in Mojave), not for shell?
Post not yet marked as solved
Hi, I was wondering if there's any limitation for the context where I initialize my xpc service.
This is the code that initialize my xpc service :
listener_ = [[NSXPCListener alloc]
initWithMachServiceName:@"com.bla.bla"];
xpcService *delegate = [xpcService new];
listener_.delegate = delegate;
[listener_ resume];
[[NSRunLoop mainRunLoop] run];
Doing it from the main method and everything works just fine. However, when calling it from different method(main)/thread(main thread)... It doesn't accept remote calls although it seems like the listener was properly initialized.
I even tried to wrap this code to run on the main thread using the following wrapper
dispatch_sync(dispatch_get_main_queue(), ^{
listener_ = [[NSXPCListener alloc]
initWithMachServiceName:@"com.bla.bla"];
xpcService *delegate = [xpcService new];
listener_.delegate = delegate;
[listener_ resume];
}
where the [[NSRunLoop mainRunLoop] run]; is called from the main method...
So my question is what are the requirements to make the XPC work.. is it mandatory to call it from the main method ?
I don't want the XPC process to be killed and restarted
Post not yet marked as solved
Software Update just hangs on No keys matching () available for download if we run it as root. neither returning success nor failure on MacOS Monterey (OS Version 12.0.1)
SUOSUNotificationManagerController having XPC issue saying No Such Process.
faced this issue in normal XPC connection also. root process trying to connect user process throws the same error
when it will be fixed. is there any work around for root to install via software update
log trace:
ues-mac-testl softwareupdate[2710]: SUOSUNotificationManagerController: Error connecting to notification manager service to set authorization: Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named com.apple.SoftwareUpdateNotificationManagerService was invalidated: failed at lookup with error 3 - No such process." UserInfo={NSDebugDescription=The connection to service named com.apple.SoftwareUpdateNotificationManagerService was invalidated: failed at lookup with error 3 - No such process.}
Post not yet marked as solved
is there any public API or Method to get resident size of current process of game like Debug Gauges to Monitor Memory?As far as i know someone use XCode instrument -> show the Debuger navigator -> Memory to get it, before i have found some API to get itfrom internet,but a big differece bettween with the result of XCode Debuger navigator .the first method like this: struct mach_task_basic_info info;
mach_msg_type_number_t count = MACH_TASK_BASIC_INFO_COUNT;
if (task_info(mach_task_self(), MACH_TASK_BASIC_INFO, (task_info_t)& info, &count) == KERN_SUCCESS)
{
int32_t _pss = (int32_t)info.resident_size / (1024 * 1024);
}another method like this: task_vm_info_data_t vmInfo;
mach_msg_type_number_t count = TASK_VM_INFO_COUNT;
kern_return_t kernelReturn = task_info(mach_task_self(), TASK_VM_INFO, (task_info_t) &vmInfo, &count);
if(kernelReturn == KERN_SUCCESS)
{
int32_t _pss = (int32_t) vmInfo.phys_footprint / (1024 * 1024);
}someone discuss:https://github.com/aozhimin/iOS-Monitor-Platform/issues/5a big differnece bettween the result of the first method and the result of XCode Debug navigator instrument, info.resident_size will not increase When the memory is allocated continuously,but xcode Debug navigator will increase.but a little difference bettween the result of the second method and the result of XCode Debug navigator instrument when use game test,but application app will same with it. so i want to know how XCode Debug navigator to calculate Memory or how to get resident size of current process more precise,any idea will help me,thanks in advance!
I apologize for attempting this monstrosity, but... It is forced on me.
I have a .jar implementing logic that I receive prebuilt. This .jar exports some API you can use (... in Java...) to request information about file-system items you hand in as paths.
This .jar is compatible with Java 11, and runs with the "Zulu" Java VM.
I need to "wrap" it in some way that will behave like a global daemon, launched and managed by launchd, available to other components (apps, user-agents and global-daemons) in our product.
Running it as a global daemon is a breeze - simply place a .plist in /Library/LaunchDaemons/myMonster.plist and have the java -jar <path to my .jar> arguments, throw in a 'keep alive' option, and it runs.
Now... It makes sense for other components to pass it "queries" and receive "results" via XPC connection. First, because this way I could deny connection to unknown components, I could have a secure connection, and also integrate nicely from the other components ObjC code.
However... XPC isn't something available in JDK, and the actual executable launched is the Zulu java command binary of course, that I can't modify.
So... I tried to use JNA (Java Native Access) and (with much tears and sweat) get my java code to create an xpc connection (as client! not "service") to another XPC Service already running.
Also, I was only able to do it via xpc.h (the old C API. not ObjC NSXPCConnection as of yet).
The documentation on old C-style XPC Services is very thin, almost nothing... and the only thing about Creating an XPC Service of this style says:
// If you are writing a low-level (C-based) XPC service, implement a minimal main function to register your event handler, as shown in the following code listing. Replace my_event_handler with the name of your event handler function.
int main(int argc, const char *argv[]) {
xpc_main(my_event_handler);
// The xpc_main() function never returns.
exit(EXIT_FAILURE);
}
Which of course, I can't do! I don't control the process 'main()' entry point (it's java...) and I can't block anything.
So here's my question:
Where in the lifecycle of a Java program can I call on my native code to set up The XPC Service?
Is there a non-blocking alternative to xpc_main(my_event_handler) I can use to start my service?
I know this calls for multi-disciplinary knowledge but any hint or idea will be greatly appreciated.
Thanks!
Post not yet marked as solved
i want to sideload kodi on apple tv gen4. i need a free developer account to do so.is this possible? If so, how?Thank you.
xcoder112.
