Memory Overhead Issues with ESF Framework During High Volume of NOTIFY Events

Question

Created Dec ’23

Replies 4

Boosts 0

Participants 2

I refer to Google's ESF project code to handle NOTIFY events, but after my notifyQueue is set to QOS_CLASS_BACKGROUND, the process memory gets larger when a lot of events occur. Is there any way to fix this without affecting performance

The code is as follows：

notifyQueue = dispatch_queue_create("notify",dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_CONCURRENT_WITH_AUTORELEASE_POOL,QOS_CLASS_BACKGROUND, 0));

if (msg->action_type == ES_ACTION_TYPE_NOTIFY) {
	es_message_t *copied_msg = [self copy_message:msg];
	if (!copied_msg) {
		return;
	}
	dispatch_async(self->_notifyQueue, ^{
		@autoreleasepool {
			[self handle:copied_msg];
			[self free_message:copied_msg];
		}
	});
}

Boost

Answer 1

DTS Engineer OP

Apple

Jan ’24

after my notifyQueue is set to QOS_CLASS_BACKGROUND

Don’t do that. ES clients are expected to handle events promptly. The .background QoS is intended to be useful for optional stuff, like optimising a database. There are situations where our systems might defer .background QoS work indefinitely [1]. Given that, it’s completely inappropriate for an ES client.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] The example that springs to mind is iOS, where Low Power Mode will stop all .background QoS work completely.

0

Answer 2

c4m3l OP

Jan ’24

What QoS type should I set for the NOTIFY event

0

Answer 3

DTS Engineer OP

Apple

Jan ’24

Something higher than QOS_CLASS_BACKGROUND (-:

Seriously though, there isn’t an easy answer to this. Creating a reliable ES client and is hard, and the naïve code you’ve posted is unlikely to work. Specifically:

You copy every message you get, which is going to increase memory and hurt performance.
You pass that copy to a block run on a queue, which means it won’t be freed until the queue runs the block.

The viability of this approach depends on the number of events you expect to receive. If you’re subscribed to a tiny number of events, it’s probably fine. If you’re dealing with a lot of events, it’s just not going to work.

So, what sort of events are you monitoring?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0

Answer 4

c4m3l OP

Jan ’24

Thank you for your reply, I think I understand what you said, I will optimize my code according to your explanation, thank you for your help, your answer is very helpful to me. (-:

0