iPhone 7 Plus frame rate slow down

We have a camera based application that uses multple threads and can be computationally intensive. What we are seeing is that behaves distinctly differently with unmodified code on iPhone 7 devices than it does on an iPhone 6S with all devices running iOS 10.0.2. What we have observed from timing various parts of the code using the system trace additions to iOS 10, is that for example around the CADisplayLink callback are the following numbers.


Process / Name Count Duration Min Duration Avg Duration Std Dev Duration Max Duration

7 Plus DisplayLink 585 9.49 s 1.41 ms 16.22 ms 4.08 ms 42.14 ms

7 Plus QT DisplayLink 355 954.66 ms 1.78 ms 2.69 ms 666.53 µs 6.58 ms

6S DisplayLink 541 2.86 s 4.18 ms 5.29 ms 968.72 µs 13.37 ms


We are using 1080p60 images from the camera.


Note that the average duration is much higher on the 7 Plus than it is on the lower powered 6S, as well as the max duration values. The other thing to note and this seems significant is that when you attach QuickTime to the iPhone 7 Plus things improve dramatically. This and other numbers we recorded show that when we attach QuickTime (QT) all the times are approximately 5 times faster. There is some internet traffic about the little core on the iPhone 7 being 5 times slower than the big core.


We have tried to adjust the QOS settings on the threads, to make sure they all use QOS_CLASS_USER_INTERACTIVE when appropiate. We have also increased the thread priority to the maximum value for the pthread scheduling. This does help what we see, but it still does not compare to the performance we see when QT is attached to the phone. We don't need these changes on the 6S with a less powerful cpu. What is it about the 7 Plus that requires us to do this?


Other observations the cpu time taken by mediaserverd is about 17% on the iPhone 6S, on the iPhone 7/7 Plus it can be as high as 70%, even when using the Apple camera app. We thought this may be impacting our application, and its ability to get time on the cpus.


The questions we have are,


  • Has anyone else observed this with their applications?
  • Why is it that attaching QuickTime to the iPhone 7 Plus boosts the performance by up to five times. Is it because it magically forces everything to run on the big core out of the big little pair?

    What can we do in our code to have the same effect as attaching QuickTime.

Why has the cpu usage gone up on the mediaserverd on the 7 devices?


Many thanks
Dave

We have not observed this, but would love to have a piece of sample code that reproduces the slowdown. Could you please file a bug at bugreport.apple.com and then come back here and post the bug number?


Thanks.

Hi,


Sorry it took us so long to submit a bug, we were working on trying to provide a reproducible case with sample code. We could not make that work outside of the more complicated environment of the library. I have provided I hope enough information to reproduce what we see on the bug report.


The number is

29003309


Thanks

Dave

Hi Dave,

I've the same problem on the app I'm developing. It works on all devices except on the iPhone 7 Plus where it often drops frames massively.

Our app get video from the camera, makes some video processing with OpenGL and writes the result to disk thanks to an AVAssetWriter. We also process and record audio in parallel.

By analysing with Instruments, we saw the mediaserverd making some cpu load peaks that result in a very high system load. At that time, everything takes a lot of time to be executed, as in the measurment you did. These load peaks happen sometimes, maybe once each 5 to 10 seconds and go on for 1 or 2 seconds.

The same app, running on iPhone 6S Plus, has a quite constant cpu load and no lost frame.

What do you mean by "when you attach QuickTime". What attaching means for you?

Are you also writing media to disk?

Are you also processing audio?

Thanks,

Gabe

After some additional investigation, it seems that:

- The problem appears on all iPhone 7 and 7 Plus.

- Sometimes the mediaserverd process is taking a lot of cpu and the entire system is overloaded. At that time, everything seems to take a lot of time to execute but it's just because the CPU is less available.

- The CPU load is not constant, event if we do the same video processing on all frames.

- The system is less "real-time" or less deterministic than on previous iPhones.

I submitted bug 29419010.

Thanks Gabriele,


Interesting that you found a similar issue. I would love to know if you hear anything back from Apple.


When I say attach QuickTime if you launch quick time on the Mac then select File->New Movie Recording near the drop down on the record button you can select your iOS device, and see its screen on the Mac. If you do this then we don't see the frame stuttering.


We aren't writing any media to disk and we also aren't processing any audio.


Cheers

Dave

I noticed this same issue on an Application I'm working on.


iPhone 7 takes 70% cpu where my iPhone 6 only uses around 15% cpu while running my app.


* background threads processes images using GCD.

* images are passed to AVAssetWriter to output m4v files.


On the process thread, the iPhone 6 can handle an image in approximately 6ms. On the iPhone 7 it takes 10 or more.

On the 6, All frames are recorded. On the 7, after a few frames are recorded frames begin to be dropped as there are new new pixel buffers available or it hasn't finished writing the previous image.


Anyway, just thought I'd ping this issue to see if any of you had additional input on what might need to be changed.


Additional FYI I downloaded a benchmark app and ran cpu/gpu benchmark on both phones. It seems to perform inline with results collected from other phones and definitely has way higher numbers than the 6.

The APP of my development team faces the same thing.Are all still in this situation?

Facing the exact same issue, now with iOS 11 betas too (currently on beta 5).


Anyone found any workaround?

iPhone 7 Plus frame rate slow down
 
 
Q