JPEGDecompressSurface error when constructing UIImage from raw JPEG data

Question

Created Apr ’19

Replies 5

Boosts 0

Views 2.9k

Participants 2

Hi all,

I am having a problem when constructing a UIImage from raw JPEG data. I transfer JPEG data through the network via UDP so some packets are expected to be lost. I want to be able to detect when the JPEG cannot be decoded properly and not display it in my UI view. If I attempt to display it I get the error:

JPEGDecompressSurface : Picture decode failed: e00002d1

Decoding failed with error code -1

and my image is displayed with some grayed parts (as expected). I do not see any exception thrown, other than the error message. Here is my code:

let image = UIImage(data: imagedata, scale:1.0)
self.imageView = UIImageView(image: image)
self.imageView.frame = self.view.frame
self.imageView.contentMode = .center
self.view.addSubview(self.imageView)

The imagedata are the raw JPEG data that sometimes are incomplete due to unreliable network. It is a streaming application so I am okay with some images to be skipped. I simply want to handle whenever the JPEG decode fails and move to the next image rather than displaying a broken image.

Many thanks in advance!

Dimos

Answered by DTS Engineer in 358385022

I transfer JPEG data through the network via UDP so some packets are expected to be lost. I want to be able to detect when the JPEG cannot be decoded properly and not display it in my UI view.

Do not use this approach. In addition to the problems you’re seeing, it’s deeply insecure. The JPEG decoder is meant to be able to reject corrupted JPEGs, but there’s a long history of security vulnerabilities in decoders like this, where an attacker carefully crafts malicious data to trigger a bug that allows them to hijack your app. You do not want to be encouraging this.

At a minimum you should transfer your JPEG data with a checksum so that you can detect accidental brokenness (like a dropped datagram). However, I strongly advise you to protect this data cryptographically, for example, by using DTLS.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Boost

Answer 1

DTS Engineer OP

Apple

Apr ’19

Accepted Answer

I transfer JPEG data through the network via UDP so some packets are expected to be lost. I want to be able to detect when the JPEG cannot be decoded properly and not display it in my UI view.

Do not use this approach. In addition to the problems you’re seeing, it’s deeply insecure. The JPEG decoder is meant to be able to reject corrupted JPEGs, but there’s a long history of security vulnerabilities in decoders like this, where an attacker carefully crafts malicious data to trigger a bug that allows them to hijack your app. You do not want to be encouraging this.

At a minimum you should transfer your JPEG data with a checksum so that you can detect accidental brokenness (like a dropped datagram). However, I strongly advise you to protect this data cryptographically, for example, by using DTLS.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

0

Answer 2

dimstamat OP

Apr ’19

Thanks a lot, that's really helpful! I'm not planning to release this app, it is only for personal use. I haven't thought about security much but I doubt that someone would be interested in injecting malicious jpeg images in my app. This would be the case if it was a publicly available and popular app. Adding a checksum would require more computational power and would probably drop my frame rate. This looks like a direction of making UDP behave like TCP.

Anyways, adding a checksum or -at least- a counter for the number of packets per JPEG image sounds like a better approach and it will detect the problem earlier, instead of attempting to decode the image no matter what. The other question is why I don't use a video encoding library but I use a sequence of JPEG images to contruct a video stream. I tried with FFMpeg and I was not satisfied with the quality at all. I might be able to tweak the settings and get better results but for now the JPEG images fit perfectly with my needs.

Thanks a lot for the help!

0

Answer 3

Max25 OP

Apr ’19

ios13

0

Answer 4

dimstamat OP

Apr ’19

What about it? 🙂

0

Answer 5

DTS Engineer OP

Apple

Apr ’19

Adding a checksum would require more computational power and would probably drop my frame rate.

Modern machines can calculate checksums way faster than they can transfer data over the network. As with any argument about performance, the acid test is the profiler, but I’m confident that you’ll find that the CPU performance taken up by the checksum won’t even register.

This looks like a direction of making UDP behave like TCP.

I think you’ve misunderstood the key difference between UDP and TCP in a real-time app. The issue is not the performance associated with sending the data, it’s whether the data gets retransmitted if something goes wrong. In TCP you always get that retransmission. In UDP you never get it [1].

This is critical for real-time networking because, when you have a steady stream of, say, video frames travelling over the network, retransmission actually hurts. The retransmitted frame will be too late to be useful, and the retransmission will delay the subsequent frames, making the problem worse.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

[1] If you want retransmission, you have to implement it yourself. This is generally a very bad idea because then you are re-implementing TCP, and it’s hard to beat TCP at its own game. You have to be a very experience network protocol designer to do that.

0