ARKit tracking eyebrows with a unified motion instead of individually.

Question

Created May ’22

Replies 3

Boosts 4

Views 2.1k

Participants 4

Hello, I'm a VTuber, or a content creator who uses a virtual avatar instead of my real life face for presentation. I've been trying to use the iOS implementation of ARKit to get face tracking working, but noticed a major issue, especially for my purposes.

ARKit appears to move both eyebrows the same, regardless of whether or not I am physically moving my eyebrows the same. Every ARKit based application I've used simply mirrors eyebrow movement. Is there a fix to this, or is this just a bug currently in the software?

Boost

Answer 1

banquetburger OP

May ’22

I'm experiencing this issue as well.

I've spent a couple of weeks scouring the internet and asking questions in various places, and it's as though nobody's noticed this or thinks it's a big deal.

I guess 99% of people don't use their eyebrows for emoting?

All I can assume is that the hype around the ARKit face tracking was coming from hobbyists who played with it like a toy, and that it actually has no real usefulness as an actual indie-professional solution.

Happy to be proven wrong, but I'm already in the process of setting up a production pipeline using Faceware instead.

2

Answer 2

Beyatral OP

Feb ’23

I experience it as well. All the software I have tried, none of them have independent eyebrow tracking. I reached out to some devs of those softwares Unreal LiveLink, iFacialMotion 3DFaceMotion, VBridger, Vtube Studio, Waidayo to name a few and they all say it is a problem with the arkit. Some of them even send bug reports. This was around 1 year ago already. I would love to see this fixed, as it would widen the usage of great tracking from the iPhone. All I was able to find out is, that this issue started with iOS 12. I would love more people to point this out, to give this some more attention. Since the movement of the eyebrows are one of the most important body parts to make a variety of expressions possible.

1

Answer 3

korsynx OP

Jun ’24

This issue is still happening. I have tried all the major available apps to do face mocap on the iphone now. They all have the problem of both eyebrows moving at once. We are using iphone 15 PM

It has been 2 years with no response from Apple on this?? I guess apple has given up on face capture.

Our company had been testing the iphone to see if we could do mocap with it. This problem makes iphone unsuitable for our professional mocap needs.. We will have to look for a different face mocap solution

0