Well, +[NSWindow windowNumberAtPoint:belowWindowWithWindowNumber:] sort of does this. The problem is that it gives you a window number and there are few APIs that let you do something meaningful with a window number, especially for windows not owned by the calling app. There's also the CGWindowList API (a.k.a. Quartz Window Services). It can return information about windows, including a window number. To my knowledge, there's no documented correspondence between these two notions of window number, but they are the same.
CGWindowList can also provide the process ID of the process which owns the window. You can use that to activate the owning app. However, you can't easily put focus on a specific window within the app.
The other API to investigate is the Accessiblity API. That's designed for assistive devices and software, and focus follows mouse could be thought of as an assistive feature of sorts.