I write iOS plug in to integrate MetalFX Spatial Upscaling to Unity URP project. C# Code in Unity: namespace UnityEngine.Rendering.Universal { /// /// Renders the post-processing effect stack. /// internal class PostProcessPass : ScriptableRenderPass { RenderTexture _dstRT = null; [DllImport ("__Internal")] private static extern void MetalFX_SpatialScaling (IntPtr srcTexture, IntPtr dstTexture, IntPtr outTexture); } } void RenderFinalPass(CommandBuffer cmd, ref RenderingData renderingData) { // ...... case ImageUpscalingFilter.MetalFX: { var upscaleRtDesc = tempRtDesc; upscaleRtDesc.width = cameraData.pixelWidth; upscaleRtDesc.height = cameraData.pixelHeight; RenderingUtils.ReAllocateIfNeeded(ref m_UpscaledTarget, upscaleRtDesc, FilterMode.Point, TextureWrapMode.Clamp, name: "_UpscaledTexture"); var metalfxInputSize = new Vector2(cameraData.cameraTargetDescriptor.width, cameraData.cameraTargetDescriptor.height); if (_dstRT == null) { _dstRT = new RenderTexture(upscaleRtDesc.width, upscaleRtDesc.height, 0, RenderTextureFormat.ARGB32); _dstRT.Create(); } // call native plugin cmd.SetRenderTarget(m_UpscaledTarget, RenderBufferLoadAction.DontCare, RenderBufferStoreAction.Store, RenderBufferLoadAction.DontCare, RenderBufferStoreAction.DontCare); MetalFX_SpatialScaling(sourceTex.rt.GetNativeTexturePtr(), m_UpscaledTarget.rt.GetNativeTexturePtr(), _dstRT.GetNativeTexturePtr()); Graphics.CopyTexture(_dstRT, m_UpscaledTarget.rt); sourceTex = m_UpscaledTarget; PostProcessUtils.SetSourceSize(cmd, upscaleRtDesc); break; } // ..... } Objective-c Code in iOS: head file: #import <Foundation/Foundation.h> #import <MetalFX/MTLFXSpatialScaler.h> @protocol MTLTexture; @protocol MTLDevice; API_AVAILABLE(ios(16.0)) @interface MetalFXDelegate : NSObject { int mode; id _device; id _commandQueue; id _outTexture; id _mfxSpatialScaler; id _mfxSpatialEncoder; }; (void)SpatialScaling: (MTLTextureRef) srcTexture dstTexure: (MTLTextureRef) dstTexture outTexure: (MTLTextureRef) outTexture; (void)saveTexturePNG: (MTLTextureRef) texture url: (CFURLRef) url; @end m file: #import "MetalFXOC.h" @implementation MetalFXDelegate (id)init { self = [super init]; return self; } static MetalFXDelegate* delegateObject = nil; (void)SpatialScaling: (MTLTextureRef) srcTexture dstTexture: (MTLTextureRef) dstTexture outTexture: (MTLTextureRef) outTexture { int width = (int)srcTexture.width; int height = (int)srcTexture.height; int dstWidth = (int)dstTexture.width; int dstHeight = (int)dstTexture.height; if (_mfxSpatialScaler == nil) { MTLFXSpatialScalerDescriptor* desc; desc = [[MTLFXSpatialScalerDescriptor alloc]init]; desc.inputWidth = width; desc.inputHeight = height; desc.outputWidth = dstWidth; ///_screenWidth desc.outputHeight = dstHeight; ///_screenHeight desc.colorTextureFormat = srcTexture.pixelFormat; desc.outputTextureFormat = dstTexture.pixelFormat; if (@available(iOS 16.0, *)) { desc.colorProcessingMode = MTLFXSpatialScalerColorProcessingModePerceptual; } else { // Fallback on earlier versions } _device = MTLCreateSystemDefaultDevice(); _mfxSpatialScaler = [desc newSpatialScalerWithDevice:_device]; if (_mfxSpatialScaler == nil) { return; } _commandQueue = [_device newCommandQueue]; MTLTextureDescriptor *texdesc = [[MTLTextureDescriptor alloc] init]; texdesc.width = (int)dstTexture.width; texdesc.height = (int)dstTexture.height; texdesc.storageMode = MTLStorageModePrivate; texdesc.usage = MTLTextureUsageRenderTarget | MTLTextureUsageShaderRead | MTLTextureUsageShaderWrite; texdesc.pixelFormat = dstTexture.pixelFormat; _outTexture = [_device newTextureWithDescriptor:texdesc]; } id upscaleCommandBuffer = [_commandQueue commandBuffer]; upscaleCommandBuffer.label = @"Upscale Command Buffer"; _mfxSpatialScaler.colorTexture = srcTexture; _mfxSpatialScaler.outputTexture = _outTexture; [_mfxSpatialScaler encodeToCommandBuffer:upscaleCommandBuffer]; // outTexture = _outTexture; id textureCommandBuffer = [_commandQueue commandBuffer]; id _mfxSpatialEncoder =[textureCommandBuffer blitCommandEncoder]; [_mfxSpatialEncoder copyFromTexture:_outTexture toTexture:outTexture]; [_mfxSpatialEncoder endEncoding]; [upscaleCommandBuffer commit]; } @end extern "C" { void MetalFX_SpatialScaling(void* srcTexturePtr, void* dstTexturePtr, void* outTexturePtr) { if (delegateObject == nil) { if (@available(iOS 16.0, *)) { delegateObject = [[MetalFXDelegate alloc] init]; } else { // Fallback on earlier versions } } if (srcTexturePtr == nil || dstTexturePtr == nil || outTexturePtr == nil) { return; } id<MTLTexture> srcTexture = (__bridge id<MTLTexture>)(void *)srcTexturePtr; id<MTLTexture> dstTexture = (__bridge id<MTLTexture>)(void *)dstTexturePtr; id<MTLTexture> outTexture = (__bridge id<MTLTexture>)(void *)outTexturePtr; if (@available(iOS 16.0, *)) { [delegateObject SpatialScaling: srcTexture dstTexture: dstTexture outTexture: outTexture]; } else { // Fallback on earlier versions } return; } } With the C# and objective code, the appear on screen is black. If I save the MTLTexture to PNG in ios plug in, the PNG is ok(not black), so I think the outTexture: outTexture write to unity is failed.

I have tested SpatialUpsacling with a Unity URP sample project: https://github.com/mao-test-h/MetalFXSamples. I used iPhone 13 with iOS Beta 7. The performance and quality are both worse than Native.

I use Macmini with MacOS Ventura 13.3.1, while the Mac running MetalFX sample code, and choose Temporal Scaler, makeTemporalScaler return nil value, and print "The temporal scaler effect is not usable!". If i choose SpatialScaler, it is ok. guard let temporalScaler = desc.makeTemporalScaler(device: device) else { print("The temporal scaler effect is not usable!") mfxScalingMode = .defaultScaling return } Sample code: https://developer.apple.com/documentation/metalfx/applying_temporal_antialiasing_and_upscaling_using_metalfx?language=objc

In my game project, there is a functions.data file in then /AppData/Library/Caches/[bundleID]/com.apple.metal/functions.data, when we reboot and launch the game, this file was rest to about 40KB, normaly this file's is about 30MB, this operation was done by the metal, Is there any way to avoid it?

Hi Apple, we have purchased Silicon-based Macs at our studio and intend to use Unreal Engine with it, Epic games said it's left to you to allow this on the hardware, while I also learnt that the M chips allow ray-tracing. Are you in talks with Epic Games? or should we expect this feature soon? if so, how soon? or will it not be possible on these present Silicon-based Macs? An answer would really point our company in the right direction. Thank you very much.

Is there a reference for how much time should MetalFX temporal antialiasing + upscaling take? On a M1 Pro, for a test frame with an otherwise extremely short render time (see attached image, render takes ~600µs), the upscaler adds an additional ~10ms just for Metal FX temporal antialiasing + upscaling (see attached image). This obviously leaves very little time to keep the frame time under 16ms (60fps) and makes 8ms frame times (120fps) impossible. The upscaling is working with an input render size of 1,904x1,370 and output size of 2,856x2,055, so it's not like the image is being upscaled to an outrageous size or anything. Any thoughts? Is this to be expected? What can affect the Metal FX Temporal Upscaling frame time?

It would be quite helpful if there was frame interpolation in MetalFX, similar to what NV and AMD are adding to their upscaling technologies. I have found MetalFX temporal upscaling critical to a real-time ray traced application, and frame interpolation could halve the compute cost, letting me try more expensive RT algorithms. I am hoping this technology will debut at WWDC 2023, if not then maybe during the winter round of OS updates.

I just upgraded to Xcode 14.3. I have started seeing the following debug message show up in my console and I am not sure why. [GPUDebug] Null texture access executing kernel function "ColorCorrection" encoder: "Keyframing.SurfaceWarpFuser.InverseWarpKeyframe", dispatch: 2 It seems Metal related, but I am very confused by it. My project uses a very minimal amount of Metal. Only to get depth data from ARKit and to draw points, and I definitely do NOT have any kernel functions named ColorCorrection or an encoder named "Keyframing.SurfaceWarpFuser.InverseWarpKeyframe" I haven't changed any of my metal code, so I don't know if this is something bigger I should be concerned about. Sincerely, Stan

I recently learned that in Windows there is a way to force older apps to scale up so they are readable. I learned this when I installed Adobe Encore CS6, which is from 2012, and when launched as is in a 4K monitor that is set to 5k resolution, it looks so tiny and almost impossible to read. So searching for a solution I came upon a setting in the compatibility tab that forces the program to scale up as if it were designed for HiDPI. Of course this is a very basic zoom into the program's GUI, so the text looks a bit blurry, but perfectly readable. I know macOS doesn't have a setting like this, but I was wondering if it would be possible to edit any file in the package for the old app to force this, like adding something to the info.plist file, or editing something else inside the package in Xcode. My main interest in this case is for all the older apps in the Native Instruments Komplete 9 package, a huge set of virtual instruments from 2014, many of which still have great synths and sampled instruments, but while NI updated them to work with current versions of DAWs like Logic Pro X, they haven't touched the GUI, so when I use those, I have to constantly press Control and swipe the Magic Mouse up and down to zoom in the while macOS GUI. Please note that what I'm asking might be totally ridiculous, I'm not a developer, I figured this would be a good question for developers that know XCode or other development tools. I bet that Quartz, Metal, or whatever macOS Ventura uses for GUI scaling would do a much better job than Windows.

I updated to MacOS 13 Ventura, and wanted to try MetalFX, downloaded the sample code, where there is only a swift example even if I select Obj-C, which is very strange, but when I try to run it it says, no module found MetalFX, I went to the framework folder and there is indeed no MetalFX.framework folder. You know how I could fix this, and how I could get a the sample code for Obj-C? Quick help would be appreciated 😊

Hi, trying to wrap my head around Xcode's FXPlug. I already sell Final Cut Pro titles for a company. These titles were built in motion. However, they want me to move them to an app and I'm looking for any help on how to accomplish this *What the app should do is: Allow users with an active subscription to our website the ability to access titles within FCPX and if they are not an active subscriber, for access to be denied.