I have a bare-bones Metal app setup where I attach a CAMetalLayer to a window that inherits from a NSWindow with a custom delegate. Everything else is vanilla. I'm also using metal-cpp and metal shader converter.
I'm running into a issue where the application runs fine in the beginning, but once I resize the window, it starts hitching. It turns out that [CAMetalLayer nextDrawable:] frequently (but not always) takes around a full second (plus or minus a few milliseconds) to return once drawableSize has been updated.
I've tried setting allowsNextDrawableTimeout to false which doesn't work; it returns a valid drawable after a second instead of nil. Setting displaySyncEnabled to false reduces the likelihood of this happening to around 50% from 90%+ but does not eliminate it. Setting maximumDrawableCount to 2 or 3 does not seem to make a difference.
By dumping the resource IDs of the returned textures I've noticed something interesting: Before resizing, the layer seems to shuffle between 2 textures or at least 2 resource IDs, but after resizing it starts to create new textures for each returned drawable. Occasionally it seems to reuse a previous resource ID, but it does not seem to have anything to do with whether the method returns quickly or not.
Why does this happen, and how can I fix it? Should I create a new CAMetalLayer when resizing the window instead of updating drawableSize?
Metal Shader Converter
RSS for tagAutomatically convert all of your existing HLSL GPU shaders to Metal.
Posts under Metal Shader Converter tag
4 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
I see some demo show convert HDR video to SDR Pixelbuffer,such AVAssetReader、 AVVideoComposition 、AVComposition 、AVFoundation.
But In some cases,I want to render HDR Pixelbuffer and record video.
AVCaptureSession *session = [[AVCaptureSession alloc] init];
session.sessionPreset = AVCaptureSessionPresetHigh;
AVCaptureDevice *videoDevice = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
if ([videoDevice isVideoHDRSupported]) {
NSError *error = nil;
if ([videoDevice lockForConfiguration:&error]) {
videoDevice.automaticallyAdjustsVideoHDREnabled = NO;
videoDevice.videoHDREnabled = YES; // 开启 HDR
[videoDevice unlockForConfiguration];
} else {
NSLog(@"Error: %@", error.localizedDescription);
}
}
Real-time processing of HDR data requires processing of video frame data (such as filters), ensuring that the processing chain supports 10-bit color depth and HDR metadata. And use imagesBuffer to object tracking, etc.
How to solve this problem?
Is there new API for generating Indirect Commands for the Metal Shader Converter? Is there any example project? I currently use a shader to copy indirect commands. Is there a way to do that with the new Shader Converter pipeline?
Unity 2022.3.33f1
For some reason modifying MeshRenderer material shader SetVectorArray doesn't work on IOS, but it works on android and windows builds!!
I was working on Fog Of War, where I used SimpleFOW by Revision3 it's very simple FOW shader where it manipulates the alpha based on UVs vertices.
This is the FogOfWarShaderControl.cs script
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
namespace SimpleFOW
{
[RequireComponent(typeof(MeshRenderer))]
public class FogOfWarShaderControl : MonoBehaviour
{
public static FogOfWarShaderControl Instance { get; private set; }
[Header("Maximum amount of revealing points")]
[SerializeField] private uint maximumPoints = 512;
[Header("Game Camera")]
[SerializeField] private Camera mainCamera;
private List<Vector4> points = new List<Vector4>();
private Vector2 meshSize, meshExtents;
private Vector4[] sendBuffer;
private MeshRenderer meshRenderer;
private void Awake()
{
Instance = this;
Init();
}
// Initialize required variables
public void Init()
{
meshRenderer = GetComponent<MeshRenderer>();
meshExtents = meshRenderer.bounds.extents;
meshSize = meshRenderer.bounds.size;
points = new List<Vector4>();
sendBuffer = new Vector4[maximumPoints];
}
// Transform world point to UV coordinate of FOW mesh
public Vector2 WorldPointToMeshUV(Vector2 wp)
{
Vector2 toRet = Vector2.zero;
toRet.x = (transform.position.x - wp.x + meshExtents.x) / meshSize.x;
toRet.y = (transform.position.y - wp.y + meshExtents.y) / meshSize.y;
return toRet;
}
// Show or hide FOW
public void SetEnabled(bool on)
{
meshRenderer.enabled = on;
}
// Add revealing point to FOW renderer if amount of points is lower than MAX_POINTS
public void AddPoint(Vector2 worldPoint)
{
if (points.Count < maximumPoints)
{
points.Add(WorldPointToMeshUV(worldPoint));
}
}
// Remove FOW revealing point
public void RemovePoint(Vector2 worldPoint)
{
if (worldPoint == new Vector2(0, 0))
{
return;
}
if (points.Contains(WorldPointToMeshUV(worldPoint)))
{
points.Remove(WorldPointToMeshUV(worldPoint));
}
}
// Send any change to revealing point list to shader for rendering
public void SendPoints()
{
points.ToArray().CopyTo(sendBuffer, 0);
meshRenderer.material.SetVectorArray("_PointArray", sendBuffer);
meshRenderer.material.SetInt("_PointCount", points.Count);
}
// Send new range value to shader
public void SendRange(float range)
{
meshRenderer.material.SetFloat("_RadarRange", range);
}
// Send new scale value to shader
public void SendScale(float scale)
{
meshRenderer.material.SetFloat("_Scale", scale);
}
}
}
And this is the FogOfWar.shader
Shader "Revision3/FogOfWar"
{
Properties
{
_MainTex ("Texture", 2D) = "black" {}
_PointCount("Point count", Range(0,512)) = 0
_Scale("Scale", Float) = 1.0
_RadarRange("Range", Float) = .5
_MaxAlpha("Maximum Alpha", Float) = 1.0
}
SubShader
{
Tags { "RenderType"="Transparent" "Queue"="Transparent" }
LOD 100
ZWrite Off
Blend SrcAlpha OneMinusSrcAlpha
Pass
{
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
// make fog work
#pragma multi_compile_fog
#include "UnityCG.cginc"
struct appdata
{
float4 vertex : POSITION;
float2 uv : TEXCOORD0;
};
struct v2f
{
float2 uv : TEXCOORD0;
UNITY_FOG_COORDS(1)
float4 vertex : SV_POSITION;
};
sampler2D _MainTex;
float4 _MainTex_ST;
float _RadarRange;
uint _PointCount;
float _Scale;
float _MaxAlpha;
float2 _PointArray[512];
v2f vert (appdata v)
{
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
o.uv = TRANSFORM_TEX(v.uv, _MainTex);
return o;
}
float getDistance(float2 pa[512], float2 uv) {
float cdist = 99999.0;
for (uint i = 0; i < _PointCount; i++) {
cdist = min(cdist, distance(pa[i]*_Scale, uv));
}
return cdist;
}
fixed4 frag (v2f i) : SV_Target
{
// sample the texture
fixed4 col = tex2D(_MainTex, i.uv);
i.uv *= _Scale;
if (_PointCount > 0)
col.w = min(_MaxAlpha, max(0.0f, getDistance(_PointArray, i.uv) - _RadarRange));
else
col.w = _MaxAlpha;
return col;
}
ENDCG
}
}
}
Now I create a gameobject called FogOfWar as follows
And then in Unit.cs script and Building.cs script I add the following logic
private Vector3 lastPos;
private void Update()
{
if (lastPos != transform.position)
{
FogOfWarShaderControl.Instance.RemovePoint(lastPos);
FogOfWarShaderControl.Instance.AddPoint(transform.position);
lastPos = transform.position;
FogOfWarShaderControl.Instance.SendPoints();
}
}
Now this gives me the effect of FOW as follows on IOS
where the result should be as follows on other devices
I don't know what causes this to happen only on IOS devices.
The logic works fine on android/windows/Linux/editor but not IOS devices.
So why metal API doesn't support shader set vector array?!