You're likely using Blender to make your USDZ files which by default doesn't export USDZ files that can be referenced in. Alternatively, you may be using some tool to make them that puts the Materials outside the hierarchy of the default prim.
If you are using Blender, you can fix this by changing the Root Prim setting in blender to /root (include the forward slash before root). This will be the default in Blender 4.1 onwards.
If you are using another tool to make the USDZ files, you need to make sure that all your prims are under the defaultPrim of the USD, otherwise only the contents of the defaultPrim will be referenced in, resulting in broken material assignments.