By setting the ARView's camera mode to nonAR, you are telling the ARView to use a PerspectiveCamera, which is automatically managed by RealityKit *unless* you provide your own PerspectiveCamera.
arView.cameraMode = .nonAR //RealityKit now adds and uses a default PerspectiveCamera
If you want to provide your own PerspectiveCamera, you could do something like this (in addition to setting the cameraMode to .nonAR)
let cameraEntity = PerspectiveCamera()
cameraEntity.camera.fieldOfViewInDegrees = 140 //Custom field of view
let cameraAnchor = AnchorEntity(world: .zero)
cameraAnchor.addChild(cameraEntity)
arView.scene.addAnchor(cameraAnchor)
Your "pointOfView" variable would effectively be the cameraAnchor's Transform in this case, or you could access ARView's cameraTransform property, which always provides the Transform of the active camera in the scene.