How can a local AI agent use MLX/Metal unattended while remaining confined to an authorized workspace?
I am developing an AI-driven local media-processing workflow on an Apple-silicon Mac and am trying to understand the correct architecture for allowing it to run unattended without giving the AI agent unrestricted access to my primary personal computer.
I am not a software engineer, so I may be missing an established macOS mechanism or using the wrong terminology. I would appreciate guidance from people familiar with MLX, Metal, sandboxing, and macOS security.
What I am building
I use OpenAI Codex as the local execution/software-development agent.
The working system currently:
ingests and verifies original video and still media while preserving immutable originals;
performs visual semantic analysis and divides video into meaningful time-coded segments;
separately analyzes spoken language rather than assuming audio and video are semantically equivalent;
uses MLX Whisper locally on Apple silicon for time-coded speech transcription;
stores visual and language semantics in a relational SQLite media catalog.
These five stages are working. My current test corpus contains 148 original media files, 126 visual semantic segments, and 765 speech segments.
The next stages are AI editorial construction from the semantic database and generation of instructions/scripts for a DaVinci Resolve rough cut.
The security architecture
I want Codex to operate autonomously within a deliberately bounded development environment.
I do not want to solve this simply by granting an autonomous agent Full Disk Access to my primary personal Mac. The concern is ordinary fault containment. Codex generates and executes scripts, invokes applications and command-line tools, and manipulates files. A mistaken path or defective generated script should not have unrestricted consequences for the rest of my computer.
I therefore separated AI execution from ordinary personal files.
Codex is configured for Workspace Write access with explicitly authorized project roots. Canonical media resides on a separately authorized external SSD, and temporary AI working artifacts are kept separately.
Ordinary Python and FFmpeg operations now run autonomously within these authorized areas.
The problem
The difficulty appears when the workflow invokes capabilities that cannot operate inside the ordinary Codex sandbox.
The clearest example is MLX Whisper.
I am using:
MLX Whisper 0.4.3
mlx-community/whisper-small-mlx
Apple silicon
local transcription
MLX Whisper works successfully and its transcription quality is sufficient for my semantic-retrieval application.
However, MLX could not access Apple Metal/GPU execution from inside the ordinary Codex sandbox. Codex therefore requested permission to execute the transcription operation outside the sandbox.
Once approved, MLX/Metal worked and the entire corpus was successfully transcribed.
The processing therefore works, but the workflow cannot run genuinely unattended.
A future operation should be able to run:
new media
→ integrity verification
→ visual semantic analysis
→ MLX Whisper transcription
→ language semantic analysis
→ SQLite update
→ QA
But if execution stops midway waiting for a human to click Allow, the pipeline is not operationally autonomous.
What I have already tried
I initially encountered permission problems even with ordinary file operations. I therefore:
separated Codex work from ordinary personal documents;
created dedicated project/work areas;
explicitly authorized the required working roots;
configured Workspace Write;
separately authorized the external media repository;
tested shell/Python and FFmpeg operations within those boundaries.
Those changes worked. Routine Python and FFmpeg operations now run without approval prompts.
The remaining issue occurs with MLX/Metal and some other application/runtime operations that require sandbox escalation.
My question
Is there a supported architecture for allowing a local AI agent to invoke MLX/Metal and other deliberately authorized development tools unattended, while still confining the agent to defined project/workspace boundaries rather than granting unrestricted access to the entire Mac?
For example, should I be investigating:
App Sandbox entitlements;
a signed helper tool or XPC service;
security-scoped resources;
a dedicated executable with appropriate entitlements;
a different method of launching MLX/Metal;
or another macOS mechanism?
In particular, can Metal/GPU access coexist with persistent bounded filesystem access without requiring interactive approval each time the AI invokes it?
I am also unsure which security layer is actually responsible here: the Codex sandbox, macOS App Sandbox, TCC, executable/code-signing rules, Metal restrictions, or some interaction among them.
If this kind of bounded unattended execution is intentionally not supported, that would also be useful to know. My alternative would be a dedicated Apple-silicon Mac containing only the AI-development environment and replaceable project data, where broader permissions would have a much smaller failure domain.
I can provide the Codex configuration, exact successful and failing commands, directory/root configuration, macOS/hardware information, and sandbox diagnostics.
I would particularly appreciate guidance on which security layer is causing the MLX/Metal escalation and what the supported architecture would be for this use case.
Thank you.