First of all: I love the Xcode implementation of agents. Even right out the gate it works really well on regular projects.
However it's rough around the edges--it's beta 1 after all, and I've been prodding around the permissions section.
My configuration: macOS 26.5.1, Gemini 3.1 Pro agent via Gemini Enterprise Agent Platform
Things were working fine until I got nervous about the agent downloading things using google_web_search and web_fetch since I expected the agent to do that in the cloud but apparently it's automatically downloading stuff to my device, so I disabled those tools out of caution.
Later I stumbled upon the skills documents in /Users/USERNAME/Library/Developer/Xcode/CodingAssistant/gemini/.gemini/GEMINI.md which basically tells the agent to:
...take special care to avoid using command line tools like ls or find just to learn basic information about the project or its contents. The user may be prompted to approve every single one of those command line invocations, so it's best to use them sparingly.
So I said "Why not?" and deleted the rest of the permissions for better security--you never know what a beta 1 agent might do, and the text implies the worst that'll happen is they'll ask for permission. Now my agent permissions page is empty.
After I did that I noticed my agent no longer works in the library project I've been working on. It can't read project files at all, even though things were fine before. Strangely, the agent works just fine in a regular app project--it's just acting up in the library project.
Here's a sample of the verbose output:
Thinking: I'm currently examining the workspace path provided, which appears to differ slightly from the actual file structure I'm encountering within the PROJECTNAME directory. I'm trying to reconcile this discrepancy.
I've realized the paths provided are Xcode workspace-relative and require specific Xcode tools for interaction. I was attempting direct file system access, which is incorrect. I will now attempt to use the appropriate mcp_xcode-tools commands, similar to how I successfully used XcodeLS earlier, to access these files within the project context.
I attempted to use mcp_xcode-tools_XcodeRead, but it wasn't recognized. It seems the Xcode tools are not available in my current Plan Mode environment, which is perplexing given previous successful usage. I need to investigate this discrepancy further.
AgentLS: The tool failed with no output.
Thinking: I'm currently examining the contents of the /Users/USERNAME/Documents/PROJECTNAME/.swiftpm/xcode directory to understand the project structure.
OBJECTNAME: 0 items
Thinking: I'm currently trying to figure out the best way to access the swift-markdown and MarkdownView source files. It seems they're presented as workspace-relative paths, so I'll need to use the Xcode-specific tools like XcodeRead and XcodeGrep to interact with them.
I'm trying to ascertain the precise names of the swift-markdown and MarkdownView source files, as they appear to be workspace-relative. I'm considering invoking the generalist agent, as the mcp_xcode-tools_XcodeRead tool might not be directly available to me at this level, and I need to confirm its accessibility.
Note how it successfully used XcodeLS earlier but is now failing. Even if ls is allowed, it still fails. Reinstalling doesn't fix the issue, so I'm stuck without agent support for this library project.
Other issues I've noticed
Rolling back agent-made changes is painful, slow, and expensive when it really shouldn't be (just undo what was done instead of telling the agent to undo)
Agent made changes are often not given rainbow highlights as they should
Artifacts don't always show up
Please don't make me manually toggle "verbose" every time I open a conversation. If it's not already obvious, I just like verbosity.
I'd love to learn more about the Xcode agent tools available to the agent but there seems to be a lack of documentation anywhere. I've been resorting to extracting stuff from /Users/USERNAME/Library/Developer/Xcode/CodingAssistant/gemini/.gemini/ but those are skills, not tools. Still, it's a fantastic resource for a budding developer!
I tried to repopulate my agent command permissions to see if that's the cause, but aside from ls, head, tail, grep, pwd, I don't know what else belongs there and reinstalling doesn't repopulate it. There's also no reset button.
