How can I grant full disk access to a Node-RED instance?

There are a lot of parts to this. First, check out On File System Permissions, which has a bunch of useful backstory.

Second, we don’t support folks reading the Reminders database directly. I mean, it’s your Mac, so you should go to town, but the location and format of that database is not considered API, so this might break at some point in the future.

Third, you can access reminders through the EventKit framework, and that’s going to be more sustainable approach. It’ll also get you out of the Full Disk Access business.

Finally, on the Full Disk Access front, that issue is complex and it very much depends on how the accessing process is started. I’m not familiar with Node-RED or fswatch, so I can’t offer any insight on that front. The general idea, however, is that you must grant Full Disk Access to the responsible code, which is a term I defined in the above-mentioned post. Tracking down the responsible code can be tricky but, in general, if a process is blocked from accessing something then the responsible code will show up in System Settings > Privacy & Security > Full Disk Access even when no alert is presented.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

This was very informative. Thanks so much for your response. I was able to solve my problem by granting full disk access to /usr/local/bin/node. I'll try and refine that access later, but it does prompt a follow-up question...

My initial naive attempt was to grant access to the script red.js that is what is used to launch Node-RED. I get that the node executable in the shebang at the top is what would need access. That makes sense. I feel pretty dumb for having tried to add the script first. I should have realized that on my first attempt, but I was just being hasty.

However, granting access to that executable means any node script has access, not just Node-RED.

So my next question is, is it theoretically possible to grant access to the specific Node-launched process (e.g. for Node-RED) via the disk access system settings pane, if all I have is a script for creating that child-process? I expect not, but I'd just like to see if you can confirm that.

It goes without saying that me accessing the sqlite files is not supported, though I understand that that's a disclaimer you need to make. Regarding EventKit, I assume it would have the same downsides (that lead me to figure out how to access the DB directly) that AppleScript and Shortcuts has (the main one being that it is prohibitively slow if you have a large reminders database, but also issues with not returning reminder in sub-lists)?

Incidentally, my sqlite queries run in a fraction of a second whereas just opening the Reminders app to get the same info freezes it for what feels like 10 seconds and half the time results in the app crashing, so I just make webhook calls to node-red to make the sqlite query. And if I ever need to change a reminder, I use an applescript that makes select queries and then uses sported AppleScript commands to make the modifications.

Even if EventKit doesn't come with the problems I've had with AppleScript and Shortcuts, I would expect that the learning curve would be a time-prohibitive investment for me, because the closest I've come to Apple app development was code-signing a Java app and messing with containerizing it. I'll keep the EventKit suggestion in my back pocket the next time a dramatic change is made to the reminders architecture. This change to Reminders however seems at the moment to just have been a handful of field changes (at least for the fields I use) and a splitting of the table to store list metadata and reminders data into 2 separate tables, so it didn't take long to make the necessary updates. I'll find out later if I need to make more updates now that I can run my reminder automations.

Thanks again!

granting access to that executable means any node script has access, not just Node-RED.

Correct.

is it theoretically possible to grant access to the specific Node-launched process … if all I have is a script for creating that child-process?

No. TCC tracks access via code signing constructs, and those are properties of the executable. When a process runs an executable, it takes on those properties. So (using the shell here because I’m no not an expert in Noed.js) when you run a shell script the process is running sh (or whatever shell you use) and that’s all that TCC knows about it.

Regarding EventKit, I assume it would have the same downsides

I dunno. The only way to know for sure is to suck it and see.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Savoring those rare moments when I get to disagree with Quinn...

No. TCC tracks access via code signing constructs, and those are properties of the executable. When a process runs an executable, it takes on those properties. So (using the shell here because I’m no not an expert in Noed.js) when you run a shell script the process is running sh (or whatever shell you use) and that’s all that TCC knows about it.

This is correct, but I think you could still set things up so that "/usr/local/bin/node" didn't actually have FDA. The trick here is that access can also be inherited, not just granted directly. This is actually how granting FDA to Terminal.app allows "cp" to copy things it couldn't otherwise copy. Terminal doesn't copy anything, but all of it's child processes inherit it's access level, so now Terminal.app-> tcsh* -> cp means "cp" has FDA as well.

*I'm to old to change.

I think you could do exactly the same thing here- set up a tool that specifically runs that particular script using node, then give THAT new tool FDA. Assuming node's implementation is straightforward, node will then inherit FDA from your tool.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Savoring those rare moments when I get to disagree with Quinn

Hey hey, you should hear our lunchtime conversations!

I think you could do exactly the same thing here

That might work. The algorithm used by TCC to track from a process to its responsible code is both undocumented and complex. Tracing up the parent process chain seems like something it should do, but I’ve had such expectations dashed many times in the past.

Still, you could try it and see what happens.

The nice thing here is that you’re building code just for yourself. If something breaks, you’re the only one inconvenienced. Thus, it’s fine to rely on implementation details. If you were building a product that you shipped to a wide range of users, you wouldn’t have that leeway.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

That all makes sense. There might be an easier solution however. I haven't looked through the code in red.js yet. (I just know that it's that script that launches node-red.) but in the same directory where red.js is, there's a bin dir with 1 executable in it named node-red-pi. With any luck, that's the "node-red executable" and red.js just runs it with params or something.

Regarding developing for just myself... I had thought about putting all this in a github repo, but RN, there are too many disparate and diverse dependencies: node-red flows, a custom Perl module that I use with all my command line scripts that I never released as its own thing (it's the one that was trying to glob the SQLite files that was failing), webhook-relay, multiple shortcuts and Perl scripts, fswatch, a second node instance on a raspberry pi that's involved, an old Wemo motion sensor, a dependency on iCloud Drive in order to synch config files, multiple apple scripts, and a stereo connected to the Mac... and probably a half dozen other things I'm not thinking about. I futz with things now and again to refine it. It's been surprisingly stable for years, given all the parts that could fail.

I kind of think of myself as a sort of doc brown automation (and reminders) hobbyist. Every DIY home automation platform is subject to failure when reverse engineered proprietary systems change, such as in this case. The DIY community is used to that sort of thing. One such tool I developed around 2019-ish was a node red node that used the undocumented Life360 API. A number of other home automation platforms also had their own versions for integrating Life360, and maybe about a year ago now, Life360 essentially cut off public access to their API. The DIY community tried for awhile to work around it to no avail. So we're all used to being beholden to the companies that are trying to make a buck. We all had to discontinue our Life360 interfaces.

I have often thought that I would at least release components of my system, but the only thing that holds me back is how many other plates I have spinning in the air right now. I can't even make time to blog about this stuff on my blog. I was thinking though, that this experience here might be worth a blog post at least.

I did resolve to make a change to the one Perl module that I have been developing since 1999 and never released as its own thing: CommandLineInterface.pm. Someone on stack pointed out that if I provide a specific parameter to the glob method I use for resolving file globs, I would have seen the operation not permitted error. I have been meaning to release that module for years. The only thing that holds me back with that particular component is that since I started it in 1999, there is a ton of poorly written legacy code in it. I started learning Perl in 1999. I've made tons of improvements to it along the way, and added every conceivable test to harden it, but it's still a bit of a mess.

That all makes sense. There might be an easier solution however. I haven't looked through the code in red.js yet. (I just know that it's that script that launches node-red.) but in the same directory where red.js is, there's a bin dir with 1 executable in it named node-red-pi. With any luck, that's the "node-red executable" and red.js just runs it with params or something.

Actually, that did make me think of another broader option here. I don't think we've ever published a formal list, but there are many directories on the system which are inherently NOT protected by the sandbox. For example, the reason sandboxed apps can directly run most command line tools is because the default sandbox includes an exception for the directories those tools are in.

So, the other option would be to put the file inside one of those directories. There a lots of options (including the Unix hiearchy), but I'd probably create my own directory inside "/Library/Application Support/" and use that. Keep in mind that you can still protect it using Unix permissions, so this doesn't mean that you're actually opening in up broader access, you're just using a directory the sandbox doesn't protect.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware