1. Yes. The delegate method is implemented. And the look of the group item and the presence of the Show | Hide switch button confirm it's working.
2. Regarding the menuWillOpen: delegate method. You should not believe what the documentation says:
Problem ID# 30546694: [NSMenu Documentation] Left hand doesn't know what the right hand is doing
https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/TableView/RowSelection/RowSelection.html
says the opposite of:
https://developer.apple.com/reference/appkit/nsmenudelegate/1518156-menuwillopen?language=objc
As I don't know which document is telling the truth, I stopped considering this a possible solution when I was investigating this problem.
Solution
But your idea about changing the contents of the menu reminded me about the - (NSMenu *)menuForEvent:; method. I had tried to subclass NSOutlineView and override this method. But I had given up using it as the clicked row is not set when this method is called, therefore finding out whether the item at the clicked row was a group one seemed not possible.
Actually, it's possible to figure out, it just requires to inspect the NSEvent * parameter and check whether the row at locationInWindow for a NSRightMouseDown event type is a group item according to the delegate. And, in this case, returning nil for the menu prevents the highlight and the menu from appearing.
- (NSMenu *)menuForEvent:(NSEvent *)inEvent {
if (inEvent.type==NSRightMouseDown) {
NSPoint tPoint=[self convertPoint:inEvent.locationInWindow fromView:nil];
NSInteger tClickedRow=[self rowAtPoint:tPoint];
if (tClickedRow!=-1) {
id tItem=[self itemAtRow:tClickedRow];
if ([self.delegate outlineView:self isGroupItem:tItem]==YES)
return nil;
}
}
return [super menuForEvent:inEvent];
}