How close is to close

[flatcreek]

Last night I was drift fishing in my favorite lake and scolded by a fisherman who was double anchored (a Manny L fan I'm sure) for being to close to him. I didn't understand this as I was 100 or 150 ft. off his bow and wanted to complete the drift into shallower water, we weren't running any motors or making any noise of any kind and in no danger of hooking any of his gear, even if we did hook a fish which we did just as we went by. I replied to him" no were not, see we are catching fish" at that it turned into a double. Obviously he wasn't close enough to us.
All kidding aside, I like to be a respectful fisherman and thought I was. Many times I've been drifting and had trollers see me catch a fish and troll all around me, under power, much closer than a 100 ft. almost running over my bobbers, I wasn't happy about it, stealth is essential. It wasn't the case of me seeing them catch something, I started the drift 1/4 mile away wanting to drift over the deeper water into the shallow as I've done many times, wasn't even aware of them at the start. In my opinion just because you read a book in which the author bitches about other people hooking his anchor ropes and calls people who drift lazy doesn't mean you own 500 ft. of water all around your boat.
As we all know what opinions are like. What do some of you other fisherman think? As I truly do want to be respectful?