See
http://www.telegraph.co.uk/culture/music/rockandpopmusic/10558380/Clappy-unhappy-why-Neil-Young-was-wrong-to-tell-his-audience-to-be-quiet.html
I can see two sides to the argument here - or really both arguments (the author puts forward two).
It can be distracting to a performer. There's a guy who likes to come along to TJ's open mic nights with one of those little shaker things and shake along with the music. This idea got so popular at one point that other folks decided to go down to their local music store and buy themselves a little shaker and shake along. And some of them don't have much idea of timing. Another problem is that you can be trying to play on the off-beat and everybody's doing their thing on the on-beat - which ruins the effect you're trying to achieve. I get grumpy about it and once asked the shaker to please stop shaking during my set as it was off-putting.
But there's also the whole point of what music is about anyway and whether it's just entertainment (there's a school of thought that says that all art is just decoration, no more, no less) or whether it's serious. This is trickier than we might think. Go to a classical concert these days (other than the last night of the proms) and there's a very strict sense of protocol and you're expected to keep quiet and still whilst the orchestra is playing. But it wasn't always thus. Historical records suggest that a couple of centuries ago when what we now call "classical" was contemporary music concerts were often rowdy affairs with the audience chatting and stamping their feet (sometimes in time with the music) and dragging chairs around whilst the unamplified orchestra struggled to be heard.
Personally I like to listen - unless it's music that doesn't stand up well to actual listening, in which case it better have a good groove. But one can get too precious about it and not all noise is a competition with the performer or a sign of disrespect.
A couple of years ago Mrs X-rated and I went to a flamenco show in Madrid. I'd pre-booked, and I'd booked in a small, intimate venue that had no amplification. When we got to the venue we were told, politely, that the audience were to respect the performance and keep quite and not disturb the performance.
Now I'm not against that in principle. I have occasionally gone up to people at TJ's and asked them to keep it down. Some folks have loud voices. Others come to see one act on the bill and don't show any interest in or respect for the rest of the lineup (or the people who want to listen to them).
However, on this night in Madrid things definitely did not stay quiet. They started off that way, but towards the end of the first set there was plenty of whooping and hollering going on (including from me). I felt it was OK because it was a response to the music, might even have encouraged the performers (and the management who'd impressed on everybody that it was an unplugged performance and so no noise was to be made did nothing to discourage it).
I don't think it's precious to ask people to respect the performance, and I do wish they'd learn where the off-beat is (or stop clapping on EVERY beat), but there's a world of difference between showing appreciation for the music, even interacting with it, and the sort of thing that happened to me at one open mic night where somebody repeatedly had conversations on his cell phone during my set and eventually shouted to the party on the other end "I can't ****ing hear! There's some **** playing ****ing music here!"
And somewhere between that sort of stuff and rapt silence there's a line - but I'm not sure where it is.