Yeah if everyone who wasn't coughing just walked around all day doing nothing but breathing and keeping their 6 feet of distancing then you would probably have a point. Unfortunately though people mouth breathe, people talk, people wipe their face/mouth and then touch other things.
If we 100% without question knew that the virus was only transmittable when symptoms of distress (like a cough) were present then sure maybe the common sense route would work.
End of the day very, very, very few people have legit issues that can/will prevent them from being able to wear a mask for a brief time in a social setting. The evidence of it working is present in all the other countries that have embraced the process, so to me it's worth my minor annoyance to wear a mask now and try to prevent stiffer restrictions/penalties in the future. I really hate that it's taking local governments having to force business to close to prevent group gathers.