Why are most marine fish so territorial?
Coral reefs are an island of life in otherwise low nutrient seas. Most of the resources are tied up in the tissues of plants and animals and oportunities are limited. Predators are all too eager to consume any animals that lack shelter and any fishes sleeping in the open at night will not often live to see the next morning. If you possess a good hiding place and a food source, there is always someone wanting to take it from you!
Typically, less territorial species feed on abundant food sources such as plankton or open water shoaling fishes. They may be shoaling so they can overwhelm individuals that would successfully guard their food source from smaller numbers of fishes and the large shoals of Tangs such as Convict Surgeons (Acanthurus triostegus) are an effective way of feeding on the algal gardens of other species.
Sex is also an issue close to the hearts of most marine fish and the fact that many species compete for the role of dominant male or female serves to make them even more aggressive.