The usual reason for a compost pile to combust is that it is too tall. A pile
over 8 feet tall can form a chimney that fucuses the heat of the pile in a
draft and starts to burn.
The only remedy for this, once it starts to burn, is to pull down the pile. I
have friends who have pushed the pile into a lake and it burned for awhile
longer. Of course, this only is a concern in commercial applications since a
home pile will not reach a height of 8 feet normally
Otherwise not watering only means that the pile goes dormant, usually.