The theoretical papers in this thesis begin with the most basic signalling game and proceed towards a more general understanding of conventional signalling. I begin by investigating the importance of signal cost in the simplest possible model of communication, the Action-Response game. I demonstrate that the signals used do not have to be costly to be reliable, even when the signaller and receiver are in a state of conflict. I then consider the effect of adding costs to signals in a game in which reliable conventional signalling already exists, and demonstrate that the costly signals will be used by the weaker, not stronger, signallers. This demonstrates a stabilising mechanism fundamentally different from that of the handicap hypothesis, with its stabilisation through signal cost. Finally, I identify the conditions other than cost which are necessary for conventional signalling to be evolutionarily stable. These conditions relate to the information which both signaller and receiver must gain over the course of an interaction. Most models used to investigate signalling cannot account for behaviour seen in more complicated biological interactions because they are too simple to produce results other than that of the handicap prediction.
The other work included in this thesis addresses issues raised by the models. I review the literature on threat display use by birds, and present evidence that these displays are conventional signals. The stability of conventional signalling rests upon the existence of some common interest within a larger conflict between signaller and receiver. I present a clear example of communication attributable to common interest between fighting opponents. Cichlids of the species Nannacara anomala use a distinct colour signal, the Medial Line display, to coordinate another agonistic behaviour, tail-beating. It appears that both individuals benefit from the clearer assessment of relative fighting ability that this coordination affords. These N. anomala colour displays are quite conspicuous. It has been assumed that when common interest exists, signals will be very subtle, whereas when signaller and receiver are in conflict, signals will be exaggerated and conspicuous. Using an evolving neural-network model, I demonstrate that selection for exaggerated signals may exist even when the signaller and receiver have complete common interests.