We trained and tested a value unit artificial neural network architecture on the 5 parity problem. Our aim was to determine if critical learning periods exist in networks. We show that there are periods during learning in which lesioning can differentially affect network performance. We lesioned randomly selected hidden units from the network at various times during learning and then tested the network. Results indicate lesioning the network after about fifty presentations of the training set markedly improves performance. At other times lesioning reduces performance. In a value unit architecture hidden units that show high variance are typically those that have banded. Conversely, low variance units have not yet banded. When we lesioned low or high variance units at various times during learning we found overall network performance to be improved or reduced, respectively. Consequently, it is extremely important for network performance which hidden units are lesioned. Thus, the time effects produced by lesioning are dependent on the degree to which hidden units have banded. While the results do not asppear to be directly analogous to critical learning periods we believe that interesting likes can be drawn between network lesioning and developmental phases in the central nervous system.