Bishops started the game as if their lives depended on the outcome. After five minutes Stortford raced through and shot, but Phillips put the ball round for a corner. From the kick, Cooper struck the woodwork through a crowd of defensive legs.
Kieran Gallagher raided down the flank to good effect, but United's attack was not firing on all cylinders. The closest they came to breaking the deadlock was just before half-time when Stacy Joseph had two chances, the best of which flew off a Bishops defender with keeper King stranded on his line.
The second half saw Aylesbury in a more determined mood and on 49 minutes Luke Longman burst into the box, only to be foiled by the alert King.
United had plenty of the ball without doing a great deal in the final third of the field.
Against the run of play Bishops went in front after 74 minutes when Karl Shuttlewood headed home a Parkyn corner.
In the latter stages United pressed in the hope of getting level. Inevitably gaps appeared and Phillips twice came to the rescue. But three minutes from time Phillips was beaten by Craig Ravenscroft, recently signed from Kingstonian, who struck a sweet edge of the box shot into the net to give his side the three points.