Roni Joe gave Aylesbury the lead on 4 minutes when he latched on to a Danny Gordon through ball, held off the challenge of a Rugby defender and slotted the ball home.
Joe had a chance to make it two a minute later when his cross cum shot was tipped over the bar by the Rugby goalkeeper.
Darren Grieves then headed over a Gordon cross before seeing a powerful near post shot well saved by the home keeper. Darren had a glorious chance to double the lead 2 minutes before half time when he found himself one on one with the keeper only to see his shot sail inches wide.
Aylesbury finally doubled the lead on 56 minutes when Roni Joe danced his way through the home defence only to be brought down by a defender. A penalty was awarded and Phil Mason duly converted it.
Kieran Gallagher had a superb free kick saved before Rugby pulled themselves back into the game on 88 minutes. James Courtnage failed to connect with a corner and the ball found its way into the net.
The Ducks quickly restored their two goal advantage a minute later when Mason was set through on goal by Daniel Gray and hammered the ball in off the underside of the bar to book a trip to Tiverton in the next round.