Danny Nicholls kept faith with the team which had gone close to beating Chippenham three days earlier, although Joe was in for the injured Jamie Kearns with Paul Edgeworth slotting into the centre of the park.
As it turned out, the inventive play of ex-Kettering man Kearns was missed by Aylesbury all night. Although dominating possession they were unable to break down their hosts defence for the entire first half and chances were limited.
The first opening of the night came after twelve minutes, at which point it looked like Aylesbury would have an easy evening ahead as they passed the ball around slickly. Drew Roberts played the ball up to Gavin Jaggard in the area, but the midfielder blazed a half-volley over the bar.
Cheshunt nearly scored with their first shot on goal, a chance that came out of nothing. A weak goal-kick from Jack Rashid was pounced on by Rob Tungatt, who held onto the ball waiting for the right time to shoot, but pulled his effort wide of the far post.
With United struggling to convert their possession into openings, the hosts looked to exploit the counter-attacking option. One such move lead to the ball ricocheting through to Steve Obeng, but his shot was excellently blocked by Liam Dolman.
Just past the half-hour mark Dolman played a fantastic diagonal ball which was flicked on by Roberts, and then driven in hard and low by Daniel Mead from the left. The arriving Darran Hay was unlucky as his shot on the stretch flew over the bar.
George Gregoriou had the final say of the first half, with two quick chances. Neat build-up saw him play a nice one-two, but his final shot didn't match what proceeded as the ball went wide of the target. Minutes later Gregoriou was put in down the left flank, but with no support his only option was to go for goal, but he placed his shot well over the bar.
Half-time: Cheshunt 0-0 Aylesbury
United changed tactics during the interval, pushing Roni Joe forward to make three strikers, a switch that worked a treat as Cheshunt suddenly struggled to contain Aylesbury's attacking play.
Hay headed wide early on from Drew Roberts' accurate cross, before Joe scored the opener on 50 minutes.
The ball was played into the lanky winger's feet, and he turned smartly in the area before crashing a left-footed effort high into the net. His elation was evident, having netted for the first time since against Northwood in March 2002. 1-0 Aylesbury
The Ducks were now a true force going forwards, brought to life by their goal and two chances fell to Darran Hay. The first from a Mead corner, which he got too far underneath and headed way over the bar. Secondly, he ghosted onto the end of a Doleman delivery but again headed off-target.
With the game no longer level, Cheshunt were forced to come forward a bit more and they nearly got an equaliser right on the hour mark. A corner was headed down and as the ball bounced up towards goal Rashid was forced to make his first spectacular save of the season, tipping the ball over the bar.
Five minutes later the home side were at it again, as Tungett hit a left-footed shot just wide of the post.
It was by now end to end stuff as either side could've scored. Daniel Mead saw a drive well saved by the diving Martin Pete after Joe had flicked on a long-throw causing a penalty area scramble.
Cheshunt then launched a flowing counter-attack, and the ball was worked out to their best player of the night - George Gregoriou. He worked his way into the area, and shot across Rashid's goal, but the Aylesbury stopper matched the effort with another fine save.
Hay had another chance to score with his head, but couldn't get the necessary power in his effort, having got on the end of yet another excellent ball in from Dolman.
With the game entering the last fifteen minutes, Cheshunt were seeing more and more of the ball and were camped in United's half for much of the remaining match.
On the whole though, Aylesbury's defence were able to deal with whatever the home side could throw at them and a clean sheet at such an early stage of the season can only be a positive result.
Rashid again denied Cheshunt with a block on his line from a headed effort at a corner kick. Gregoriou again went close, this time with a glancing header which drifted narrowly wide.
Substitutions were made by both sides, Aston Goss and John De Souza both coming on for Aylesbury. The youngsters injected much needed freshness into the side, and were able to keep the ball using their pace to run at the tired Cheshunt defenders in the dying moments.
No further attempts on goal were forthcoming though, and United left Hertfordshire with a deserved three points and can look ahead with anticipation to Saturday's long trip to Mangotsfield United.
Mangotsfield United
Salisbury City
King's Lynn
Rugby Town
Hitchin Town
Northwood
Cirencester Town
Halesowen Town
Banbury United
Grantham Town
Chippenham Town
Gloucester City
Merthyr Tydfil
Yate Town
Bath City
Tiverton Town
Evesham United
Team Bath
Bedford Town
Chesham United
Cheshunt
Aylesbury United