It had all looked so good for the Saddlers when Alex Nicholls gave them an eighth minute lead but it was all downhill from that moment.
Defender and club captain Rhys Weston was sent off for two bookable offences before the break and he watched on in horror as Robert Grant and Mullin turned the tie on its head in the second period.
Buoyed by Saturday's 1-0 win at Brighton the Saddlers flew out of the traps and they soon had the lead.
Steve Jones' free-kick into the box found Nicholls in acres of space and he volleyed home his first goal since February.
That should have settled the visitors down but they were reeling before the break as former Arsenal youngster Weston lost the plot.
Weston was booked for fouling Phil Edwards in the 22nd minute but didn't learn his lessons and was back in the dressing room 12 minutes later after another needless foul on the dangerous Edwards.
Accrington threatened before the break with John Miles having a shot blocked by Stephen Roberts.
But they were level six minutes after the restart as substitute Grant levelled on 53 - just two minutes after he had been introduced for Chris King.
The ball found its way to Grant in the box and he thundered a shot home via the underside of the bar.
Grant and Miles both tested Rene Gilmartin as Stanley smelt blood but it was the introduction of rookie winger Sean McConville that proved decisive.
The winger, plucked from Skelmersdale United in January, hit the post eight minutes from time with a rasping shot but the Saddlers didn't take notice of that warning.
And in the final minute McConville raced past his marker and crossed for Mullin to head in the last-gasp winner for the League Two outfit.