Rugby Celtic League/Dragons 31 Ulster 11:Mark McCall's side were second best to the Newport Gwent Dragons at Rodney Parade on Saturday.
Dragons dominated both possession and territory to seal their first win against Ulster in five attempts.
Replacement prop Tom Court's second-half try was too little too late to salvage anything for Ulster as ill-discipline cost them dear.
Flankers Matt McCullough and David Pollock were both sinbinned and the high penalty count left Ulster always chasing the game.
Ulster came into the match in confident mood having beaten the Dragons in the last four meetings between the sides.
But coach McCall made five changes after the defeat by Edinburgh. Mark Bartholomeusz, Darren Cave, Paul Marshall, Declan Fitzpatrick and Tim Barker all returning as they looked to get their Magners Celtic League season back on track.
Cave made his first competitive start after recently impressing at club side Belfast Harlequins, scoring a hat-trick in each of his last two appearances.
Ulster kicked off but the hosts stormed into an early eight-point lead when Ceri Sweeney slotted home a penalty from in front of the posts after Ulster were adjudged to be offside after four minutes.
And the Dragons scored their first try just two minutes later.
A lovely move involving Sweeney, Gomer Davies and Morgan saw the Wales fullback make the line break before finding wing Gareth Wyatt in space to beat the despairing tackle of Tommy Bowe and score in the right corner.
But Ulster rallied and Niall O'Connor scored a long-range penalty on 17 minutes to keep the visitors in touch.
The visitors were reduced to 14 men after 32 minutes when McCullough was yellow carded for coming in at the side and Sweeney added the penalty.
Ulster responded after the interval when O'Connor scored his second penalty early in the second half to reduce the deficit to eight points.
But Ulster created their own problems when Pollock was sin-binned. Moments later, Gomer Davies took Steve Jones's pass to score under the posts before Court scrambled over from McCullough's pass.
But the Dragons held firm and Luke Charteris sealed the win with a try in the final minute.
DRAGONS: Morgan (capt); Wyatt, Davies, Maule, Mustoe; Sweeney, Evans; Gustafson, Jones, Thomas, Charteris, Hall, Lydiate, Parks, Bearman. Replacements: Smith for Maule, Thomas for Mustoe (both 41 mins), Owen for Bearman (48 mins), Black for Gustafson (54 mins), Ringer for Lydiate (56 mins), Daly for Jones (70 mins).
ULSTER: Bartholomeusz; McCrea, Cave, Bowe, Danielli; O'Connor, Marshall; J Fitzpatrick, Brady, Fitzpatrick; Harrison, Barker; McCullough, Pollock, Wilson (capt). Replacements: Cunningham for Cave (48 mins), Court for D Fitzpatrick (50 mins), Miles for Brady (55 mins), Del Fava for Barker (58 mins), McMillan for Pollock (64 mins), Hallett for O'Connor (77 mins).
Referee: George Clancy (IRU).