But the home side came roaring back in the second period, aided admittedly by Gloucester losing two men to the sin bin.
It looked as though it would be a duel of the kickers as Nicky Robinson and Valentin Courrent traded penalties and the lead as the second half ticked by.
But, with no time left on the clock, Agen number eight Opeti Fonua scored the try that clinched the win.
Such a turnaround had seemed unlikely after 40 minutes and, on the long trip home, Gloucester will mull over the fact that they probably lost the game rather than Agen winning it.
Head Coach Bryan Redpath had said during the week that winning in France is never easy and he was proved absolutely correct.
Conditions were perfect come kick off at the Stade Alfred Armandie. The pitch looked in great nick, no prospect of rain in the air and a light breeze.
The Gloucester faithful were in good voice and there was a fair smattering of Cherry and White shirts spread around the ground.
Playing in an unfamiliar blue strip, Gloucester were quickly pinned back when an Olly Morgan clearance went out on the full. The big Agen pack sought to smash their way around the fringes but Gloucester defended well.
It was fast and open stuff in the early stages with Valentin Courrent utilising a variety of cute kicks behind the Gloucester line and Bryan Redpath's slightly starved of possession.
However, Gloucester earned the first chance of points after Peter Buxton fielded a kick downfield and returned it deep into the Agen half. The home side transgressed at a subsequent ruck and Nicky Robinson slotted the penalty with ease for 3-0 after eight minutes.
The Gloucester pack promptly followed this up with a powerful catch and drive which Agen could only pull down and Robinson added his second penalty.
Robinson's opposite number struck the upright with his first attempt at goal as Gloucester blocked the chaser from the restart and then pushed a second effort wide right.
Courrent did finally find his range on 18 minutes when Gloucester were penalised at a scrum bang in front of the posts not far outside the 22 to make it 3-6.
A good spell followed from Gloucester and it was obvious the pack fancied their chances with the driving maul with Agen pulling a couple down and receiving a stern lecture.
As a result, Gloucester kicked for the corner instead of at goal on 24 minutes and the patient drive resulted in Scott Lawson powering over and Robinson converted for 3-13.
The only blemish for Gloucester was the loss of Dave Attwood with a head injury on the half hour and a spell of defensive effort followed as Agen went back to using the big forwards to punch holes around the fringes - and to good effect.
Gloucester buckled but they didn't break and the officials received some real stick from an increasingly lively home crowd as a penalty award helped Gloucester move downfield.
Gloucester threatened with one further driving maul just on half time but a knock on brought the move to a close and a 13-3 half time lead was a reflection of a job well done to this point.
It hadn't been spectacular to this point but it had been professional. The Gloucester pack were doing the business but the ability of the Agen forwards to make yards at the breakdown would have been a topic for discussion at the interval.
It was important that Gloucester took the sting out of any Agen comeback and it took a superb cover tackle by Charlie Sharples to take the flying Osea Kolinisau into touch when the home side worked a smart overlap.
The home frustration rose when Courrent missed a 47th minute penalty and then a dangerous Jamie Robinson break finished with a loose pass inside to no-one in particular.
The pressure was building and eventually Agen got some reward as Courrent slotted a 50th minute penalty for a tackle off the ball.
It was all Agen at this point. Dave Lewis had to sweep up smartly in defence to field a dangerous kick towards the Gloucester line and the pack were working overtime in defence.
The referee lost patience at yet another breakdown near the Gloucester line and Peter Buxton was yellow carded.
From the resulting scrum, Agen number eight Opeti Fonua went blindside and just squeezed in at the corner despite a valiant effort from Dave Lewis. Courrent converted from the touchline to level the scores.
The game had turned on its head since half time. Gloucester had barely got into the Agen half and the home crowd were now well and truly in the game. Bryan Redpath's side had to dig deep.
However, bad went to worse for Gloucester as they were penalised for a man in front of the ball carrier at a maul and Courrent kicked the penalty from halfway to nudge his side in front 16-13 after 65 minutes.
A lineout steal gave Gloucester a ray of hope, however, and Robinson ignored the inevitable catcalls when Agen conceded a penalty at the ruck and levelled the scores.
He then had a chance to move Gloucester in front but nudged his penalty just wide left only to show nerves of steel moments later, amidst a cacophany of boos, to make it 19-16.
The drama still wasn't over. Mike Tindall was yellow carded for tackling a man in the air and Courrent made it 19-19 with just 5 minutes to go.
Gloucester stayed patient and used their pack to make inroads and a half break by Jordi Pasqualin almost opened up Agen. However, the move slowed up and Robinson missed with a drop goal attempt.
And, then disaster for Gloucester as Fonua spun out of a maul and galloped into the Gloucester 22. Sharples and Pasqualin got across and the try was awarded after consultation with the television match official.
Courrent converted, Agen had snatched the win and the home crowd were in raptures.
|Agen Score Card|
|Gloucester Rugby Score Card|