Tommaso Iannone replied with a penalty on the stroke of half time and missed with three more shots at goal that might have embarrassed the visitors. Their try from Filippo Ferrarini, converted by Alberto Chillon, was the final move of the match.
England had earlier defended well when reduced to 14 men after Saracens centre Tom Casson was yellow carded for flopping over the ball at a ruck.
But one try was a poor return for a side that enjoyed so much territory and possession, as well as creating half-a-dozen decent scoring chances.
Mark Mapletoft's side had opened the tournament by running in five tries to beat Wales 41-14 at Gloucester, but made changes after six of the squad were recalled by their Guinness Premiership clubs.
Into the starting line-up came wing May (Gloucester Rugby) and centres Jonathan Joseph (London Irish) and Casson (Saracens). Up front, Danny Wright (Gloucester Rugby) and Charlie Matthews (Harlequins) formed a new second row pairing with Jackson Wray (Saracens) at number eight.
They didn't look unduly unsettled by the changes in the early stages when they were full of energy and opened the scoring inside three minutes after attacking from deep in their own territory.
May streaked up the left, traded passes with Wright and cut back to score under the posts with Burns's conversion opening up a 7-0 lead.
But despite dominating the remainder of the first half, they couldn't finish several chances and the error count was worryingly high against an Italian side beaten 39-0 by Ireland in their opening game.
The Azzurri had two chances to get on the scoreboard from rare incursions into England's territory in the first half, Iannone mishitting his first penalty shot after 24 minutes before making it 7-3 moments before half-time.
England shuffled Burns to full back and brought Jake Sharp on at fly half but life got harder six minutes after the restart when they lost Casson to the sin-bin and it required some frantic defending to keep Italy out.
With a full complement on the field, they flared back into life when Will Hurrell burst through the middle only for his side to spurn decent chances on both flanks.
A second penalty from Burns edged them 10-3 in front with 20 minutes remaining and he added two more in the final 10. Ferrarrini's late strike emphasised how close Italy had come to a famous upset, and how much work England have to do.
Ruggero Trevisan; Giovanni Alberghini, Tommaso Benvenuti, Sebastian Rodwell, Michele Mortali; Tommaso Iannone, Edoardo Gori (captain); Carlo Fazzari, Andrea Manici, Piermaria Leso, Enrico Targa, Filippo Cazzola, Nicola Belardo, Stefano Scanferla, Gabriele Cicchinelli.
Replacements: Nicola Corbanese, Francesco Pepoli, Lorenzo Santelli, Marco Barbini, Filippo Ferrarini, Alberto Chillon, Vittorio Callori di Vignale
Scorers: Try: Ferrarini. Con: Chillon. Pen: Iannone.
Tom Catterick (Newcastle Falcons; Jake Sharp, Saracens 41); Will Hurrell (Leicester Tigers), Jonathan Joseph (London Irish), Tom Casson (Saracens), Jonny May (Gloucester Rugby; Sam Edgerley, London Irish 67); Freddie Burns (Gloucester Rugby), Sam Harrison (Leicester Tigers; Sam Stuart, Harlequins 77); Joe Marler (Harlequins; Shaun Knight (Gloucester Rugby 65), Jamie George (Saracens; Arthur Ellis, London Wasps 78), Shaun Knight (Gloucester Rugby; Lee Imiolek, Sale Sharks 50), Danny Wright (Gloucester Rugby; Peter Elder, London Wasps 77), Charlie Matthews (Harlequins), Will Welch (Newcastle Falcons; Elder 48-55, George Kruis, Saracens 77), Jacob Rowan (Leeds Carnegie, captain), Jackson Wray (Saracens)
Scorers: Try: May. Con: Burns. Pens: Burns 3.