But Nick Walshe's charges were left to rue a first 20 minutes which left them with a mountain to climb.
A try conceded in the first 90 minutes was followed by a period of ill discipline which saw England virtually penalised out of the game.
Things were looking grim at 16-0, but got even worse when Matt Hankin of Saracens threw a pass that was picked off for a try.
However, as half time approached, there was a glimmer of hope when Sam Hill's break resulted in a try for fellow centre Harry Sloan.
Stooke smashed his way over in the corner at the start of the second half, only for Henry Slade to promptly have a kick charged down, gifting another score to South Africa.
England had a let off when South Africa were denied another try and things got interesting when Henry Purdy's try got his team back within a score at 31-24.
There were chances in the final quarter of an hour, but overlaps were squandered and turnovers conceded deep in the South African 22.
Given another few minutes, England could well have snatched this but the poor start to the game left them with just too much to do.