The court heard the boy has been with foster carers since July 1996 when he was taken into care by Middlesbrough Council, but his father, now 36, was at first allowed to see him once a week for three hours.
But the council decided the contact was 'too long and too frequent' and was having a disturbing effect on the boy and, in November last year, cut the contact visits to one hour each fortnight.
And, after the boy's behaviour became 'increasingly disruptive,' His Honour Judge Briggs, sitting at Teesside County Court in April this year, gave the council permission to cut the father's contact to just three or four short meetings a year.
There were, he said, other factors which might have caused the boy's disruptive behaviour, including his distress at finding out that his 28-year-old mother had formed a new relationship and was expecting a new baby.
'Judge Briggs gave quite insufficient regard to the other factors which were prevailing in this little boy's life,' he told the court.
But Lord Justice Thorpe, sitting with Lord Justice Mummery, dismissed the father's appeal as 'quite hopeless'.
Contact visits with the father had in the past 'gone badly' and the judge had given 'clear and trenchant reasons' for rejecting the father's explanations why he had not turned up to some meetings and had given his son no birthday or Christmas presents.
Judge Briggs had also 'found as a fact' that at one contact meeting the father had 'expressly encouraged' his son to 'play up' his foster parents as part of a 'strategy to destroy the placement' and win back the boy's custody.
Judge Briggs had found that contact with his father was the 'probable cause' of the boy's disruptive behaviour.
The appeal judge ruled: 'This is in my judgement a quite hopeless application and I would refuse it.'
Lord Justice Mummery agreed.