The three boys and a girl were taken into care by Sandwell MBC, because social workers discovered they were being neglected.
Recognising neither her daughter, who has learning difficulties, or her son-in-law, who has previously served a jail term for injuring another child by a previous relationship, were capable of caring for the children, the children's grandmother initially offered to look after them herself.
But, despite a 'glowing' recommendation from an independent social worker, the plan was given the thumbs down by social workers and a court-appointed guardian who felt the children should be adopted.
Despite praising the extended family he said they did not understand the problems rearing the children would present and had never known them very well.
And despite accepting the offer was a 'genuine one' made by 'kind, warm-hearted people' Lady Justice Butler-Sloss, sitting with Lord Justice Brooke and Sir Brian Neill, refused to overturn Judge Ward's ruling.
She told the court: 'There is no doubt the judge recognised the genuineness of the family, the genuineness of their application and that they were a decent family who had successfully brought up their own children.
'He did not criticise them but said they did not understand the implications of taking on damaged, vulnerable children who had had a very bad start to their lives.
'That bad start means they will be unusually difficult to manage. They would not present the same difficulties that normal children do.
'He felt the family would not be able to cope and it is impossible to set aside that conclusion.
'This was a difficult case and the judge came to a conclusion which cannot be faulted.'