Remember that in census records, relationships are given in regards to the head of the household--typically the husband. His children may not all be children of his wife.
And individuals listed as children may actually be step-children.
So relationships in one census may be tenuous and not entirely clear.
