Outcast. Person in a family who is different from the others. (Compare to Odd man out or March to a different drummer.)

“My brothers all went to college, but I traveled instead, I’m the black sheep in the clan.”