Annoy. Goat referring to one’s sense of well being, so to get it is to aggravate. (Compare to Get under one’s skin.)

“That old lady complains about my smoking and she really gets my goat.”