I kind of had to smile about all the objections to baby. I love endearments, speak to me in endearments and you might be a curmudgeon I'm going to have a soft spot for you. I use them a lot, babe, hon, sweetie, etc.They call them Laurenisms in my department lol. I think it is partly a southern thing. I didn't count up everyone who objected to baby so it is not a statistically relevant statement, but I've found its a very southern thing to do and most people I know here who object were not born and raised in the South.
Petey called me sweetpea, babe, baby, honey. He called our girls doll baby and baby girl. To this day they love that, my youngest especially likes baby girl.
Besides when they make you mad, it is so satisfying to say: "Oh honey don't you dare 'baby' me" :-)
It can also come in handy, when you draw a blank on someone's name, lol. . .