I have noticed no such thing at least.
If you pet is level 11 and opponent level 23, you shouldn't ding more than one level in any case. It takes 900 xp to get to lvl 12 from lvl 11 and the xp gain from your said opponent gives 935 xp with Safari Hat. So basically no matter how much xp does your pet have past lvl 11 in the beginning, it still won't be enough to reach level 13.
However, I can't say what's happened with your level 4. Xp again from lvl 23 opponent is 858 xp which should bring your pet at least to level 6. But like said I haven't seen any restrictions in multiple levels with low level pets.
- Taking your candles since 2005
Everything you need to know about WoD followers can be found in my Warlords of Draenor Follower Guide: http://www.wowhead.com/guide=2533