A pom is allegedly a contraction of pomegranate - pronounced to rhyme with immigrant ( I guess your accent has to be right for this)
My Dad was an Ozzie. He always told me "Pommy" came from "P.O.H.M" as in "Prisoner Of Her* Majesty". So for many Ozzies yes, it could be considered an insult as it reminded them of their criminal heritage.
In short, it's racist, don't do it, don't make excuses for it.
It's not. Most Paki's I know use it in self-reference when discussions or questions of their heritage come up. But like "Yank" and "Limey", it can be, if that is the intended use.
* Or "his", as the case may be - but quite sure he used "her".