It's the Grammar Gestapo here; "…WHO people invite to parties", if you're using the pronoun as the subject of the sentence, it's 'who', 'whom' is objective, very often used to avoid the generally-accepted-as-being-poor-grammatical-form of ending a sentence with a preposition.
In this instance, the pronoun is referring to you, you are the subject, therefore the correct form is 'who'.
I'm afraid I find that parse dubious.
The sentence in question was "This doesn’t happen to me often but then I’m not the sort of person whom people invite to parties." Two independent clauses, so we can disregard the first (and the coordination conjunctival phrase "but then") and reduce the problem to "I’m not the sort of person whom people invite to parties" with no loss of generality.
In that independent clause, we have a passive-voice construction which consists of a simple subject (the "I" of "I'm"), the copula (the "'m" of "I'm"), and a predicate nominative. The last is a noun phrase modified by a dependent clause in the passive voice. Inverting that clause to active voice gives a construction of the form "people invite X to parties", where X is clearly the object of the transitive verb "invite" (the subject of which is "people", and "to parties" is of course an adverbial prepositional phrase).
That should clarify the role of the remaining phrase "the sort of person": it is the object of "invite". And "whom" is the relative pronoun introducing that adjectival dependent clause. Its antecedent is "the sort of person", which per above is the object of "invite", and thus that relative pronoun should be1 in the objective case.
As it is.2
(I'd provide a diagram of the sentence showing the parse, but it'd be a pain to do in ASCII and the Reg doesn't do preformatted-text well. I'll leave it as an exercise for the reader.)
1"should be" in a weak sense of "is preferred by those who care about such things", under a so-called "scientific" view of usage that wants speakers to maintain parallel grammatical constructions with consistent case, number, and so on, and relies on the traditional-preferred interpretation of particular words having particular grammatical case, etc. Prescriptivism is a religion I do not endorse.
2And they said three degrees in English was a waste of time.