Welcome to the site, daveynutter!
I've said this already, but I'll say it again. If it is said as dialogue then I don't have a problem. People don't speak perfectly pronounced English all the time and therefore to have perfect English in dialogue is, frankly, unrealistic.
It's like the times Terry has had a character talking something that they had eaten and using the word "et" in the dialogue. It's not a proper word - you won't find it in any dictionaries as an alternative to "ate", but it what people say and therefore, in some cases more realistic.
If 'should of' was used outside of dialogue, then I'd agree with you. That sort of typo or mistake would bother me too.
And all this reminds me of a phrase I recently heard: Muphry's Law (as opposed to Murphy's Law) which states that: when correcting someones grammar online, it is inevitable that you'll make a mistake yourself.