‘if you ask me’ is C2 in the English Vocabulary Profile and is used to give opinions.
For example:
And it‘s about damned time if you ask me.
Collocations of ‘if you ask me’ in COCA show that this is used mostly to give negative opinions.
1 STUPID 10
2 NICELY 9
3 BS 6
4 DUMB 6
5 RIDICULOUS 6
6 PATHETIC 5
7 DAMNED 5
8 IDIOTS 4
9 RISKY 4
10 CLEVER 4