If I were ever to have been so fortunate as to have had a girlfriend or a wife, I would not have had a problem with her looking through my phone... but I wouldn't just assume I could pick up her phone and do whatever with it. Whether I suspected her of anything or not, I wouldn't assume I have 100% access to all things in her life. As I said earlier, if I had reason to suspect her of something, I would need to talk to her about that to determine whether my suspicions were justified OR if there was something wrong with me and how I was perceiving our relationship. But that is just how I'm wired.
On the flip side... it sounds like it's a consistent thing that he doesn't just like you going through his phone. So, either you are right to be justified in being suspicious and you should address that or consider whether you should be with him if he is hiding something... OR you are not right and you should be asking permission before doing something you know he doesn't like you doing. It feels from a distance like it either has to be one or the other... either he is hiding something and you're right or he isn't hiding anything but you persist in pushing buttons you know will trigger him. You not liking being ignored makes sense, and I feel that... but maybe he also doesn't like being suspected of things and triggered by invading his privacy.
If the genders were flipped here... and a man was complaining that his wife ignores him sometimes and doesn't like when he goes through her phone... most women would want to know why the man is so insecure and has to control his wife by being all over her phone and privacy.
Again, I'm not judging. Something is definitely in need of being worked out... but it is going to honestly have to be you and your husband sitting and talking about what you like and don't like about how things are going.