Skill Bar Keeps Resetting

I started having this issue after making some changes on passive tree, basically 3 of my skill gems just vanish from the bar everytime i change map. I still have the stats requirements to use them, I just have to replace it all the time, which is a chore.

I tried reset passive tree and nothing seems to change