Party Play - Can't leave a map and rejoin (no death involved)

I don't think this is intentional (maybe it is?!). Whenever me and my buddy are playing together and doing waystones, if I leave back to his hideout for a trade or something, I can't get back to the map. The portals are there but I can't click any of them.

I thought it was only supposed to be if you die you can't go back in because solo this works just fine. Why is it any different in party play!!

Same case here, I died in sim map with mate and portal completely disappeared and couldn't re enter.

me and mates did a map with 11 or so breaches, I went out to unload and couldn't go back in. portals completely gone.