As humans, it's our natural ability to become attached. Whether it's to people, things, memories, or moments, we all get attached. That's why there's no use trying to detach yourself from everything. Even if you try and remain unattached, the most it will ever be is a lie, because deep down, we all grow fond of things and come to care about them. It's our nature.
When I was dying, years ago, I was in that state. I had completely detached myself from everything and everyone, I no longer gave a shit if I was alive or not because I didn't care if I left everything behind. Or at least that's what I told myself. That's the way that I had groomed myself into thinking. But then my dog came into my room, sniffed my face, and I started to cry. I realized then that, even though I'd tried to hard not to be, I was attached. Not just to my dog, but to everyone in my life at the time.
After that, I stopped fighting it. I allowed myself to become attached to people, and you know what? Yeah, it hurts when they leave, when you're betrayed, but not everyone is going to leave. And the pain you experience from those who do leave is far exceeded by the joy you experience with the ones who stick around. And even if someone you expected to stay winds up leaving, you're able to remember the good memories far more often than the bad ones, because you haven't focused on the negative aspects and the idea of them being gone.
In the end, it's worth it to just stop worrying about what might happen and live your life loving and enjoying every moment. It's worth it to just experience things, let happen what will happen, and learn to be thankful for every moment that you breathe. Not everyone has the chance to live their life so freely, so why worry about what might happen later? You'll still be far more lucky that you will ever know.