Hey LA guy,
I, like you, was very hesitant to get in the water with the bag. When I did the first couple of times, I really felt uncomfortable. Mostly because there was now drag (not a lot but some) where there had never been some before. So my focus was on the bag the entire time rather than on swimming, so I didn't enjoy it. So I wore a belt (wrap) with the bag that kept it pressed against my body, and that really helped me stop focusing on the bag while I was in the water. I probably would have gotten used to it eventually, but the belt worked.
Then I got a pretty decent-sized hernia, and the nightmare scenario started to occur regularly. My bags started to come off in the water. So I started to secure them with waterproof tape around the entire perimeter of the bag. I eventually switched from the tape to Coloplast elastic barrier strips because removing the tape would take off skin when removing it from time to time. Both options were successful in keeping the bag on. Again, I only had problems once I had the hernia, but if looking for extra peace of mind, tape or barrier strips will do the job.
One other note: I switched from swimming to deep water running because the Aqua Jogger belt covers the bag area. I find most of the issues I have doing stuff are more mental than physical, and the Aqua belt helps my mind. I now get in the pool 4-5 times a week, and it is by far my favorite post-ostomy exercise.
Lastly, if you want to know anything else or talk about it more, feel free to send me a private message. This is something I really struggled with and am happy to share any and all of my experiences.
Good luck