Saying “Oh boy” in the way you would say “Oh my” or “Oh wow” is not gendered in the sense that you aren’t calling anyone around you “boy”, but rather saying it to yourself as an exclamation; just like how religious people say “Oh God!” or “Oh my God!”; if you said “Oh God”, you wouldn’t be calling the other person “God”.
Saying “Oh boy” in the way you would say “Oh my” or “Oh wow” is not gendered in the sense that you aren’t calling anyone around you “boy”, but rather saying it to yourself as an exclamation; just like how religious people say “Oh God!” or “Oh my God!”; if you said “Oh God”, you wouldn’t be calling the other person “God”.