Hi there
I think understanding of the speed controllers and how they work may help.
Basically, modern day speed controllers they rely on feedback from your transmitter to set up the internals of the speed controller. So, basically when you switch the speed controller on and then switch the transmitter on - the speed controller looks for the centre position of your stick. Also to complicate things more, inside the speed controller, keeping it very simple - your power supply from the transmitter to the speed controller and from your speed controller to your motor is all connected.
This is why you are having problems.
So, if you put a switch in between your battery and the speed controllers - it is interfering with the signal that is coming from your receiver. The next thing I think you should be looking at is a mixer if you haven't one fitted already. The type that slow down the outside props when turning, as you will find if you don't have your props turning in the right direction; Steamboat Phil will be after your model for straight running
. Take note about what Colin Bishop has said about the props turning inboard towards the rudder.
My experience of this was with my model of HMS Ajax; even though I had all the fancy mixers and whatnot fitted - I had the props fitted in the wrong direction and everyone thought it was just a free running model
. It refused to turn.
Good luck as they say, Happy sailing