My horse riding balance while trotting without stirrups is terrible?

How can I change this. It's starting to annoy me. I haven't particularly got the best confidence in the world doing work with no stirrups as I fell off in early learning to ride and every time I do sitting trot without stirrups going round the corner i nearly fall off. Also to improve my balance I've been trying to go slower but then I am constantly being moaned at by my teacher for going to slowly so I speed up and nearly bounce off the side let alone get very nervous. I then don't breathe properly and get bad stitches. This scary and annoying so I just want to be able to stop it before i fall again and knock my confidence even more.