Hey Nosey, sorry I couldnt reply earlier, not supposed to use our phones in work lol.
It sounds to me like you have done the best thing you can by walking away, and been a grownup about it by writing to him to try and explain your feelings and frustrations, and the reasons it won't work...