He's an ass, but I can somehow understand why he is so. He lost his wife and child to schemes of his uncle and his wife was originally a spy, but they fell in love regardless.
This ended in loss of two lives.
I guess he doesn't want her to fall for him so that past mistakes aren't repeated.