Prince Harry has refused to break off a strong relationship between Meghan Markle and her father