Housing Minister Sean Fraser says he will not be running in the next federal election, citing family reasons.