Former Alito clerk says it's "bonkers" that he won't recuse himself from cases involving Trump