Daniel Phillip said he grew fearful for both his and Ventura’s safety during his testimony