Planting vegetables prone to phytophthora blight near fruit trees

Is it a recipe for disaster? I am rearranging my veggie garden to give space for cherry and apricot planting in the area designated as veggie garden for almost 10 years. Some years potatoes or tomatoes will be growing as close as 8-10 feet from the trees. And soil I am planting the trees into is also garden soil, where tomatoes or potatoes were growing before. Is it high risk set up?