Once the trees get big enough, they will likely still fruit fine under the outer canopy. The big trees growing around Seattle all seem to successfully bear fruit every year, even when we went into the teens after flowering. I’ll go check the one in the International District soon, but I suspect it’s still holding fruit from the flowering in Dec., even though it got down to about 15°F last month.
See, e.g., last year, after a winter low of 17°F or thereabouts: