Unripe pluots falling, white fluff at stem

Only four pluots set for me this year, so it’s upsetting that two have already fallen unripe from the tree. I see that there’s some kind of white fluff around the stem end, could this indicate what the problem is?