I have experienced trouble in the past from a type of small fanworm that is invasive and a real nuisance, as it encrusts everything including corals. It is a real threat to soft corals, as the tubes grow right through the coral (or more correctly, the corals grow around the tubes). The corals then get tissue recession in the area of the tube.
In extreme cases, I had to frag and regrow a couple of softies and a xenia that had the mother colonies killed by the fanworm intrusion, in order not to lose them entirely.
I welcomed these fanworms at first, just as indicated by the others posters in this thread. Then I learned what a pain they are.
Now I have them under control by using metal forceps to rip them out whenever I see them grow too thick.
They are white with a fan about 3/4 of an inch across max, and build a grey leather-like tube up to about 3" long. The tube is soft and leather-like, made of mucus and detritus, not hard and made of calcium.
If you get too many fanworms and they start to invade your soft corals, thin them out.