Moni

It looks like scale to me too, but I have never heard of scale producing worms that are obvious to the eye. There is a cottony scale. But once again, no obvious worms. Mealy bugs don't really look like worms, either. But they aren't usually white. The cottony stuff around them is white but I have seen the insects themselves as red or gray and, once again, they do not look like worms. When you get all of the cottony stuff off of them, they kind of look like a cross between aphids and pillbugs with legs. I don't know. Picking them off certainly doesn't hurt. Whatever they are. Try Neem but test it on a small portion of the plant before you spray the whole thing. I am not familiar with your plant and it might be sensitive to the spray.
Moni