I saw a thread about this not too long ago, with similar concerns.
I don't recall many of the details, but essentially, if the tank is stable, the inhabitants healthy, I don't see that a small amounts of residue is a problem. I would simply strain it out, or not draw water from the bottom of...