A groundbreaking article released earlier today by Advanced Aquarist magazine points to a little discussed, but troubling fact of life in our hobby: manufacturers are free to make whatever claims they want about their products and there is no real way for consumers to know if they are telling the truth or not. In “Experimental Comparison of Measured Flow Output of Aquarium Propeller Pumps” by Michael Sanford, William Straka and Sanjay Joshi, rigorous and advanced experimental design and testing is performed on a number of commonly used pumps for water movement in our reef systems. The results are quite startling as some well known and well regarded pieces of equipment fail to live up to their performance claims. Few would quibble with minor variations from stated specs, but when stalwart and generally reliable names like Tunze fall short by nearly 50% consumers need to take note. I have no intention of turning this into a company bashing post and readers can draw their own conclusions from the article. I merely want to highlight that there is a real need for independent testing in the hobby– something that is not going to come about easily due to the time, expense and expertise needed. One can also make a general call for manufacturers to make the effort to evaluate their products more realistically– again without an independent testing body there is little incentive to do so. So, what are hobbyists to do? In cases like this where respectable companies with positive track records are at fault and true data is only garnered by sophisticated testing beyond hobbyist means there really isn’t much that can be done beyond holding companies accountable when real data exposes gross inaccuracies. In other cases, approaching your purchases with some measured critical thinking can go a long way in helping keep manufacturers honest. Rich Ross’ excellent Reefs Magazine series on Skeptical Reefkeeping offers hobbyists much good advice in this regard. I’d recommend a look at the section on the Baloney Detection Kit for Reef Products found HERE. Other high quality independent studies can be found in Sanjay Joshi’s Lighting Guide and Ken Feldman’s skimmer study. Our hobby needs to find better ways to encourage and support these kind of efforts. The problem of the veracity of manufacturer claims is not new, nor is it likely to go away any time soon. Our hats are off to the authors of this new study and to others like them who are willing to put in the time and effort needed to help lend clarity to this murky side of the hobby. If any of you out there have suggestions to help sort through this industry-wide mess, we’d love to hear from you.