By Richard Ross

Will that product make your tank look like this…

…or this?

In the last installment we talked about the role anecdotal evidence and logical misunderstandings play in how we make decisions about reefkeeping. In this installment, we’ll look at how and why manufacturers make claims about their products, why you might want to be skeptical about them, as well as some practical advice for determining the validity of those claims.

It seems you can’t turn around in the reefkeeping world without bumping into another new product that you must have to keep your reef healthy. The claims are usually the same, always some version of one of these:

-This product will unlock your reefs potential.
-Cure any and all disease in a reef environment.
-This will change the way you keep your reef.
-You’ll see colors and animal health that you have never before experienced.

And my personal favorite:

–YOU’LL NEVER HAVE TO DO REGULAR MAINTENANCE ON YOUR TANK AGAIN!

The ads are very clear in a roundabout way; Without THIS product your reef sucks.

Honestly, sometimes a new product does work. A lot of them don’t.
Even widely used products occasionally don’t do what they claim but in spite of this, somehow they’ve caught on.

I’m going to tell you the one thing you can learn to do for your reef that will improve it’s condition, and your sanity, from day one: Skeptical Thinking.

What is Skeptical Thinking, Rich?

I’m glad you asked. First, it’s not being a grump. For some people the idea of being a “skeptic” has a negative connotation, but do Shaggy and Scooby Do seem like grumps? They’re skeptics. Think about their show. At the end there never is a monster or a ghost, it’s always, as Tim Minchin would say, “the dude who runs the water slide.”

Television shows like Mythbusters, Penn and Teller: Bullsh!t and Scooby Doo are more popular than ever. These shows get right to the heart of skeptical thinking, “Just because someone said it, doesn’t mean it’s true.”

Skeptical thinking is a method, not a position. Officially, skepticism is defined as ‘a method of intellectual caution and suspended judgment’. A skeptic is not closed minded to new ideas, but is cautious of ideas that are presented without supporting evidence.

What would Scooby say about that product you’re considering?

In our hobby there are tons of ideas presented without much supporting evidence. Most claims that appear on products have “No visible means of support”. Being a skeptical reefer essentially boils down to taking advice/products/new ideas with a bucket of salt. Take your time to do some research. Follow up to get a handle on why, how and if ideas and products actually work – if at all.

Remember, a recommendation from someone is not evidence, neither are anecdotal claims. Checking the real data is hard but in the end it will save you a lot of trouble.

This is a pain in the butt. Why bother?

Two reasons: Animals’ lives and our money. The animals in our reefs rely on us to keep them alive. When we make bad choices, animals die. If you buy “Professor Polyp’s Fabu Everything Cure Juice” to stop a disease in your tank and it doesn’t work, your sick animals will probably get sicker and continue to go downhill. Luckily, most of the products on the reef keeping market won’t directly kill the animals in your tank, (there are exceptions so be careful) but if they don’t actually do what they claim you are just wasting your money, the time that might be used saving your animals and maybe even the animals themselves (and they aren’t an unlimited resource).

Why do people sell reefkeeping stuff?

In our hobby there is the feeling, since many of the product manufacturers are accessible through online forums and trade shows, that reef products exist first and foremost to help us be better reefkeepers. In the best cases this is 50% of the reason people are selling stuff.

In 9th grade I went on a fieldtrip with the staff of my high school newspaper to the LA Times newspaper offices. On our tour, the guide asked us why newspapers exist. My classmates and I, ever the optimists, chimed in: “To bring people the news,” “To keep the public informed,” and “To keep a watch on what our public officials are doing.” I’ll never forget the answer the guide gave us, “Newspapers exist to make money – everything else is ancillary.” That answer helped to stick the knife of practicality deep into whatever remained of my idealistic heart.

People sell reef products to make money.

Even if the original impetus for a product is to make a better ‘reef trap’, products don’t go to market, or at least they don’t stay on the market very long, if they don’t make money. There are just too many costs involved in getting products to market for it to be otherwise. Developing the products, the materials, the packaging, the labels, the shipping, the physical plant, the advertising, and the staff all cost money. Being in business is neither cheap nor easy. The vast majority of new businesses fail in the first year because they aren’t able to make it work.

I’d venture to say that most reef related companies start up because they want to do “good” but if the product doesn’t make money, that company has to stay in business somehow. They’ve spent a lot of money on setting up the business. They’ve got a web site, business cards, a printer and a garage full of crates of whatever they’re selling.

There are some reef companies who’s owners are independently wealthy or made millions by inventing something amazing like the singing greeting card or something equally esoteric, but even they have limits to how much they are willing to throw at a money-losing venture before they have to bail.

That’s a lot of products, how will you ever decide which to use?

In an effort to make sure that a product will be profitable some companies will do whatever they can. Do you blame them? They’ve spent their kid’s college fund creating “Professor Polyp’s Fabu Everything Cure Juice” and they need that money back. In an effort to do that they create advertising that can range from personal touches on the packaging to the outright untruths and everything in between. There is no overseeing body that checks the claims on most reefing products to make sure they are on the up-and-up; unlike most products for human consumption, a claim written on a package doesn’t mean its been verified.

We all want reefkeeping to be easy. The promise of a magic solution in a bottle that will take care of our reef problems is very enticing. Maybe someday a one bottle solution is going to be the panacea we want it to be but right now there is just too much going on in our tanks that we just don’t understand.

Cultivating a saltwater thumb by caring for a reef tank over time allows us to be sensitive to the ups and downs that occur in our glass boxes. Experience, often painfully acquired, teaches us to make changes slowly, so we can observe how our animals are affected.

Our need to feel like we’re doing something can lead us to jump on an untested product. It’s horrible to sit by and watch while things in our reef tanks are ‘off’ or ‘going south’. We don’t want to wait for our alkalinity to come down on its own, for the little bit of lymphosystis to run its course, or for algae to go through its cycle and die back. We give in to our desire to be in control and buy a product that we suspect doesn’t really do much, but, heck, it can’t hurt right? There have been a slough of “reef safe” products that have turned out to be anything but. It is my belief that using products that don’t do anything other than make you feel like you are doing something, makes it more likely that you will continue to use bottled fixes to the detriment of your animals.

Marketing

The best product in the world with poor packing will sell less than the worst product in the world with great packaging. Everything, from the shape and size of the packaging, the color of the label, the font of the text and the wording used is scrutinized until it is the most appealing it can be to the projected market. Is a candy colored skimmer going to skim better because of its color? It is important to note that not all advertising is trying to scam you. Some of it is simply trying to portray the product in the best light to get more people to buy it. As mentioned earlier, it can be difficult to tell the two apart.

Baloney Detection Kit for Reef Products

Carl Sagan first published his Baloney Detection Kit in his book ‘The Demon Haunted World.’ It was immediately helpful in codifying ways to ferret out faulty reasoning. Later, Skeptic Magazine’s Michael Shermer put together his version of a Baloney Detection Kit that lists a series of questions to ask when encountering any claim. Standing on the shoulders of those skeptic giants, I present the first version of the Reefkeeping Baloney Detection Kit, or the RBDK, for the acronym addicted. I hope it will help you decide if claims are solid or, well, baloney.

Did these colors come from a magic potion?

1. How reliable is the source of the claim?

Much of the time in reefkeeping the source of the claim is the manufacturer. You know that the manufacturer is trying to get you to buy their product. Are they a reputable manufacturer that has other products that you trust? What does the manufacturer claim this product will do? Is this a new manufacturer trying to capitalize on the newest reefkeeping fad? Are the claims supported by an ‘expert’ quote? Who is the expert and why should you listen to them? What kind of connection does the expert have to the manufacturer (i.e., are they getting paid, trying to get their name ‘out there’, or own part of the company?)? Is the expert support anecdotal (remember our old friend anecdote from the first installment?)? Was the expert quote taken out of context? Did the expert even say what the manufacturer says they said? If the source of the claim is not all that reliable, you need to dig further before you accept that claim.

2. Have the claims been verified?

Have there been any studies that support the claims? Our hobby is ‘sciency’, and for many of us, that is part of the fun. So, we often trust that the claims written on the packaging of reefkeeping products have been tested but it is hard to tell if that is indeed the case. If you are going to buy a product that claims to reduce nitrate and phosphate, you want to know that it actually works. Is there data to back the claim? A responsible company will have some kind of documentation to back up their claims and they should share it with you. An irresponsible company may be guessing at what the product does and hoping hobbyists will report back with success. If they have no documentation or support, you simply can’t be sure that the product performs as advertised. I’m going to tack on the opposite to this rule. Does an ad include weasel words like “benefit” or other vague claims? If the claims are weasely, or there is no verification of the claims, it might be better to move on to a different product.

3. Does this fit with how reef aquariums are generally thought to work (is it too good to be true)?

A product that promises to do away with regular maintenance like water changes or to completely ‘cycle’ a tank in days is offering something extraordinary. As the saying goes, extraordinary claims require extraordinary evidence. It’s pretty clear that water changes are a great way to get rid of pollutants that occur in just about every tank. It’s pretty clear that aquariums need time for bacteria responsible for the ‘nitrogen cycle’ need time to settle down, so a product that claims to bypass these ideas needs to have a lot of good supporting evidence to back it up.

4. Does the claim have an over complicated explanation?

A popular tactic for products that don’t actually have much if any support for their claims is to present overcomplicated and long explanations. These kinds of explanations often contain ‘scientificy’ sounding words that don’t actually amount to much. Often the reference experts or scientists don’t seem to exist or live in far off lands and have no published works. This is really an argument from authority covered in the previous installment. This type of advertising is designed to get you to assume that much work has been done on the product and you can therefore trust the claims. Such product claims should make you question the product even deeper.

5. Are the claims covered in ‘greenspeak’?

A popular advertising tactic is to claim that the products are all natural, green, eco friendly, herbal or good for the environment. It is often difficult to determine if the ideas behind the terms are supported. The terms themselves are so general as to border on mindlessness. Ask a few questions about an ad like this and the response you’ll get can be something akin to religious fervor. “How can you be against something that is good for the environment?” Somehow asking, “What does that mean?” is interpreted as evil. The truth is NO ONE is against things that are good for the environment. Some people that think this type of advertising is above reproach. Don’t be that person. Ask the questions. The animals that you care for, that were collected 10,000 miles from your home, deserve it don’t they?

6. Too many claims?

Some products give you a laundry list of ‘ailments’ they are supposed to address. Does a product claim to cure every conceivable fish disease? Does it claim to increase skimming efficiency, buffer ph, clean the front glass, and park your car? It strains credulity that one product can do so many things and such a product absolutely warrants more support before its purchase.

7. What do people you trust think of the claim?

Given the lack of supporting scientific evidence, you may have to fall back on the wide-ranging experience of other reefkeepers on the interwebs. If you are reading Reefs Magazine, chances are you have been around the reefing world for a while and have developed a feel for which reefers, both local and international, that you trust. Ask them what they think of a particular claim and see if it rings true for them or not. But be warned, it is easy to find support for just about any claim on the web. You need to be careful that you are not reading the parts you want to agree with, while ignoring the rest. Be aware that this trust can be as dangerous as it can be beneficial, as I have seen ideas that seem crazee gain huge followings.

Final thoughts

I have stayed away from discussing specific products. My purpose in writing these articles is not to pick on any one product or company (though a debunking team of independent ‘Reefbusters’™ would be fun – anyone interested in funding such a project let me know). Why don’t I discuss specific products? Simple, I don’t want to get sued by an angry company. Lets face it, being right or wrong in a lawsuit is almost irrelevant – the cost of dealing with even the smallest legal action can financially wreck a person. My real purpose in these articles is to get us all thinking and to give us tools to navigate the information that is constantly thrown at us. I do, however, have one example of a product that I can include because, well, it’s not real.

This product doesn’t exist.

A couple of years ago as an April Fools Day joke, Gresham Hendee and I used our advertising experience to put together a fake product. Our intention was to poke fun at our friend Jake Adams and his obsession with water flow (the idea was actually sparked by Brian Edwards who said that Jake thinks that ‘flow is more important than water’). We created a print ad and even went so far as to make up several bottles of the product to have on hand at frag swaps and conferences. Jake made a video advertisement for the product for Reefbuilders. Our fake product was called ‘Instaflow’, and the marketing claims in the promotional materials were based on claims from actual products.

We were very clear the product didn’t really exist. It says on the packaging that the product doesn’t exist and even though we thought it was obvious that the claims were so insane that no one would believe that this product actually existed, that the expert claims were so patently ridiculous that no one could believe they were real, Jake tells me that from time to time he gets email inquiries about the product’s availability. That’s marketing for you. Fear its power. Fight its power.

Next time – ‘old reefers’ tales examined, or something equally as interesting.

If you have a reefing subject that you think would be good to discuss in one of these articles, please let me know by emailing [email protected]

Links, further reading and references

– http://homepages.wmich.edu/~korista/baloney.html
– http://www.nizkor.org/features/falla…dex.html#index
– http://www.skeptic.com/
– “The Demon Haunted World: Science as a Candle in the Dark” by Carl Sagan, 1997 (Ballantine Books)