You are right the peltier cooler can't toch the water, you would need to use a heat exchanger. The glass would not work well as it would act as an insulator. I have been looking at the heat exchangers used in PC cooling and they are all copper or aluminum. I found a promising heat exchanger at...
I too am looking into the peltier coolers as used on PC chips. I think they may have the capacity to bring your tank water down a couple of degrees however I have not been able to find components without copper.
Another idea is idea is a liquid cooler. You would use a fan to blow ambient air...
Does anyone know a place that sells plastic, 1/4", 12 volt solenoid valves that could be used for RO/DI control or tank top off control? I have searched the web and all I find are 3/4" or larger 120 volt valves for irigation systems.