Trading Research

There is no substitute for statistically valid historical research when developing mechanical trading systems.

In practice, this means learning how to program a computer to run simulations of trading system performance.
There is a lot of good information on curve-fitting, over-optimization, trading statistics and testing methodologies on the web and in books, but the information is a bit hard to find amongst the hype and bull. Be skeptical, but keep an open mind, and your research will pay off.