Compare vs. the Competition

At any given moment there are hundreds of trading systems that are available to the public. While there are indeed a few good ones out there, most trading systems will eventually fail. Unfortunately that is a fact. If you are like us and have tried over a dozen systems from numerous developers, only to be disappointed with most, (needless to mention the financial losses), then by now you probably know what to avoid. However, many among the unsuspecting public continue to fall victim to either unscrupulous developers, or systems that were developed by developers with good intentions, but somehow in the development process something went wrong and from the day it was born (released to the public), it already had the "genetic code" to blow up.

Regardless which system you go with, to help you in your search for a good system that puts the odds in your favor that over the long term you will end up making money, here are a few thoughts to consider. Few systems will make it through the bullet items that we have listed below. Those that do are systems that we would trust and trade ourselves, otherwise beware!

Thoughts to Consider:

• Post-Release Performance: how long has the system been out? Post-Release Performance is the most important performance record since this is data that was not available when the system was developed - it hadn't happened yet. Unfortunately the majority of systems break down and go by the wayside a couple of years after release. When they are launched they come on with strong advertising claims to back them up. Being the new sensation, the brokers jump all over them and push them on their clients. Everything starts off great the first year, then Ok the second year. By the third year they are discontinued by the developer, or more likely some new version of them comes out with a new name (where obviously they have been changed to keep the back-tested historical performance results looking good). This is the typical pattern seen repeated again and again and again.
  
  Sometimes developers will release subsequent versions with the excuse that market conditions have changed and thus the system "needed updating". This is most likely tampering with the parameter values to re-optimize / re-curve fit the systems so that in hypothetical historical performance continues to look good. Yet in other even more serious cases the developer will toss away the old system and launch a new one every couple of years with a brand new name. Be very suspicious of this. If a system isn't broken then why fix it with a new version every couple of years, or get rid of it?

• Historical Track Record: while it is obviously better to go with a system that has proven itself in post-release performance, obviously a good new or newer system will not have this by no fault of its own. After all, a new system won't have much of a post-release performance. In such a case, the only alternative remaining is to see how far back its back-testing historical results go. It is amusing to see systems that only show the last couple of years or no more than 5 years! What about before then? How would the system have performed before then? It leads one to be suspicious. We recommend at a minimum 10 to 20 years of data.

• Fully Disclosed / Totally Transparent Systems: this is the second most important item to consider after performance track records. The vast majority of systems these days are sold in closed "black box" format, meaning that you the customer does not have access to the logic, i.e. the trading rules, the source code. You are meant to blindly trust the system without knowing how it works, or perhaps in some cases only "partially" knowing its algorithm logic. Developers of such systems claim that they cannot fully open up their system due to the dangers of piracy and plagiarism that is so prevalent over the internet these days. They are protecting their intellectual property that they worked so hard on to develop. Now this is a legitimate excuse - it is totally understandable and in some cases legitimate, however, it is also a convenient excuse to produce a system that is massively optimized to the n'th degree and curve-fit. Then the system is packaged and distributed as a black box and sold to the unsuspecting public.
  
  We feel that the more a trader knows about the system the better. Yes there is the risk for us developers of falling victim to piracy, plagiarism and copyright violations, hence it is a difficult choice for developers to make. Hardly anyone makes their intellectual property totally open, no matter what it is, being a trading system or a gamming software. Nevertheless, if you are going to put the trader first, and give him or her the benefit of the doubt, then clearly it serves the trader to fully understand what he or she is trading, vs. just blindly following the trading signals that a black box system generates.

• Single vs. Multi Market Systems: this is another area where caution is warranted. Beware of systems that are designed to trade only one specific market or just one specific sector. The risk is higher that once again such systems have been curve-fit, either intentionally or inadvertently. While not profitable in every single market, a good system should show good results over a broad and diverse spectrum of markets - always applying the exact same rules and parameter values across all markets! Otherwise, if the parameter values are tweaked based on the market, what you really have is many "sub systems" within a system - one for each market, which once again leads to curve-fitting. An example would be a system that let's say uses moving averages where the period used for the Swiss Franc is 29 days while for Corn it is 17 days. Why the difference? Furthermore, if the system is a closed black box type systems (as most are), then there is simply no way for you to know.

• Non-Optimized: this is a key point that we have been emphasizing all along. A system cannot be optimized for what worked best in the past is not what will work best in the future. Markets constantly change. A good system built on solid concepts should not fall apart when markets change, but should adapt. Yes there will be losing periods, known as drawdowns in the industry. All systems and traders alike experience them, even the very best ones since nobody wins every day, but the good ones eventually recover and go on to make new equity highs. If a system has been optimized to yield the best possible results based on historical past data, it is simply impossible that those results will be consistently repeated in future data that has not yet happened.
  
  When it comes to optimizing based on the market being traded, some will argue that each market has its own personality and characteristics. After all, the 10 Treasury Year Note is very different than Cotton - right? Nothing could be more different. Yet how do you explain those systems that do manage to produce profitable results across a broad & diverse spectrum of different markets, while always applying the exact same rule formulas and parameter values for all markets?

• Simple systems with simple set of rules with few parameters. Every now and then you come across a system that has all the criteria mentioned thus far: good track record, fully disclosed, trades multiple markets without optimization, etc... then when you go to learn about its trading rules or take a look at its source code to understand how it works it is like reading Martian! They are so complex and made up of numerous convoluted formulas that only a PhD. in mathematics can decipher the logic. We believe in keeping things simple, for over time simple systems have actually performed much more consistently than those complex ones. We believe in being able to explain a system's logic in plain English to someone with nothing more than a high school education. FACT: the more simple a system is, and the fewer formulas it has, the more likely it is going to perform in the future as it has in the past. So don't give much credence to all those fancy, high-tech sounding terms used to describe how "sophisticated" or advanced a system is that it can fly a space rocket, on the contrary - run from them!

• Easy to trade: obviously this helps. What kind of data does the system use? Is it intraday or end of day? Do you have to watch it during the day, which is not practical for most folks. Of course, you can always turn it over to a broker to run and trade the system for you, but even then, the easier it is for the broker the fewer the mistakes he or she will make, which like it or not do show up on your account. A good system should also minimize slippage. Shorter term systems, especially day-trading ones can be enormously impacted if they suffer even the slightest slippage. Those systems assume good order fills and thus look great on paper, but once they are traded live and begin to suffer from just a few ticks of slippage per trade, there goes all that fantastic performance out the window! A good system should be able to sustain significant slippage and still produce good results over the long term.

• Slippage & Commission Deduction: any system whose performance track record does not include a generous deduction on a per trade per contract basis for slippage and broker commission is simply not living in reality. It is amusing to see systems, most notably day trading ones that factor in only a $5 or $10 deduction for commission & slippage. When this value is increased to just $20 it then wipes away a significant portion of the systems profitability. Make sure that more than enough is deducted on a per trade per contract basis for commission & slippage without destroying profits. We use $50 for single contract examples, and $100 on a per contract basis for multi-contract, position sizing / money management examples.

• Totally Symmetric Trading Logic: no bias towards long or short trades and can go short just as easily as long. It is amazing to see that systems are still being designed to trade in just one direction. The argument is that the market they trade has mostly gone only one way in the past few years - usually up. What happens if that changes? What happens if that changes and what was thought to go only one way suddenly changes and goes the opposite way? The so called geniuses in the sophisticated financial industry got it wrong since amazingly it even happened in the Real Estate market!!

• Tested and Verified by Futures Truth, an independent third party entity dedicated to testing and tracking trading systems. If a developer has refused to submit their system to a respected institution like Futures Truth for an independent, un-biassed third party evaluation it leads to concern. Some developers claim that they do not like to do this because it frequently requires that Futures Truth have access to the system's source code. All that can be said is that based on our experience of more than a decade with Futures Truth, to our knowledge not once have they revealed a system's logic or source code to anyone without prior authorization from the developer. We recommend you obtain a private opinion letter on Andromeda & Pegasus from Futures Truth (1-828-697-0273) along with their own up to date test results on our systems.

• Cost of the System: we left this for last since this is the only item on this list which should be irrelevant - the price tag of the system. Nevertheless, as human beings it is in our nature to worry about how much we are paying for something that we are purchasing and is it worth it. Keep in mind that what you pay for a system is insignificant compared to how much you can win or lose by trading the system. If a system costs only $100 but you end up losing $30 thousand before you quit in disgust, your real cost was $30,100. Going for cheap can backfire. On the other hand, an expensive system does not imply that it is a better system. We have seen systems that are distributed completely for free - like tutorial sample systems that are included in $25 trading text books or ship as tutorial examples of trading platforms that have handily outperform the most expensive systems. Price has no relationship to quality when it comes to trading systems - absolutely none. Don't concern yourself with the price tag of a system, either way, be it cheap or expensive, and focus on all the other items listed above. In any case, Andromeda & Pegasus are competitively priced and fall in the middle of the typical price ranges of systems that are sold to the public.

Futures Truth Comparison Study

We asked Futures Truth to conduct an identical "apples to apples" test on several of our well known competitors and match them up against Andromeda. The result was a test on 4 systems that belong to the same category: long term multi market systems. The test was applied on over 40 markets - no "cherry picking" of only the good performers. It was conducted on as much data that Futures Truth had available: 34 years or whenever a market first began trading. All systems were tested under the exact same conditions, on the exact same portfolios, the exact same time frame and exact same data. This is the only way to perform a fair comparison. All results are based on single contracts on as many markets and as much data as possible that Futures Truth had.

Futures Truth is an independent third party dedicated to testing, evaluating, tracking and ranking trading systems. They may be reached at 828-697-0273 (USA) or via email at info@futurestruth.com. We encourage you to order a "Private Opinion Letter" from Futures Truth on Andromeda, Pegasus or any system that you may be considering. The Private Opinion Letter also includes detailed performance reports also compiled by Futures Truth.

Following is the summary of the test results.

*There are several Turtle type versions out there. The results posted are for version that we got from Futures Truth.

How to evaluate these results:

If you select a system purely on its profits you are making a very expensive mistake. You must include drawdowns for they represent risk. Drawdowns will also determine how much money you must commit to an account in order to trade a specific system on a specific portfolio. The best way to evaluate systems is to look at the profits to drawdown ratios. Here you are including both performance in terms of profits and risk in terms of drawdowns. The higher the ratio the better.

Andromeda had the highest profit to drawdown ratios:

• The total profits to max drawdown ratio was 23.5 – the highest figure (see middle column)
• The ratio regarding average profit per year to average worst drawdown per year (34 years) was 1.31. See far right column. Andromeda was the only one with a ratio above 1. This translates to say that Andromeda was the only one whose average profits per year were higher than the average worst drawdowns per year (for the 34 years tested)