So please consider subscribing to the show because it gets me really good data and lets me know the kind of things that you’re interested in so that this way I can continue to create hopefully valuable content, not waste your time, even if it’s only for five, 10 minutes a day. Also, if you go to the top corner of MartinKronicle, you can download a free copy of the audiobook version of The Inner Voice of Trading, which I give away for free.
Today I want to talk quickly about backtesting. I have harped on it quite a bit, so I thought I’d do some pros and cons and there are some cons. So when you think about back testing, there’s some simulators out there where you can test one idea at a time, meaning one name over maybe a few years to see what indicators you were using and what your entry and exit criteria were, what your position sizing algorithm is. I think those are the worst of them in that there’s not enough data for you to be able to tell how things would work. Two years is not enough time. Generally speaking, if you speak to those of us in the business you want probably 10 years of data or more so that you can see how the name reacted under different economic scenarios. You might be onto something now, but that market might reflect higher interest rates. Where if you were going back say 10 or 15 years, you were in a low to no interest rate environment, right?
So then you go back even 20 years. And that takes us back to 2002. It’s a post 9/11 environment. How did things work? Especially during this 2007, 2008 subprime morass – emphasis on the second syllable. So when you do that, you give yourself an idea of how the name would’ve performed, not even the name, but especially what your entry and exit, what your rules did. The problem with testing one idea at a time though, is that it doesn’t, it’s not terribly robust. It doesn’t teach you how the whole portfolio would’ve done. And I don’t really advocate trading just one instrument. So for those of you that are sitting at home and you’re just trade, like any of the indices, especially the emini, this episode’s going to be a waste of your time. So if you clicked off I’ll save you three minutes.
Now there are engines that you can use to back test at the portfolio level. And those are Mechanica and Trading Blox, which again are just the engines, the spreadsheet macros if you will, where you can set the parameters and then hit “Go.” Of course you need data. So you have to go to a data provider that will give you clean data in a format that’s compatible with the trading engine and then upload it and run it that way. Now those aren’t necessarily expensive, but they’re not cheap, right? So if you have $5,000 account, you’re not going to do this because the simulators probably run a few hundred to a few thousand. You know, if you want the full versions, they have various levels of service and abilities. I think the lower end ones are like $700.
The higher end ones run you between $2-$3k. And then the data that you want will probably run you anywhere from $50 to a couple hundred dollars a month. So you have to figure out what you want to do there in terms of running your business or what if you want to, especially if you want to run other people’s money, then you have to look at the data itself as the data clean, are there bad prints? How do they fix them? If there are, then you also have to look at the survivors, right? Because if you’re looking at what’s trading and you type in a ticker symbol, that’s a survivor. But what happens if you type in, and I’ve mentioned this before, ENE which was Enron, or BKB, which was Bank of Boston, which didn’t go out of business, but was acquired by Fleet Bank.
You know what happens if you put in BSC for Bear Stearns Corporation, right? Those names aren’t going to be in your normal data feed. So you’d want to put those back in because it’s very valuable to know how your model would’ve worked if it had the opportunity to pick stocks that either got acquired or blew up, right? Otherwise you’re just looking through rose colored glasses. [And if you knew Rose the way I did, you wouldn’t want her eyeglasses anyway.] So that’s the backtesting thing. So be careful on these single name for two year thingies, because it could get you thinking that you’re onto something when you’re actually not now most of the time. If they give you two years of data, you might be looking at five minute bars or one minute bars. And you’re like, “Hey, five minute bars over two years and off a big sample space, I’m still going to argue that you’d want to see five minute bars again during 2007 and 2008 when times were darker.
And the mood and the atmosphere around Wall Street was negative. That’s very important data to see, because if you start losing money, your behavior may change, especially if you don’t have a lot of experience. So having said all that, when you look at the output, you run one of these things and it’ll give you a big report. It’ll give you all kinds of ratios and things that would help a person diagnose the problems and where to make the tweaks, but also what are the financial indicators as to whether or not the system should be followed. So in and around that data, the most important thing is probably the expected value of a trade. Some of you who are into the accuracy game can see what’s the frequency of winning versus losing across all the trades that were taken. What’s the size of the average winner versus the average loser.
What’s the biggest winner. What’s the biggest loser, Sortino and sharp ratios. And then also very, very importantly, what was the biggest draw down and how long did it last? Some of them like Trading Blox will show you all the drawdowns over that period of time. So you can kind of see how it worked and how well the model’s still working, even if you’re in a drawdown. But a lot of people take that as the model not working, there will be times over your trading career where the market is just not amenable to your trading style. And that’s just the way that it works. So the big question is how do you survive those periods of time? Do you cut your position size? Do you trade less frequently or do you try to bull your way through it and double your positions to earn the money back, which is typically not recommended.
So there are some cons in that a lot of those numbers that you get are actually averages. So it’s like the old question in calculus class: “A car starts in Los Angeles and drives to Chicago. And over that period of time, the car averaged 60 miles an hour. What can you say about the trip?” And all you can really say about that is that at one instance in time, the car traveled at 60 miles an hour. So when you look at your trading results from your back testing, what they say would be your compounded annual growth rate is generally an average and you might not actually ever achieve that. You might get more than that. You might get less than that, but that’s true for a lot of those indicators, some of the better engines have a Monte Carlo function where they’ll vary the start date, which you know, helps to understand that there’s a lot of randomness that goes into success.
The one that I use, that’s a striking example. That’s easy for everyone to understand is you sell a house, you had $300k and in the 1980s, and you decided to invest that money in a simple blue chip S&P 500 fund on October 1st, 1987, versus having the sale close and having the funds available on November 1st, 1987. So just by varying the start date by 30 days, you would’ve side stepped, for example, the big crash on black Monday, right?
Random, but that’s important stuff to know, because there’s certain things in actually quite a few of them that are out of your control. That just happened. You know, sometimes people get shot by friendly fire. It’s unfortunate and it sucks, but it is the way it. So then you have to figure out for all the time, money and effort that goes into the back testing. What kind of peace of mind is it have togive you when 80% of trading is psychological and emotional, would it be valuable for you to know ahead of time that your system was robust and it worked across large, small and mid-caps over a 20 year period of time, right?
You also have to think about data mining and curve fitting and all of this and that, which is too much to get into here. But I mean, you have to figure out what’s the benefit to you. When you look at the backtest, does it give you peace of mind or is it a waste of time? Because things are so random. Anyway, it really comes down to the individual. Some people want a little bit more science and other people have a much more, “well I’m going to wing it and learn as I go” which is kind of what I did. And I’m just going to keep my losses small because it’s this experience of the trading. That’s going to give me the education that I need. And although I don’t know my tactics or my trading so well, I will figure it out along the way. You have to have to have a special disposition the way I have it.
I was kind of born with it. So I’m lucky that way, where I don’t look at losses as personal indictments on stuff. I know I have to try. And it’s the trying that you win. It’s not even if you lose money because you did it, you still won. So you have to figure out what’s your temperament. What kind of science do you need? If you’re a wing-it kind of person, you could also find yourself buying some of the dog shit that we talked about yesterday, because you like the words that the people chose to kind of convince you to give them money. I can assure you, there is no easy way out nor is there an easy way in. I think there’s a lot of ways that you could turn this into a way to vaporize a ton of cash, but, and that’s why I say measure eight times and cut once because the money you save is money that you don’t have to earn back.
Two long episodes back to back. I’m sorry about that. I like to be much more brief, but I think these yesterday in today’s issues are very important to me, as I do feel whether it’s right or wrong, a bit of a paternal kind of vibe for the newer folks, because left to your own devices, you could easily step into some very dangerous situations.
And I was in that spot too many decades ago, but, and I didn’t have anybody to help me. So I feel like it’s my knowing what I know. It would be very selfish to not share that information with the community.
This is a computer generated transcript.
Click here to get your free copy of The Inner Voice of Trading audiobook.