I will be slow to post for a while. I got authorisation from my beautiful wife to cut my regular hours to part-time so I could work on some research for a while. Initially I thought I would have several months (many years ago she gave me a year to start a software ‘biz; I had a lot of fun, but didn’t make any money, so rejoined the ranks of the working stiffs after 9 months), but she has been getting wishy washy on me and now I think I am down to a few weeks. I am putting in three – half days on my regular job, so have lots of time if I can focus. My employer, the contracting company and the agency all graciously allowed me this time, I really appreciate it. Not only does this mean I don’t have to look for a new job, but I keep my clearance active and keep some (about 30%) income coming in.
Initially, when I thought I had several months, I was planning on working on my image compression / meta-data extraction program, but given what appear to be the new time constraints I didn’t figure I would make enough progress to be able to keep motivated to continue work once I was back to ‘slaving’ 40 hours a week. Instead I opted to work on my securities prediction algorithm I have been noodling for nearly 20 years. When I was in my stats class during my MBA program (’94, I think, so 18 years) my professor talked about an approach he was using that would enable him to predict, with very high accuracy, the next value in a chaotic dataset (it was nearly useless for predicting any past that, though). He wanted someone to take the time to apply the technique to the stock market and I was quite interested in doing so, but my programming skills (if one were to care to dignify my level of ability at the time as a ‘skill’) were such that about all I could do was fantasize. Later, as my skills increased, I ran up on the Catch-22 situation I have been battling with pretty much since I became a professional programmer. I only really have 6 or so hours of programming ability in me on a daily basis and that was pretty much exhausted by the time I got home from work (I did start a couple of times, but the approach I was using at the time was error prone and I never worked through all the bugs). I wonder, now, why I didn’t think of pursuing it during my 9 month hiatus I mentioned above, but what I recall was I was intending to build an evolutionary algorithm-based shift-scheduling algorithm and when I was having problems with that I decided to switch careers (unsuccessfully, as it turns out) to information security. Anyway, over the years I have refined the idea substantially and think I have simplified the overall approach to the point where it is practical to have the core of the algorithm built in just a couple of weeks. I feel confident I can test the program working 40 hours, it can be running while I am sleeping and while I am at work, so really I wouldn’t have much to do anyway.
The basic approach (which based on some Googling isn’t that novel any more, I just hope that my implementation is successful) is to use genetic programming techniques coupled with machine learning to produce an algorithm to predict future trends in the price of market securities (stocks, bonds, etc.). It is an ideal programming technique for the lazy bastard that I am as the program is doing all the work and I just sit back and watch. The core of the program (the same core, btw, that is in my image compression algorithm, so even without any predictive success on the markets my time is not wasted) randomly generates programs (not actual binary, though I have given a lot of thought to that level of optimisation, but a sort of byte code that is interpreted), then evaluates each program against a known data set (where the machine learning comes in) and using the power of evolutionary computation the algorithm evolves until it can predict (or rather post-dict) the known data set. Knowing the power of this programming technique there is no doubt in my mind that it will, with appropriate tweaking of the evolutionary parameters, be able to produce an algorithm that will post-dict every stock price for, say, DELL, and I think there is an excellent chance it can predict the entire market. The question is does the algorithm have any sort of predictive value whatsoever, something that will probably take weeks or even months of testing to determine. Possibly it can predict long-term market trends, possibly it can predict with enough certainty most stocks next closing value to make a few bucks, possibly, of course, not a damn thing. If it does work, though, it should be like legally minting money!
I will be sure to let my reader(s) know if I get anything that looks promising. My plan, if that fantasy is realised, is to build a hedge fund around it. I have read a lot about hedge funds and they look like the perfect thing for me, presuming, of course, I can find enough people to pony up $100 million or so.
Wish me luck!
