If you've been following this blog, you would know that I am running an experiment that started at the beginning of the baseball season. I picked up a copy of Out of the Park (OOTP) baseball, which is a fun game where you get to manage any team you desire. The software has many features, but the one the caught my attention is the Simulator Module in the software.
I read how this simulator was able to predict the past two seasons World Series wins. For both seasons, the simulator was able to predict most of the teams that made it to the playoffs for both years.
This got me wondering how the simulator might perform when run on a daily basis. I started from the beginning of baseball season each day compared the scores of the actual games with the prediction produced by the simulator. I haven't done a full analysis on the percentages yet, but it seems to me that the simulator is average around 65%. It's on my to-do list to compile the data that I have captured and analyse the percentages.
I wanted to try this experiment for last year's season as I also bought a copy of last year's simulator. For that version, however, it did not update the player and team stats in real time. That would have meant having to update each of these entities for every time every day. You can imagine how difficult (if not impossible) that task would be.
Want Your Own Copy of the Latest Version of Out of the Park Baseball? I am authorized to sell this software and you can obtain your copy here. If you purchase through this website, I will receive a commission. The price is not affected by this arrangement and it helps in a small way to keep this website rolling. Besides, it's a fun game to play!
This year, the Simulator Module has access to stats for each day. It's not real time in the sense that as the players are playing, the stats are updated at that point in time. It's a daily update and that is exactly what I need for this experiment.
While I found the results of this experiment interesting to date, the value of publishing results on a daily basis seems to be diminishing. It's becoming a bit ho hum for some of my readers. But, there is another reason why I've decided to make a change to the frequency of publishing. I received too many comments from people who say they are using the predictions as a way to gamble somehow. I am not quite sure how they would pull this off as the Simulator Module has not produced anywhere near 100% predictions correctly.
I don't gamble and my motivation for this experiment is to test the viability of the simulator to produce accurate predictions. The developers of the software stated that the simulator is based on an AI engine. If you know anything about AI engines, you would know that they usually get better with more data. I haven't seen improvements in prediction accuracy yet. In fact, thus far the past few days have been more incorrect predictions than correct ones.
I admit to not knowing how or why the simulator behaves other than what I see when I run the results. I first thought I may not be running the Simulation Module correctly, but I have seen YouTube videos by a few authoritative OOTP people run the simulator the same way. The OOTP manuals aren't much help in this area.
Here are the changes I have decided to implement. Instead of a daily publishing schedule, I will now publish on a weekly frequency. I will continue to run the simulator and record how the predictions did on a daily basis. However, I won't publish these results until the end of the week. My next scheduled publication is Friday, April 19 2019.
I haven't yet determined how I'll format the posts, but I will come up with something decent. By publishing weekly, I can report on whether the simulator is learning from is incorrect predictions or if the best we can expect is 60% or more.
This change in frequency should also discourage people from using the predictions to attempt to make money. That had never been my motivation with this experiment. If people try to use my predictions they will be doing so after the games will be played, which would be foolish.
Hopefully, my change in publishing frequency renews the spirit of my eaders regarding the experiment. I plan on finishing this out for the season, at least at this point in time.
James is a data science writer who has several years' experience in writing and technology. He helps others who are trying to break into the technology field like data science. If this is something you've been trying to do, you've come to the right place. You'll find resources to help you accomplish this.