I'm actually glad no one mentioned this because this event in and of itself was overhyped publicity. The Open AI bot had hard coded behaviors and was defeated handily within hours of being available to the public. It's since been iterated upon and there's still a handful of pros who are regularly beating it WITHOUT exploiting the holes in the logic. This is effectively just as weak as the article regarding facebook.
> and was defeated handily within hours of being available to the public.
The public exploited tricks to beat it. They did not beat it 'handily'.
Afterwards, the pros who do beat it only manage to do so ~2-3 times for every 100 games played. I believe they have been playing the same version that was shown at The International and not an iterated version.
Correct, we've been playing a number of pros using the same bot played at TI. We do have a stronger version which is just two days more of training (gets a 70% win rate vs the one at TI), but haven't seen a need to test it out. We'll likely do a blog post in upcoming weeks with more stats and commentary from the pros; would be curious what people would like to know!
Incidentally, the various exploits that people used are all similar to how we actually develop the bot. We try to find areas of the strategy space it hasn't explored, and then make a small tweak to encourage it to explore that area. Lots of progress comes from removing hardcoded restrictions, which are nice to get started. So the fact there exist exploits wasn't surprising to us — what would be surprising would be exploits we couldn't fix.
1v1 has always been a proof-of-concept for us. The fun part is really 5v5, which is what we're working on now (and hiring for! ping me if you're interested: gdb@openai.com).
I understand this is a perfectly reasonable early test, but there are so many complaints about "it was just a restricted subset of the game and 1v1".
This is like complaining that Google doesn't release first-pass code (with minimal unit tests and no stress testing) to their production sites across the world. Everything that loops starts with the first iteration.
Also, keep up the good work, OpenAI! And please remember Asimov's 3 rules.
