Like many other games that are on a table and which are governed by clear and shared rules, even poker has received the attention of those who deal with artificial intelligence systems. In particular, its Texas hold 'em variant, which was codified in Texas (eh already) in the 1960s and which has long been the most popular. Compared to games such as chess, or the very difficult go, Texas hold 'em has some peculiarities: first of all, up to a maximum of ten players can play it simultaneously and, more importantly, every player and every artificial intelligence player is all obscure of the cards in the hands of each opponent. The table, in other words, does not contain all the information necessary to develop a winning strategy.
Despite these peculiarities, even Texas hold 'em has had a fate similar to that of chess and go and, as the New York Times Magazine wrote, “he was conquered by artificial intelligences.” However, more than in other games, there remain pockets of resistance to this conquest, and considerable space for a human approach.
Texas hold 'em – often also called just hold' em – provides that all at the beginning of a hand each player receives two cards from a deck of 52. They are the so-called hole cards and each player looks at his own but does not know those of his opponents. At this point, in a series of successive phases, five common cards are added to the table for all players, who seeing first three (flop) and then, one at a time, the next two (turn and river) can decide whether to bet, tie, raise or fold, in this case leaving the previous bets on the table. The aim is to use one's cards and community cards to obtain, according to the rules of poker, the best possible combination of five cards.
There are also variations, specific terms and various complications, but the the gist is that you have to get to the end, episode after episode, and hope for two outcomes: that the others, not convinced by their cards, drop out first; or, if someone is still in the game, that one or two of their cards guarantee, if combined with the common ones, the victory of the hand and therefore of the pot, i.e. the sum of all the bets made by the players during the hand.
Since poker existed before the rules of Texas hold 'em were codified, already in the first half of the twentieth century, on the wave of studies and reasoning on game theory, there were the first attempts to develop effective models aimed at understanding what an optimal game strategy was for each player. How convenient poker could be to game theory is well explained in a sentence by mathematician John von Neumann, co-author – together with economist Oskar Morgenstern – of the book Theory of Games and Economic Behavior. Complaining about how unsuitable many games were for his purposes, von Neumann said: “Real life is made up of deceptions, little lies, it's full of moments when we wonder what others think I want to do, and that's what you play in my theory. “
The deceptions, bluffs and incomplete information that each player has about everyone else are also what for decades made poker a very human activity, in which , as the New York Times Magazine wrote, “even computers had to limit themselves to interpreting each other's plays without knowing for sure which cards they had in their hands.”
Neil Burch, a computer scientist who now works for the artificial intelligence company DeepMind and who had previously spent a couple of decades studying poker as a researcher at the University of Alberta, told the New York Times Magazine that for a long time the results were disappointing: “If we put a good poker player against a computer and we made them play, the computer fell apart. “
As explained by the New York Times Magazine, those who deal with game theory use, to represent the various avenues that a game can take, a tree diagram. “In games like paper-scissor-rock, the tree is small: three branches for paper, scissor and rock, each with three branches for each of the opponent's three possible moves.” For Texas hold 'em, even if you want to take a simple version, with only two players and with predetermined bets and winnings, the tree has 316,000,000,000,000,000 branches, that is 316 million billion branches. In the case of even more complex versions, Burch said, the branches become even greater than the number of atoms in the universe.
Despite the many zeros of these numbers, for some years artificial intelligences have succeeded to conquer even poker.
Computer scientists and researchers started from simplified versions, in which for example a pair of 9s was considered equivalent to a pair of 10s, so as to allow artificial intelligences to take the but no. Then they raised the bar, mainly thanks to an algorithm based on counterfactual regret minimization, the counterfactual minimization of regrets.
In other words, again those of the New York Times Magazine, told to identify optimal strategies by playing against themselves billions of times and noting which decisions, which streets in the game tree, had been least profitable. ” In short, they were told to remember which moves had generated regrets, so as to minimize them in subsequent moves, and it worked, as explained in an academic article published in Science in 2015. All of this, however, continued to concern the one-on-one hold 'em games. one and with fixed bets.
That article – entitled “The one-on-one hold 'em limit has been solved” – may not say much to many, but for those who played online poker (an activity already very popular in 2015) was a kind of DeepBlue-beat-Kasparov moment. «I remember well» said former professional poker player Terrence Chan «that after reading it, I and others said to each other 'it was good while it lasted, but now it's over'”.
– Read also: Deep Blue 1 – 0 Garry Kasparov
Things weren't that drastic, but they sure gave online poker a big hit. Between those who wanted to develop programs to win online, and those who wanted to develop programs to understand who used other programs to win online, there was a certain commotion. Also in 2016, Polish programmer and former poker player Piotrek Lopusiewicz began selling PioSOLVER online for a couple of hundred dollars, a program that “approximated the solutions for the more complicated versions of the game”.
Since then, PioSOLVER has been used, along with other similar programs, by many professional players and these programs, as the New York Times Magazine wrote, “have radically transformed the way poker is played, especially at its highest levels “. The programs, also known as solvers (solvers) are now quite accessible and it is very difficult to find high-level players who do not use them before or after a game, to prepare or to critically analyze their hands.
One of them is Jason Koon, who has been playing at a high level since 2006 and has amassed tens of millions of dollars in prizes in his career. “In the pre-solver era,” he said, “I was a decent player, but from the moment the solvers arrived, I worked a lot and started improving very, very, very quickly”.
Specifically, Koon uses PioSOLVER to train himself to understand, at any given moment, which is the optimal choice, that is, the one that – based on the limited information available to the software – offers the greatest chance of success.
Indeed, the solvers are able to suggest which bets are better to make and, in their own way, to evaluate even if and when it may be the case to bluff, that is behaving in such a way as to make others believe they are having cards you don't have. During the games, Koon and all those like him who use solvers obviously cannot consult them, they can however – as chess players do after training with computers – try to do what, according to their memory and experience, a solver would do. .
However, poker players must also keep in mind that solvers tend to act by assuming they are playing against other players intent on optimizing their play. Simplifying a bit, it can therefore happen that inexperienced players or very skilled in doing non-solver things can to some extent send the solvers or those who try to emulate them into crisis. «Against weaker players» wrote the New York Times Magazine «Koons sometimes deviates from theoretically perfect poker, bluffing more than necessary or betting heavy when instead the artificial intelligence would suggest being cautious in order to better exploit the mistakes of others».
Simplifying even more, software has the advantage of not being emotional, of not getting excited after many good hands or darkening after it runs badly for a while; but also, at times, the disadvantage of being too analytical.
In some of their quick and simplified versions, solvers can also be used online: there are already cases of players suspected of having used them and, if in doubt, there are those who now participate almost only in live tournaments, despite many sites making an effort to prevent this from happening. However, it seems to be one of those cases in which prohibitions and counter-moves are unlikely to be able to cope with ever greater, better and faster ways to cheat.
The New York Times Magazine also wrote that the research of optimal strategies similar to those of solvers has changed the way of playing hold 'em at the highest level, in many ways making talent less important and study more important. Doug Polk, who pretty much stopped playing in 2017, said: «I think that have taken the soul out of poker: before there was a lot of room for creativity, now it's just a matter of memorizing things and putting them into practice. “
For Lopusiewicz, the developer of PioSOLVER, solvers are not other than “a more powerful weapon” in a context, based on what is essentially a game of chance, in which there had been an arms race for some time “, for example with software that made it possible to analyze and study every previous move of every possible opponent. He has a similar idea Koon, who said: “Solvers don't tell you why they do what they do, they just do it; it's up to you to understand why “.
– Read also: The world of bridge against an Italian
