back to article What an ACE-HOLE! This super-software will whip you at poker, hands down

Researchers at the University of Alberta, Canada, think they've made the perfect poker-playing program – and are inviting people to try their hand against it. The software, dubbed Cepheus, is a machine-learning system that has been taught to play a variant of Texas Hold 'em called heads-up limit, where players can only bet …

  1. petef

    Numbers printed without challenge

    I looked at a number of news sources and they all seem to repeat the same misrepresentation of the numbers. It is said that 4000 CPUs were playing 6 billion hands per second for an aggregate of 24 trillion hands per second. Bloomberg manages 28.8 trillion, with all that precision they must be accurate ;-) Those "hands per second" sound awfully like clock speeds, perhaps dual core at 3 GHz.

    Similar research reports tens of thousands of hands per second per CPU.

    1. Cliff

      Re: Numbers printed without challenge

      They should give up poker and move into chip design!

      1. ratfox

        Re: Numbers printed without challenge

        Poker chip design, natch.

        The website is not working for me. It must have overloaded…

        1. chivo243 Silver badge

          Re: Numbers printed without challenge

          Maybe IE only? I've tried Safari, Chrome and Firefox on my iMac, and blank screen.

          1. theOtherJT Silver badge

            Re: Numbers printed without challenge

            It's been slashdotted. It'll probably come back in a few hours once everyone loses interest.

          2. Preston Munchensonton
            Boffin

            Re: Numbers printed without challenge

            It's not an IE vs Firefox/Chrome thing. The application is actively refusing new connections on its special TCP port.

    2. Justin Pasher

      Re: Numbers printed without challenge

      Exactly. The math is off somewhere (or the description of the numbers)

      4,000 CPUs each doing 6 billions hands a second = 24,000 billion hands a second

      They say more a billion billion hands (1,000,000,000 billion)

      1,000,000,000 / 24,000 = 41,666 seconds (less than half a day)

      They ran the simulation for 2 months = about 5,184,000 seconds

      At that rate, it would calculate about 124 billion billion hands.

      As suggested, clock cycles makes much more sense.

    3. DLKirkwood
      Alien

      Re: Numbers printed without challenge

      I do research and find it is wise to always look at who is presenting news to see how legitimate their information is; these days people think bloggers are proper journalists just because they can give an opinion linked to a legitimate website. Also, (at least here in the US) you really need to get news from Reuters or BBC, or ??? and not rely on getting all but slanted "facts" if it comes to politics.

      One thing to consider with this research …. ;-) does anyone remember the movie (was it Alien?) and the computer "Al" ? Not sure I like a computer being smarter than I am after seeing that movie, and I am brilliant.

      1. Martin Budden Silver badge

        Re: Numbers printed without challenge

        I am even brillianter ;-)

  2. P. Lee

    I wonder...

    How many times did they try it out in the online poker casinos?

  3. Wanting more

    I'm dubious with the fact it was trained against itself. Yes it might play the perfect logical way, but playing against humans is a different kettle of fish altogether and is what makes poker interesting. It might be able to play billions of hands and come out ahead, but what happens when it just plays 30 hands against a human?

    1. streaky

      Strictly speaking the way a human plays is an irrelevance. Solving a game means you have a fair idea that you're going to win totally regardless of what another human does.

      Now, when you do play poker - and this is the great thing about poker, and why it isn't just pure luck and I imagine what you're alluding to - a human opponent can maximise their wins and minimise their losses and if they're any good still monetarily beat you even if they're statistically (hands won at showdown whatever) losing.

      Without having actually looked at the way their system works and I assume it doesn't do this but you can look at the system's degree of confidence about a hand and decide to bully people off the pot or call large all-ins. It would be absolutely fascinating to see it play against both pro and not pro players. There is a piece of software they could plug it into that will actually play hands for them at various poker sites based on their strategy. The trouble with a "please don't bully me off the pot strategy" is if somebody suspects that you'll generally fold large bets thrown at you unless you have say AA/AK preflop or some such is they'll just keep throwing huge bets at you which in heads-up is the end of the game essentially.

    2. Eric Olson

      The story on The Verge talked about how the program will win over the long run. It will always break-even or be ahead. Additionally, the computer is supposed to be playing a style that has no bearing on the opponent, as by playing the odds and using its experience, it will always come out ahead over the long-term.

      So to answer your question, it might not win over 30 hands. But it has a statistically significant probability to come out ahead in those few hands. Keep in mind that the version of Texas Hold 'Em being played here is the most basic variant, where there is no such thing as all in, and with only a single opponent, there is no chance that there will be collusion (something banned by the rules but can happen).

      The Verge article also noted that an earlier program had played this variant against some of the top poker players in the world, and over six games, it had three wins, two losses, and a draw. So even it's granddaddy was able to hold its own against people who play the game for a living.

    3. Anonymous Coward
      Anonymous Coward

      The developers also want to know how it does against humans. That's why they've invited humans to try it outI suppose.

  4. Anonymous Coward
    Anonymous Coward

    What am I missing?

    I can't see why the system had to "learn". That's a perfectly valid way of working out the best strategy but the probabilities can surely all be calculated relatively easily at any moment in time?

    Take the example of Blackjack for instance. The rules for what the dealer can/must do are fixed, you know what you've got, you know one of the dealer's cards and the decision at any moment in the round as to whether you should stick or twist can be determined - you can look up the odds online of your chances of winning should you stick versus the chances of winning if you twist for any combination of score in your hand at present and card that the dealer is showing.

    Now rather than do the (relatively simple) maths of working out whether the next card in the deck is likely to improve your score or make you bust, you could just program a computer to sit there for a period of time "learning" - but all you'd be doing is arriving at the same endpoint as if you worked out the odds - just as if you programmed a computer to roll a six-sided die enough times, you'd "learn" that the chances of throwing a four were one-in-six.

    Why is this version of poker any different? The only variable is that if you're playing against a human opponent, there's a randomness as to what they will decide to do at any moment in time... but that can't be determined and at best all you can say is that of the last million times I was in this position, when the opponent raised they were bluffing x% of the time.

    So this sounds to me more like they've got good PR rather than having "solved" anything new!

    1. James Micallef Silver badge

      Re: What am I missing?

      In the case of blackjack, you play against dealer and dealer has to do what you instruct (stick/twist). In poker the opponent has independent agency, so I guess training against itself was required to calibrate what types of bets were consistent with what types of hands.

      However as others mentioned, it needs to be trained vs humans to understand that the opponent might not always make 'rational' decisions based solely on the cards available. Otherwise it will never learn to call a bluff

    2. auburnman

      Re: What am I missing?

      Because creating a system that can "learn" is the objective of the exercise. Sure it would be much easier to strap a decision tree to a probability calculator and call it a day, but that's not what they're after. The poker game is nothing more than a simplified field of play for them to prove their concept on. A framework for a true "learning" program could be ported to boundless applications and possibly earn someone some serious $£$£.

  5. Aerobatic

    Yawn

    Limit Poker is a very mathematical game... its quite dull to play. The limits on betting make it harder to bluff as the raises aren't scary (they're limited). Creating an AI that plays a two player limit holdem game is a very specific problem to solve, and I've no doubt its solvable as its a very small and highly mathematical subset of the poker games that are played.

    The popular version of poker thats played by most people, both online and in real life is No-Limit or Pot-Limit Texas hold-em - usually played with 6 or more players at a table (and often 9 or 10), which is a much gutsier, more psychological and less mathematical game compared to Limit Holdem Poker. To play NL Holdem well you need to use a combination of psychology, math and acting skills and learn how the players behave and try and anticipate their intentions. Im sure its still possible to create an AI to play it, but its a much more difficult problem and i look forward to this team solving that one too, one day.

    1. Killing Time

      Re: Yawn

      A fair assessment but you missed out the importance of ‘position’ (relative to the dealer button). This is all important pre-flop and on the subsequent ‘streets’. The information you get purely from your position within the round far outweighs any acting skills you think may help.

      Besides, if you can't keep a straight face then you can always emulate the 'Unibomber' (Phil Laak).

    2. badger31

      Re: Yawn

      Agreed. I have worked on poker AI quite a bit over the past seven years, so while I applaud the work that these people have done, I won't be impressed until it can play no-limit against a full table (>6 opponents). I imagine that what they have actually achieved isn't much more than the pure statistical models we already have, especially if it has only trained against itself. Every good (human) poker player knows that what cards you hold is irrelevant unless there is a show down. What is far more important is what your opponents think you have, and what you think your opponents have*. The latter is achieved by player modelling, the former by how you represent your hand (bluffing). I'd be astonished if this new AI could learn all this by playing itself, leave alone all the psychology involved in the game.

      And all this talk of eleventy trillion billion training games is also bunk. Unless it comes across a new strategy, all that will be happening is that the learned model will focus in on the local optimal with greater accuracy. I'll wager that the learning curve flat-lined pretty quickly.

      * I remember once in a friendly game of NL-TH, one of my friends, who was under the gun, came right in with a big pre-flop raise. This was a bold move, considering we had only had one card dealt.

      1. Killing Time

        Re: Yawn

        @ badger31

        This work is on a limit game variant so its solution will be achievable due to the constraints the limits impose on the game. When there are fewer constraints and much more nuance (as in No Limit -Texas Holdem) their approach just won't cut it.

        Your mate's move was, quite frankly, a bad one not a bold one. Successful players shove it in early on when they think they are dominating the table and it has to be monster when under the gun due to the risk of a re-raise. Therefore you all had to put him on an ace, if he blagged his way through to the flop and no ace turned, he didn't have a great betting position on the turn, a big bet / reraise after him would have him in big trouble.

        He was lucky it was a friendly game.......

        1. badger31

          Re: Yawn

          @Killing time

          No, my mates moves was a stupid one - he just got a little over excited. He had decided on his approach before the hand was dealt and we all laughed and pointed at him for getting caught out. That part of my comment was just an amusing aside, but it highlighted the fact that a lot of the time, in NL-TH, the actual cards in your hand are irrelevant as nobody gets to see them.

          1. Killing Time

            Re: Yawn

            @badger31

            Yep, understood that, but rather than laugh at him, a successful player would have taken him for every chip he could because that is the objective of the game. Equally, that player would put him and every other player at the table on a range of hole cards whether he gets to see them or not.

            Your point regarding the actual cards in a hand is correct but its significance is directly related to position, particularly less so in late position as you can make it very expensive for early caller/raisers to see more cards. If you are going to bluff from early position you have to be willing to put your whole stack in that hand (through to the river) and each and every time you do it, Not a particularly smart strategy when there are multiple players still at the table or one you would pull more than once in a game.

            That's why a raise from 'Under The Gun' generally gains respect.

    3. Hud Dunlap
      Boffin

      Re: Yawn @Aerobatic

      The software to analyze faces is already here.

      http://www.emotient.com/products

      The above is not quite the site I was looking for but it gives you the idea.

      I couldn't figure out how to word the search but some basketball teams are using software to look at college basketball prospects while they are playing. The claim is the looks on their faces while they are playing is an indicator of how well they will do in the pro's.

  6. This post has been deleted by its author

  7. fpx
    FAIL

    So What?

    I have an unbeatable roulette strategy: I always put my money on black, and if red wins, I double my bet for the next round.

    I know this strategy only works as long as there is no house-wins-all 0 on the roulette wheel, but you get the point.

    The basic problem with my roulette strategy is that you must have enough reserves to keep playing through multiple losses. If you start with $1, you need a reserve of $2047 if you anticipate at most 10 consecutive losses. So you start with $2048 in your pocket but can only win $1 each.

    Same with a perfect poker strategy: you must have enough reserves to be able to keep playing through a number of losses until statistics catch up with you, meaning that you have to go in with a lot of cash for the prospect of making small wins.

    And as has been commented elsewhere, it's the psychology between human players that makes poker interesting.

    1. Bunbury

      Re: So What?

      I recall your strategy from a statistics textbook - think they may have called it the gambler's fallacy? It's not going to give you any better long term return than any other strategy. it all goes well until you have a run of losses. Which you will. Ignoring the zero, you'll get ten straight losses, errr, 1 n 2 the the tenth power times - 1024. You might doule your reserve to deal with that but there's a 50% chance that having had ten straight losses you'll have an 11th.

      The only way to win at roulette is to cheat or to own the casino.

      1. Graham Marsden

        @Bunbury - Re: So What?

        The Gambler's Fallacy is that dice/ roulette wheels/ coin flips whatever have "memory", ie that if you get a run of Heads the next flip is more likely to be Tails.

        Of course (unless you're using loaded dice or double-headed coins etc) this is nonsense since each event is not linked to the previous one, but people still believe it.

      2. Anonymous Coward
        Anonymous Coward

        Re: So What?

        No, the double up will always win every time even if there is a 0 or even a 00 but it relies on two things.

        Unlimited funds and no betting limit.

        The issue is, no person has unlimited funds and few casinos have unlimited limits. Doubling escalates rapidly and when you are betting $524,288 just to win a $1 you are probably going to be sweating a bit.

        Therefore you are playing sometimes high stakes games just to make a $1. If you raise the stakes to a starting bet of $100 to make reasonable returns then you are 8 turns closer to hitting the casino or your own limit.

      3. Frumious Bandersnatch

        Re: So What?

        think they may have called it the gambler's fallacy?

        Doubling down on a loser is called a martingale (strategy). In probability, the word has various specific meanings, but the term used does derive from the earlier meaning in gambling.

        Actually, with infinite resources (and no limit on the bet), doubling down on loser always wins eventually.

    2. Alan Johnson

      Re: So What? - Nonsense

      The problem with the double the bet every time you lose strategy is that it only works if the player has an infinite amount of money. If you had an infinite amount of money then you do not really care. For any finite amount of money the expectation is that you lose money but structured in a way where you have high probabilities of small wins and a low probability of a big loss but in the long run the big losss outweighs the gains and you lose.

    3. This post has been deleted by its author

    4. Anonymous Coward
      Anonymous Coward

      Re: So What?

      "...as long as there is no house-wins-all 0..."

      There is also always at least a 0 and sometimes a 00 on a roulette wheel. That is the Casino profit, without it they wouldn't make any money at all (on average) they would continually break even.

      It also isn't a "house-wins-all", you can still bet on 0 it just doesn't factor in the regular odds.

    5. Cliff

      Re: So What?

      What kills that strategy is table limits - you can have the deepest pot in the world but a table limit of $500 cuts you off after 7 spins/flips, and that's deliberately to kill martingales

  8. Little Mouse

    Anything to declare sir?

    If "bluffing may cause it a problem" then I'm guessing that there's still some work to do before we see any airport security related breakthroughs from this research.

    It's clearly a big step up from gaming tic-tac-toe, and any number of steps above my own meagre capabilities, but does it bring anything new to the game theory table apart from the huge number of permutations it is able to consider?

    1. Eric Olson

      Re: Anything to declare sir?

      You might have to check the paper itself. The article I read on The Verge said that much of this was a technological in nature, such as being able to compress the data created by the learning in such a way as to not take up many petabytes of space, the ability to find algorithms that can efficiently compute what they called the regret points of any given hand so the program can react at least at human speed, and then the geek-cred of solving what is called an imperfect information game, where a player doesn't know what the other player possess, and has no way of knowing for certain what will be coming.

      That's why chess and other strategic but perfect information games were solved a while back on much less impressive hardware.

  9. Bunbury

    There's a problem with research in this area

    If you develop a program that can play a gambling game better than a human to a significant degree a rational* person would be in one of two states:

    A. The game is such that I cannot make enough money at it in the real world - not enough money to be made before it fails the opponents Turing test. Thus I publish this paper.

    B. The game is such that I can make enough money in the real world before the Turing test is failed. Why publish a paper to inch my career along a bit when I can clean up at the tables?

    * assuming the person is money motivated not geekdom motivated.

    1. auburnman

      Re: There's a problem with research in this area

      Casinos tend to take a very dim view of you bringing a supercomputing cluster to the Poker tables. If you genuinely 'solved' a Poker variant you'd be better off teaming up with casinos to either prevent their marks from exploiting your method or putting on a spectacle game where punters line up to try their luck at beating the 'infallible' computer.

  10. phil dude
    Coat

    casinos....

    "....attract two sorts of people....suckers and mathematicians."

    P.

    1. Hud Dunlap
      Happy

      Re: casinos.... @Phil dude

      Richard Feynman had a funny story about his first visit to a casino. He had looked at the math of some game and decided he could play all night on x amount of money. Fifteen minutes later he was out of money trying to figure out what happened.

  11. Anonymous Coward
    Anonymous Coward

    "Always bet on black and double my bet"

    Apart from the fact that most houses have a maximum bet limit, around £1000 or something, which pretty much limits your range. I've seen red come out 27 times in a row before, which would mean you'd lose, 0 or no 0.

  12. Aristotles slow and dimwitted horse

    Who is playing the role of the independent dealer here?

    Who has independently audited that this poker playing computer doesn't know which cards have been dealt the human player?

    The computer being both dealer and player in an online game would scare most players off.

    1. Anonymous Coward
      Anonymous Coward

      Re: Who is playing the role of the independent dealer here?

      Right on, mate!

      This is the single sensible post in this thread. Me too, I will never play against the same machine that deals the cards or rolls the dice.

      To put this simply, when I'm playing Minesewwper in Windows and I'm heading to a record time, nothing can convince me the code doesn't move a mine exactly under the square I'm going to click, let's say 7 times out of 10 just not to raise suspicions.

      So in conclusion, I'll pay attention to their technological achievement only when an independent human will deal an manipulate physical playing cards I can see. If we come to think of it, would anyone accept to play in a casino where the rest of the cards are being kept hidden from you ? Sadly, I'm aware there are suckers playing on tablets in some casions these days.

      1. Cliff

        Re: Who is playing the role of the independent dealer here?

        Casinos actually generally play it straight - any hidden edge could just as easily benefit a punter, no need when maths is on your side. It also gets you shut down and a lot of angry armed people wanting their money back. That's not to say the video machines don't make a big show of near-misses and fanfares for modest wins, but mathematically they are true.

        The maths and tricks behind video poker and ilk machines are fascinating (and truly cynical). Michael Bluejay (vegasclick.com) has some good resources if you're interested.

  13. Henry Wertz 1 Gold badge

    Two comments..

    First, I was reading the text as meaning the 4000-CPU cluster was running 6 billion hands/second, as opposed to each CPU running that. (The numbers still don't work, that'd come out to like 31 million billion in two months...)

    Second, I must say, online poker of any sort is pretty different than in person; online, it's largely a game of mathematics (the mathematics being determining your odds and so on) and strategy ("strategy" being mainly detecting the foibles of other players.) In person, you can get reads from other players, they may detect tells from you, and there's the psychological aspects of being there in person interacting with other human beings in addition to the usual aspects of online play.

    I saw a round once on TV (a few years back when World Poker Tournament had some popularity on TV)... the one guy (who had gotten into the tournament via online play) had a straight flush -- STRAIGHT FLUSH! -- and had a 98% chance of winning. The other guy (who looked somewhat like Tom Petty, and got in via live play at casinos) had dick, like a pair of 6's or something, but was bluffing hard... he raised, the guy with the straight flush looked real nervous and sat there. The guy with the 6's just kept sitting there stairing at him with the crazy eyes... after like a minute, he's like "Are you going to call or what!!" and kept stairing. The guy with the straight flush hadn't played much in person, he couldn't deal with the crazy eyes, they psyched him out and he folded!

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like