Since it is impossible for me to offer detailed comments on each of your papers I have written a sort of mock assignment in order to hopefully provide some useful information with regards to writing a paper. The “mock assignment” is not a paper in its entirety but rather a mixture of sample writing and comments.

 

Please do not focus on the subject of the paper since I have fabricated it. What is important is to note how the paper is organized and how the sections work to build into each other and compliment each other.  For example, look how the final criticisms of the reviewed article are directly related to and work as a set up to the research proposal.

 

Also, please look at the conclusion since it was the section which most often needed the most improvement in your papers.

 

Please remember that you need to learn how to communicate as much as possible, as clearly as possible, with as few words as possible. That is the challenge of effective writing.

 

For example, you ask your friend if he or she wants to go for a drink. 

 

He or She responds, “I don’t know, I’ve got to go and do some homework. I’ve got an assignment for math and one for computing science. Plus I might want to watch TV. So I’m not really sure.”

 

He or she used 34 words to say ambiguously what could be said clearly and succinctly in 12. “No, I must do some homework and I want to watch TV later.”

 

You need work on getting that 34 word phrase into a 12 word phrase.

 

Finally, please everyone review the comments for paper one on the Cmput 603 webpage. They will be helpful to everyone still since many of you did not pay much heed to them before you did assignment one this term.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

                                                Abstract

 

            The importance of opponent modeling for computer poker programs cannot be understated. Until an effective method of opponent modeling is constructed poker bots will never be able to consistently have success against the world’s best human players. The ability to successfully integrate opponent modeling is essential to the creation of an effective poker bot

            . In their recent article, “Poker bots and Opponent Modeling: Strategies for Integration,” (2006) Simard et.al propose several methods for improving on current opponent modeling designs. The most innovative new method which Simard et. al have constructed is titled “Exploit the weak, learn from the strong.” In brief this method calls for a poker bot to classify an opponent as weak or strong and then alter its strategy accordingly. If an opponent is weak the poker bot will play in a more reckless fashion to maximize its profits. If the opponent is classified as strong then the poker bot will play carefully while attempting to learn from its opponent in order to improve its future play against other opponents.

 Unfortunately, Simard et al. underestimate the ability of human players to recognize and adapt to opponent strategies. As such this paper proposes an improvement on the method of Simard et al. whereby the poker bot will continuously monitor the play of its opponent in order to determine if the initial classification of the opponent was incorrect.

In addition to the critique of Simard et.al an unrelated experiment using datasets and classfiers was conducted. The goal of this experiment was to determine which one of four classification algorithms had the highest rate of successful classifications based on an examination of a finite number of attributes. The experiment determined that in most situations the NaiveBayes classifier is most effective.

(IF YOU WERE DOING AN EXPERIMENT RELATED TO THE ARTICLE BEING REVIEWED THE FINAL PARAGRAPH OF YOUR ABSTRACT WOULD READ SOMETHING LIKE THIS)

An experiment was also conducted to compare the performance of Simard et. al’s “Exploit the Weak, Learn from the Strong” strategy and the improved strategy proposed in this paper.  Two poker bots,  “Simard 1” and “Lauritsen 1,” played several thousand hands against both human and computer opponents of varying skill levels. Additionally, the two poker bots played roughly one hundred hands against one another in order to get a head to head comparison of performance.

 

Note how the author’s names, article title and date are all included here. Also note how the key elements of Simard’s article as well as the key criticisms are briefly mentioned. Furthermore, the main improvement this paper will propose and the experiment which will test the new method are also briefly outlined

                                   

 

 

 

 

 

                        Introduction

 

The recent rise in popularity of poker, especially Texas Hold’em poker, both online and televised has had a massive impact on the field of computing science in several areas. First, there is the potential for game manufacturers to cash in on poker’s rampant popularity and reap a hefty profit if they can design and market a challenging, fun poker program. More importantly, however, it has created several important challenges for computing scientist to tackle, most notably opponent modeling in non-zero sum games.

For the last several decades the chess match was considered the pinnacle of human/machine competition. However, given the victory of IBM’s Deep Blue over reigning world champion and chess Grand Master Gary Kasparov in May of 1997 it seemed that the goal of creating a competitive (at least with Grand Masters) chess program had been solved, thus the value of poker as a competitive game for computing scientists.  Chess is a zero sum game where there is one absolute winner and one absolute loser. Additionally, there is no hidden information in chess since all the pieces are in plain sight on the chess board.  Not so with poker. Poker has hidden information and a player can win or lose by differing margins. These two factors alone make the problem of creating a poker playing program that is competitive a tantalizing and challenging problem for researchers.

Texas Hold’em is the most popular style of poker. It was first brought into mainstream culture by the 1998 John Dahl film “Rounders” and is currently the most televised form of poker. The goal of the game is to win as much money as possible from your opponents, sequentially eliminating them one by one as they lose all of their money. The game is played as follows: Each player will make the best five card hand possible using seven total cards. First, each player is given two cards which no other player at the poker table can see. Then the players take turns wagering (“raising”) and re wagering (“re-raising”) based on the perceived strength of their two card hands.  After this first round of wagering three cards are dealt face up on the table. These are community cards which are to be used by all players at the table. The players then wager based on the perceived strength of their five card hands. After the second round of wagering one more card is dealt face up on the table. The players then wager based on the perceived strength of their five card hands incorporating the new card if it strengthens their hands. The sequence of dealing the community cards is referred to in poker parlance as “the flop,” “the turn,” and “the river,” respectively.  Finally, one last card is dealt face up and the players commence one final round of wagering. After the wagering is completed the player with best hand wins the money.  It should be noted that at any time throughout the wagering a player may “fold” his hand and give up the money he has already wagered in order not to risk losing any additional money.

It is immediately evident the problems such a game causes for computing scientists hoping to design an effective program (henceforth referred to as a “poker bot”) for winning at texas hold’em poker.  Most notably, the poker bot must make all decisions based on several levels of incomplete information. Not only does the poker bot not know what cards its opponent has but it also has no idea what cards will be dealt on the flop, turn, and river. Furthermore, not only must decisions be made based on incomplete information but the decisions must have a dollar value attributed to them. A poker bot must decide not just if it will continue to play but how much money it will invest each betting round.

As stated earlier the goal in poker is not just to win, but to win by as much as possible, thus the poker bot faces another challenge: To maximize the dollar value of its wins and minimize the dollar value of its loses. This challenge is the second major problem faced by computing scientists in their quest to design a successful poker bot. Some leaders in the field of poker bot design have advanced the theory of opponent modeling as a possible solution to this second problem. One such team of designers are Tim Simard et al. who have built on previous attempts at opponent modeling to construct a program called “Exploit the Weak, Learn from the Strong,” which they feel will allow their poker bot (Simard 1) to effectively compete against the most skilled human opponents. “Exploit the Weak, Learn from the Strong,” builds on previous opponent modeling programs by attempting to classify its opponent as either weak or strong and alter its strategy accordingly.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

   Summary of Research Paper

 

            While previous opponent modeling algorithms have attempted to integrate some type of classifier which classifies opponents as either strong or weak none have had greater success than Simard et.al.  In their 2006 publication, “Poker bots and Opponent Modeling: Strategies for Integration,” Simard et al. describe in detail their improvements to the Southerly classification algorithm (previously considered the most effective algorithm for opponent classification in poker bots).and provide data culled from thousands of test hands played by their poker bot, Simard 1.

 

 

1.         -Here you would BRIEFLY describe Simard’s algorithm and show explicitly how it improves on the Southerly Classification algorithm.  What is important here are the key differences. You need not necessarily describe the whole algorithm only the areas where Simard has improved things. If you wanted to you could include both algorithms in their entirety in an appendix.

-You must state explicitly what the differences are and explain why the changes Simard makes are beneficial and relevant.  

-You MUST NOT write a lengthy play by play of the entire article “Poker bots and Opponent Modeling: Strategies for Integration,” by Simard et. al.

 

2.         -Next you would BRIEFLY summarize the experiments or test Simard has used to test his poker bot program. What is his methodology? What are his results? Do they indicate better performance than previous algorithms?

 

3      -NOW YOU MUST CRITIQUE SIMARD’S ARTICLE. PERHAPS YOU FEEL THERE IS A PROBLEM WITH HIS EXPERIMENTAL METHODOLOGY. MAYBE YOU FEEL THAT HE HAS MADE AN ASSUMPTION THAT IS UNTRUE OR OPEN TO ATTACK. THIS IS THE PURPOSE OF THE REVIEW.

 

YOUR CRITIQUE COULD READ SIMILAR TO THE FOLLOWING.

 

            Unfortunately, Simard et al. make a careless assumption which could be problematic for their poker bot. They assume that a human player will not attempt to fool the poker bot into incorrectly classifying said player. For example, a smart human player may intentionally play as a weak player for the first twenty or thirty hands in order to mislead the poker bot and provoke an incorrect classification which could in turn be exploited.

            While Simard’s tests indeed demonstrate that Simard 1 performs better than poker bots using the Southerly algorithm, Simard does not clarify whether or not any of the opponents Simard 1 faced attempted to fool the poker bot into incorrect classification. Furthermore, the results of Simard 1’s matches against human players indicate only a marginal level of success and further investigation is required to test whether or not the deception played an important role in the successful human strategies.

 

Usually, your critique will end up being related to your research proposal. Perhaps you may find a way to improve on an aspect of Simard et. al. or you may design an experiment to provide further evidence that would support one of Simard’s conclusions. For example, maybe you believe Simard should have included certain other conditions in his experiment which would have improved the experiment or expanded its scope.  In our example, you may want to test Simard 1 against human players and have them attempt to fool the poker bot into incorrect classification.

 

 

                                               

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

RELATED WORKS

 

In this section you will list works which are DIRECTLY related to the paper you are summarizing and reviewing. Just because one paper is in the same field or deals with the same subject as another does not necessarily mean that they are related. Thus, it is essential that you make direct connections between the reviewed work and the papers discussed here.

 

You might begin as follows:

 

            Simard et al.’s research would not have been possible without the contribution of Southerly et al. who devised the Southerly classification algorithm in, “Effective Opponent Modeling Classification” (2004).  Southerly et al’s algorithm was a landmark for opponent modeling because it was the first algorithm which allowed a poker bot to effectively gauge the strength of an opponent.

 

             Here you would describe Southerly’s algorithm. The real trick will be to not make this explanation seem redundant in the face of the previous discussion in the “Summary of Research Paper” section. This explanation should be as brief yet informative as possible.

Next you have to indicate exactly why Southerly’s algorithm was effective. Why did it allow for effective opponent modeling? What was different about it? Did Southerly use a new approach not previously considered by computing scientists? You need to address these types questions and at the same time be careful not to eat up your word count or get caught discussing tangential issues.

Finally, you need to re-iterate the connection to Simard. Again this will be tricky since you have already made the connection in the previous section. However, it is not acceptable to simply reference the previous section. Do not write something like, “The connection between Simard et al. and Southerly was made in the previous section.”  

After you have finished discussing Southerly you may want to look at other papers which have contributed to opponent modeling. Are there any papers that have attempted opponent modeling using different methods or approaches? Were these approaches effective? Why or Why not? REMEMBER THAT YOU MUST MAKE DIRECT, RELEVANT CONNECTIONS TO SIMARD’S ARTICLE.

 IF YOU CANNOT MAKE THE CONNECTION BETWEEN A WORK, FOR EXAMPLE, “LAURITSENT ET AL,” AND SIMARD THEN THE TWO WORKS ARE NOT RELATED AND LAURITSENT ET. AL SHOULD NOT BE DISCUSSED.

 

 

 

 

 

 

 

 

                                    Research Proposal

 

As I have outlined in my review of Simard et. al, Simard’s experiment was somewhat incomplete as it did not consider and test the possibility that some human opponents may attempt to fool the poker bot into incorrect classification.  Thus, in order to determine if Simard 1 is vulnerable to a strategy of classification deception I propose the following experiment.

The experiment will be divided into four stages. In the first stage 50 human players who would be considered inexperienced or poor players will play 300 hands against Simard 1. The results should give a strong indicator of the level of success Simard 1 can expect against low skill level players.  The second stage will see 50 human players who would be considered expert players play 300 hands against Simard 1 in order to give an estimation of how Simard 1 can be expected to fare against expert skill level players.

In the third stage of the experiment the 50 players will play 300 hands against Simard 1. However, the first 50 to 60 hands will be played by “poor” poker players. After these initial hands the experts will step in and play the remaining hands. This method should provide an illustration of what would happen if Simard 1 was fooled into incorrect classification. Most likely, Simard 1 will not even last 300 hands given that it will be taking risks that would pay off against an unskilled opponent but would be poker suicide against an expert.

The final stage of the experiment will see all three hundred hands played by expert players. However, the experts will attempt to play as poorly as possible for the first 50 or 60 hands in order to see if they can fool the computer into an incorrect classification. Unlike the third stage where we can be certain that the classification will be poor since they initial hands are actually being played by poor players, the final stage will attempt to determine if an expert can alter his or her play enough to achieve an incorrect classification, thus setting up the conditions necessary for exploitation.

We believe that this experiment will show that Simard 1 is vulnerable to classification deception. We will be able to gauge to what extent Simard 1 can be exploited by comparing the results from stage 4 to the results from the three previous control stages. Most likely, the results from stage 4 should mimic the results from stage 3 since it is quite likely that expert players possess enough skill and guile to mislead Simard 1 into an incorrect classification.

 

Note how the methodology of the experiment was explained. It is important to discuss not only what you wish to test but to illustrate how you intend to do it. Furthermore, you need to have a hypothesis. What do you think the results of the experiment will be? Clearly, you are designing your experiment (in an unbiased manner) to prove or disprove something and this should be discussed.

 

 

 

 

 

 

 

 

                                    Experimental Results

 

 

The following experiment is unrelated to the review of Simard et al. and was performed in order to determine the effectiveness of different classification algorithms. Four classification algorithms were selected and were run on four different datasets.

The four classification algorithms chosen for this experiment were NaiveBayes, DecisionTree, Bayesian, and DecisionStump. Here you would very briefly explain each of the algorithms and high light their differences.

Each of the chosen four datasets contains 2 or more categories and each category can be identified by a set of attributes. For example, one data set may be named “Cancer” and contain 2 categories, “Sick” and “Well.” The category “Sick” can be characterized by the following attributes: “tired,” “weak,” and “pale.” The category “Well” may be characterized by the following attributes: “energetic,” “strong,” and “flush.” Thus, if a person has the characteristics “tired” and “weak” they could fall into the “sick” category.

Each classifier will first be trained on each dataset. Here you would briefly explain what “training” the algorithm encompasses. I don’t really know because I’m not a computing scientist.

 After the algorithms have been trained they are run on each of the different datasets and the percentage of correct classifications is calculated. A classification will be considered accurate if a unit which contains the characteristics attributed to a selected category is placed in that category. For example, if a person has the characteristics “tired” and “weak” and the classifier listed him as “sick” the classification would be considered accurate. However, if the classifier listed the same person as “well” the classification would be considered inaccurate.

Here you would summarize your results. You would discuss the results of running each classifier on each dataset then compare the results and discuss them. For example, under which circumstances were which algorithms effective? Was one clearly superior? If so why do you think it was better? Etc.  This is a good place to use visual data representations to aid in your explanations. Remember if you have a chart or graph you need to explain it. You cannot simply say, “the results are summarized in figure 2.”

Note how everything is explained. It is clear which algorithms were chosen and what exactly a “dataset” is. Also, I’ve made clear what entails an accurate classification. Many of you did not bother to explain these details. It seems trivial for such a mundane experiment but many experiments are unique and thus careful explanation will be required.

 

 

 

 

 

 

                                        Conclusion

 

Opponent modeling is a key component of any effective computer poker program that hopes to effectively compete at a high level against human opponents.   Given the fact that poker is a game of incomplete information and that the goal of poker is not only winning but maximizing profit poker poses a unique problem for computing scientists hoping to achieve effective opponent modeling. Simard et al. have attempted to find a solution for the problem of opponent modeling using the Southerly algorithm as a template. With Southerly as a starting point  Simard et. al have designed a more advanced opponent modeling algorithm called “Exploit the Weak, Learn from the Strong.”

“Exploit the Weak, Learn from the Strong,” is designed to classify an opponent as either weak or strong and react accordingly. Simard et al. successfully accomplish this by altering the Southerly’s algorithm. HERE YOU WOULD BRIEFLY HIGH LIGHT THE ALTERATIONS SIMARD ET. AL HAVE MADE. FOCUS ON ONLY THE KEY ASPECTS THAT ARE OF THE MOST IMPORTANCE.

Unfortunately, as I have indicated here, Simard et. al have failed to consider the possibility that their poker bot may be vulnerable to a strategy of classification deception which will allow it to be exploited by savvy human opponents. However, this paper (meaning the paper that I am writing) has proposed an experiment whereby the ability of Simard 1 to resist exploitation by classification deception will be tested. As mentioned in the “Research Proposal” section a selection of expert human players will attempt to fool Simard’s poker bot using a strategy of initial weak play in an attempt to achieve false classification. After their initial weak play the experts will begin to play as strongly as possible to see if they can exploit their computer opponent.

In addition to the review of Simard et. al. an unrelated experiment to determine the effectiveness of  classification algorithms was conducted.  Four classification algorithms were trained on four datasets and then run on the datasets. It was determined that under most conditions and in most circumstances the NaiveBayes classifier  performed most effectively having an average classification accuracy of  79.5% across all datasets and under all conditions of the experiment. Here you would offer possible explanations for why this may be the case or perhaps point out how your results indicate possible uses for the NaiveBayes algorithm.

 

Normally the section regarding the experiment would be related to your research proposal and so it would be possible to segue into a nice end to your paper. As it stands here the ending is a bit of a lame duck but it is tough to work back into a discussion of Simard and the state of opponent modeling in computer poker since the experiment is unrelated.