AIxploratorium
Decision Trees


 

Other Learning Models

[[This is still be written. Stay tuned]]

  • The model, so far, assumes that the games are "independent and identically distributed" (iid): that is, we are assuming the results of one game are independent of the results of the previous games. That is, think of each game, in a specific context (eg, Home, 7pm, JoeStarts=Yes, ...) as a coin flip, with a weighted coin. Here, we assume that the order of these flips does not matter: eg, the outcome of MallRats/FrogLegs "Home, 7pm, JoeStarts=Yes, ..." game on 10/Dec game does not depend on the outcome of the MallRats/SnowBlowers "Home, 7pm, JoeStarts=Yes, ..." game on 15/Jan game

    This means, in particular, that the ORDER of the examples is irrelevant --- that we would make the same predictions whether the situation was "3 losses followed by 17 wins" or "17 wins followed by 3 losses". This is typically not true; instead most teams progress over time, either getting better or worse. (Of course, this is local: there can be many ups and downs over a single season.)

    [[to be completed]]


  • As a related point: Imagine now that you were the coach of the MallRats, and as such, you wanted your team to win. You would then consider which factors you could adjust, and try to find a setting that would allow the MallRats to win.

    Here, you would probably be affecting things --- eg, investigating what happens if ... Joe plays center on this specific game... does that make a difference? (Note this is a different situation than the one described earlier, as we are no long "passive observers".)

    This means the resulting games would not be iid (even if they would have been iid, without our interference.)

    See KDD article

    [[to be completed]]


  • Our goal, so far, was to accurate predict the outcome of the game, and the error of mistakenly asserting that a won-game was lost, is the same as mistakenly asserting that a lost-game was won. This is not always the case:

    [[to be completed]]

    As a related point: sometimes the values of the attributes are not initially known, but must be determined. While we can, of course, collect ALL of the attributes, this is not always necessary --- sometimes a subset of the attributes are sufficient. (Especially if done sequentially.) Moreover, the cost of obtaining the value of an attribute may not be justified, if it does not significantly help determine the outcome.

    For example, suppose you have a $10 bet on the outcome of the MallRat/Snowblower game. Here, it is not worth paying the Snowblower coach a $1000 bribe to determine whether his 6'10" center will play, even if that was sufficient to correctly predict the outcome of the game. See [Greiner,Grove,Roth].


  • The current algorithm requires that all values be discrete --- that is, come from a fixed-size (finite) set, such as {Yes, No}, or {5pm, 7pm, 9pm}. What if some attributes had continuous values --- eg, if we considered the height of the opponent center (rather than being grouped into either "above 7' feet or "below 7 feet"). Now what?

    Here, most learners will first discretize the attribute value. See article.


  • So far, we are only consider who won the game -- which has only 2 options: Won or Lost. We can use this same basic formalism even if there were 3 options --- say Won, Lost or Tied (which makes more sense in say Hockey), or even if there were a larger, but still pre-determined, number of classes; Eg, Won-by-MoreThan10, Won-by-5to10, Won-by-Under5, Lost-by-Under5, Lost-by-5to10, Lost-by-MoreThan10.

    In other situations, there may be a huge, or even infinite, number of classes: eg, the number of seconds that the MallRats were leading, or perhaps the "point spread". Here, there are techniques called "regression".

    [[to be completed]]

 

Modifying the Algorithm Decision Trees Learning in General