Below are some possible topics for Comput651 projects. For each, we provide some suggested readings; these are the references to start with. Do not consider these references exhaustive; you will be expected to review the literature in greater depth for your project. While you are not forced to choose one of these topics, it is strongly advised that you talk to the instructor if you want to deviate significantly from the topics below.  (I marked with ** the ones that I am particularly interested in pursuing.)

1. COMING SOON:
   Abstractions
   NetFlixPrize
2. 1. superresolution for MRI data
   2. Custom filters for MRI data
   Contact dana at cs.ualberta.ca
   Combining classifiers, for Proteome Analyst
3. Structure Learning
   The challenge here is finding a good graph structure over a set of variables, in either a directed or undirected graphical model. Potential projects include...
   • Comparing structure learning algorithms for Bayesian networks (eg, hillclimbing, PDAGs, optimal reinsertion) in terms of quality of density estimation, sensitivity of the size of the data set, classification performance, etc.
   • Structure search given a fixed ordering -- If you are given a total ordering of the variables x₁...x_n where the parents of x_i are a subset of x₁...x_i-1, structure learning becomes simpler than search over the space of directed acyclic graphs (K&F 15.4.2)
   • Learning the structure of an undirected graphical model (Abbeel et. al. 2006, Parise and Welling 2006)
   • Learning compact  representations for conditional probability distributions -- In discrete Bayesian networks having a large number of parents means a node's CPD is large. It is possible that given a particular assignment to a few of the parents, the rest of the parents do not matter (context-specific independence), which can lead to a compact representation of a CPD (K&F 15.6).
   • Bayesian model averaging -- insteading of finding the single best structure for a Bayesian network, compute a posterior distribution over structures (K&F 15.5)
   • Optimal structure learning -- the naive algorithms are super-exponential in the number of variables, but both the optimal MAP (Singh & Moore 2005) and optimal BMA (Koivisto & Sood 2004) structures can be computed in exponential time at the cost of exponential memory.
   • Find the structure the produces the best discriminative model; see (Guo&Greiner 2005).
     
   References
   Koller & Friedman Chapter 15
   
   Pieter Abbeel, Daphne Koller and Andrew Y. Ng.
   Learning Factor Graphs in Polynomial Time & Sample Complexity.
   Journal of Machine Learning Research, 7(Aug):1743--1788, 2006.
   
   Inferring Graphical Model Structure using L1-Regularized Pseudo-Likelihood. Martin Wainwright, Pradeep Ravikumar, John Lafferty. NIPS 2006
   
   D. Margaritis. Distribution-Free Learning of Bayesian Network Structure in Continuous Domains. Proceedings of The Twentieth National Conference on Artificial Intelligence (AAAI), Pittsburgh, PA, July 2005.
   
   Yuhong Guo and Russ Greiner (2005), Discriminative Model Selection for Belief Net Structures".  In Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI-05).
   
   R. Greiner, T. Van Allen., "A Model Selection Criteria for Learning Belief Nets: An  Empirical Comparison".
   International Conference on Machine Learning, July 2000

   Ajit Singh and Andrew Moore (2005), Finding Optimal Bayesian Networks by Dynamic Programming. Tech Report CMU-CALD-05-106
   
   Mikko Koivisto and Kismat Sood (2004), Exact Bayesian Structure Discovery in Bayesian Networks. JMLR 5.
   http://jmlr.csail.mit.edu/papers/volume5/koivisto04a/koivisto04a.pdf
   
   Sridevi Parise and Max Welling (2006) Structure Learning in Markov Random Fields, NIPS 2006
   http://www.ics.uci.edu/~welling/publications/papers/StructLearnMRF-submit.pdf

4. Inference
   The most common use of a probabilistic graphical model is computing queries, the conditional distribution of a set of variables given an assignment to a set of evidence variables. As this problem is NP-hard, there are now a large number of algorithms, both exact and approximate. Potential project topics include,,,
   • Comparing approximate inference algorithms in terms of accuracy, computational complexity, sensitivity to parameters. Some exact algorithms include Junction trees and Bucket elimination. On larger networks one typically resorts to algorithms that produce approximate solutions, such as sampling (Monte Carlo methods), variational inference, and generalized belief propagation.
   • Adaptive Generalized Belief Propagation (Welling 2004) & Expectation Propagation (K&F 11) -- Compare these methods to each other and Gibbs sampling.
   • Convex Procedures -- Methods that performance approximate inference by convex relaxation (Wainwright 2002)
   • Linear programming methods for approximating the MAP assignment (Wainwright et. al. 2005b, Yanover et. al. 2006)
   • Recursive conditioning -- An any-space inference algorithm that recursively decomposes an inference on a general Bayesian network into inferences on a smaller subnetwork. (Darwiche 2001).
   References
   
   Koller & Friedman Chapters 7-11
   
   Adnan Darwiche
   Recursive Conditioning
   In Artificial Intelligence Journal. Vol 125, No 1-2, pages 5-41. 2001.
   http://reasoning.cs.ucla.edu/fetch.php?id=18&type=ps
   
   T. Jaakkola.
   Tutorial on variational approximation methods.
   In Advanced mean field methods: theory and practice. MIT Press, 2000.
   http://people.csail.mit.edu/tommi/papers/Jaa-var-tutorial.ps
   
   An Introduction to Variational Methods for Graphical Models M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. In M. I. Jordan (Ed.), Learning in Graphical Models, Cambridge: MIT Press, 1999.
   http://www.cs.berkeley.edu/~jordan/papers/variational-intro.pdf
   
   Yedidia, J.S.; Freeman, W.T.; Weiss, Y., "Generalized Belief Propagation", Advances in Neural Information Processing Systems (NIPS), Vol 13, pps 689-695, December 2000
   http://www.merl.com/reports/docs/TR2000-26.pdf
   
   Yedidia, J.S.; Freeman, W.T.; Weiss, Y., "Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms", IEEE Transactions on Information Theory, ISSN; 0018-9448, Vol. 51, Issue 7, pp. 2282-2312, July 2005
   http://www.merl.com/reports/docs/TR2004-040.pdf
   
   M. J. Wainwright, T. Jaakkola and A. S. Willsky. A new class of upper bounds on the log partition function. IEEE Trans. on Information Theory, vol. 51, page 2313--2335, July 2005
   http://www.eecs.berkeley.edu/~wainwrig/Papers/WaiJaaWil05_Upper.pdf
   
   M. J. Wainwright, "Stochastic Processes on Graphs: Geometric and Variational Approaches", Ph.D. Thesis, Department of EECS, Massachusetts Institute of Technology, 2002.
   http://www.eecs.berkeley.edu/~wainwrig/Papers/Final2_Phd_May30.pdf
   
   M. J. Wainwright, T. S. Jaakkola and A. S. Willsky, 
   MAP estimation via agreement on (hyper)trees: Message-passing and linear-programming
   approaches.  IEEE Transactions on Information Theory, Vol. 51(11), pages 3697--3717. November 2005.  
   http://people.csail.mit.edu/tommi/papers/WaiJaaWil_TRMAP_arxiv.pdf
   
   Linear Programming Relaxations and Belief Propagation - an Empirical Study
   Chen Yanover, Talya Meltzer, Yair Weiss
   JMLR Special Issue on Machine Learning and Large Scale Optimization, Sep 2006
   http://www.jmlr.org/papers/volume7/yanover06a/yanover06a.pdf
   
   Max Welling, On the Choice of Regions for Generalized Belief Propagation, UAI 2004

   Max Welling, Tom Minka and Yee Whye Teh (2005) Structured Region Graphs: Morphing EP into GBP, UAI 2005
   

5. Variance of Response (**)
   Most inference system return a single value; eg, P(+cancer | +headache, -sorethroat) = 0.02. In many cases, however, this response itself is actually a distribution. That is, the response depends on the parameters (CPtables), which are often based on a (random) datasample. Here, these parameters are necessarily random variables, which means the result of the computation, here the response, is also random. The technical note in BN Variance webpage proves that this distribution is asymptotically normal, and provides the associated mean and variance.

   That analysis assumes as given the correct structure over discrete variables, with CPtable values represented as a table. This suggests many extensions:

   • dealing with a distribution over structures (rather than a single correct one)
   • dealing with non-tabular CPtables -- eg, noisy-ors, or decision trees
   • dealing with continuous variables
     
     
   This analysis also indirectly assumes the datasample was complete; it would also be useful to extend these ideas to deal with incomplete data. This would allow us to deal with important cases, like HMM structures.

   There are several applications of this technology:

   • The Mixture using Variance paper describes a way to use these variance values to combine difference bayesian-net-based classifiers, learned from different subsets of instances. Can this analysis be applied to combine different subsets of features?
   • In general, can we use this analysis for selecting features, of a single classifier... eg select only the features that contribute least to the total variance.
   • Can we use this analysis to determine the change of "Good decision, bad outcome"? And perhaps for inbalanced data? See KDD Cup 2006.
   • Can this help in active learning ... related to Active Learning with Statistical Models
6. Temporal Models
   There are lots of applications where we want to explicitly model time (control, forecasting, online-learning). Hidden Markov Models are one of the simplest discrete-time models, but there are many others: Kalman filters for continuous state-spaces, factorial Hidden Markov models for problems with many hidden variables that allows for efficient variational inference, and dynamic Bayesian networks which allow arbitrarily complex relationships between hidden and observed variables. Projects include
   • Comparing the performance of factorial Hidden Markov Models (Ghahramani & Jordan 1997) to dynamic Bayesian networks (K&F 18).
   • Expermental evaluation of approximate inference algorithms for DBNs, such as Boyen-Koller, Particle Filtering, and Thin Junction Trees (Paskin 2003). Kevin Murphy's thesis provides a good overview of inference in DBNs.
   • Comparing Kalman filters against more general DBN models.
   References
   
   K&F Chapter 18
   
   Ghahramani, Z. and Jordan, M.I. (1997).  Factorial Hidden Markov Models.  Machine Learning 29: 245-273
   http://www.gatsby.ucl.ac.uk/~zoubin/papers/fhmmML.ps.gz
   
   Kevin Murphy's PhD Thesis.
   
   Kevin Murphy's book chapter on DBNs:
   http://www.cs.ubc.ca/~murphyk/Papers/dbnchapter.pdf
   
   Xavier Boyen and Daphne Koller, Tractable Inference for Complex Stochastic Processes, in UAI'98.
   
   Xavier Boyen and Daphne Koller, Exploiting the Architecture of Dynamic Systems, in National Conference on Artificial Intelligence AAAI '99, 1999.
   http://ai.stanford.edu/~xb/aaai99/index.html
   
   Mark A. Paskin (2003). Thin Junction Tree Filters for Simultaneous Localization and Mapping. In G. Gottlob and T. Walsh eds., Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence ( IJCAI-03), pp. 1157–1164. San Francisco, CA: Morgan Kaufmann.
   http://ai.stanford.edu/~paskin/pubs/Paskin2003a.pdf
   
   
7. Hierarchical Bayes
   In text classification, one can view a corpus as generated by a hierarchical process. For example, select an author. Given an author there is a distribution over topics he is interested in. Select a topic according to this distribution. Given a topicm there is a distribution over words used in a document. Finally, generate a bag of words from this distribution. One approach to modelling this process is hierarchical Bayes (often nonparametric hierarchical Bayes). 

   Another application of nonparametric hierarchical Bayes is clustering, where instead of selecting the number of clusters a priori, the model averages over the number of clusters to produce a posterior over clusterings of data points. 

   Potential projects include

   • Implementing Latent Dirichlet Allocation for document clustering
   • Compare nonparametric Bayesian clustering methods such as Dirichlet Processes and Chinese Restaurant Processes.
   • Implement a hierarchical clustering model, either for topic modelling or clustering.
     
     
   References
   D. Blei, A. Ng, and M. Jordan. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993–1022, January 2003.
   http://www.cs.princeton.edu/~blei/papers/BleiNgJordan2003.pdf
   
   Dirichlet process, Chinese restaurant processes and all that. M. I. Jordan. Tutorial presentation at the NIPS Conference, 2005.
   http://www.cs.berkeley.edu/~jordan/nips-tutorial05.ps
   
   D. Blei, T. Griffiths, M. Jordan, and J. Tenenbaum. Hierarchical topic models and the nested Chinese restaurant process. In Neural Information Processing Systems (NIPS) 16, 2003.
   http://www.cs.princeton.edu/~blei/papers/BleiGriffithsJordanTenenbaum2003.pdf
   
   D. Blei and M. Jordan. Variational inference for Dirichlet process mixtures. Journal of Bayesian Analysis, 1(1):121–144, 2005.
   
   Y. Teh, M. Jordan, M. Beal, and D. Blei. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 2005.
   
   Ian Porteous, Alex Ihler, Padhriac Smyth and Max Welling (2006) Gibbs Sampling for (Coupled) Infinite Mixture Models in the Stick-Breaking Representation UAI 2006
   http://www.ics.uci.edu/~welling/publications/papers/ddp_uai06_v8.pdf
   
   Mark Steyvers and Tom Griffiths
   Matlab Topic Modelling Toolbox.
   http://psiexp.ss.uci.edu/research/programs_data/toolbox.htm
   
   
8. Build models for metabolomic pathways, etc.,
   ... perhaps related to HMP, Kyoto Encyclopedia of Genes and Genomes (KEGG), Proteome Analyst

9. Modeling Protein-Protein Interactions
   Implement, then extend: Probabilistic Modeling of Systematic Errors in Two-Hybrid Experiments See also auxiliary data sets.

10. Relational Models
    Almost all of the machine learning / statistics methods you have studied assume that the data is independent or exchangable. In many cases this is not true. For example, knowing the topic of a web page tells you something about the likely topics of pages linked to it. The independence assumption fails on most graph-structured data sets (relational databases, social networks, web pages).

    Potential projects include

    • Implementing a restricted case of Probabilistic Relational Models (eg, no existence uncertainty) and compare the performance against some baseline non-relational models.
    • Implementing Relational Markov Networks and compare the performance against some baseline non-relational models
    • Applying these ideas to address the Netflix challenge
    References
    Learning Probabilistic Relational Models,  L. Getoor, N. Friedman, D. Koller, A. Pfeffer. Invited contribution to the book Relational Data Mining, S. Dzeroski and N. Lavrac, Eds., Springer-Verlag, 2001
    
    Discriminative Probabilistic Models for Relational Data,  B. Taskar, P. Abbeel and D. Koller. Eighteenth Conference on Uncertainty in Artificial Intelligence (UAI02), Edmonton, Canada, August 2002.
    
    Hierarchical Probabilistic Relational Models for Collaborative Filtering Jack Newton and Russell Greiner, SRL2004: Statistical Relational Learning and its Connections to Other Fields (within ICML 2004), July 2004.
    
    L. Liao, D. Fox, and H. Kautz. Location-Based Activity Recognition. in Proceedings of the Neural Information Processing Systems (NIPS), 2005.
    
    
11. Hybrid Bayesian Networks
    Many real systems contain a combination of discrete and continuous variables, which can be modeled as a hybrid BN. Potential projects include
    • Compare inference algorithms for hybrid DBNs against those that first discretize all the continuous variables, and then just use the standard algorithms (variable elimination, junction trees).
    References
    K&F Chapter 12
    
    Hybrid Bayesian Networks for Reasoning about Complex Systems, Uri N. Lerner. Ph.D. Thesis, Stanford University, October 2002.
    http://ai.stanford.edu/~uri/Papers/thesis.ps.gz
    
    
12. Influence Diagrams
    A Bayesian network models a part of the world, but not decisions taken by agents nor the effect that these decisions can have upon the world. Influence diagrams extend Bayesian networks with nodes that represent actions an agent can take, the costs and utilities of actions, and most importantly the relationships between them.

    In multiagent setting finding the Nash equilibrium is hard, but graphical models provide a framework for recursively decomposing the problem (opening up the possibility of a dynamic programming approach). Dynamic programming algorithms like NashProp (Kearns and Ortiz, 2002) are closely related to belief propagation.

    Projects include

    • Implementing algorithms for selecting a good or optimal strategy in the single-agent case (K&F 21)
    • Finding Nash equilibria in multiplayer games (Koller & Milch, 2003)
    References
    K&F Chapter 21
    
    D. Koller and B. Milch (2003). "Multi-Agent Influence Diagrams for Representing and Solving Games." Games and Economic Behavior, 45(1), 181-221. Full version of paper in IJCAI '03.
    

    M. Kearns and L. Ortiz; Nash Propagation for Loopy Graphical Games. Proceedings of NIPS 2002.

    Carlos Guestrin, Daphne Koller and Ronald Parr;
    Multiagent Planning with Factored MDPs;
    NIPS 2001, pp. 1523 - 1530, Vancouver, Canada, December 2001.

    Carlos Guestrin,
    Planning Under Uncertainty in Complex Structured Environments, Ph.D. Dissertation, Computer Science Department, Stanford University, August 2003.
    http://www.cs.cmu.edu/~guestrin/Publications/Thesis/thesis.pdf
    
    

13. Max-margin Graphical Models
    Typically the parameters of a graphical model are learned by maximum likelihood or maximum a posterori. An alternative criteria for parameter estimation is to maximize the margin between classes, which can be thought of as a combination of graphical models (to represent structured relationships between inputs and outputs) with kernel methods. Projects include,

    An example of a domain where this approach works well is handwriting recognition, where the structure encodes the fact that knowing what the previous letter was tells you something about what the next letter is likely to be.

    • Compare max-margin to likelihood based methods (eg, character recognition, part of speech tagging)
    References
    Max-Margin Markov Networks,  B. Taskar, C. Guestrin and D. Koller. Neural Information Processing Systems Conference (NIPS03), Vancouver, Canada, December 2003.
    MMMN
    Taskar's thesis:
    Thesis
    
14. Active Classifier / Value of Information (**)
    Most classifiers are "passive", in that they can only use the feature values given, even if many of those values are missing. By contrast, an "active classifier" has the option of first obtaining the values of some attributes, before returning a label. Given a cost model that specifies both the costs of purchasing each attribute value, and also the penalty of returning each type of incorrect label, we can seek the optimal "policy", that identifies when to purchase which attribute value and when to return which label; see webpage.

    Possible projects include...

    • Apply active classification to dynamically determine which sensor (within a network) you should sample from.
    • Extend the model to deal with cost models that give discounts when you purchase many attribute values at once; or alternatively, if you purchase the values of some attribute for all instances, etc.
    
    References
    A. Krause, C. Guestrin. "Near-optimal Nonmyopic Value of Information in Graphical Models" .  UAI, 2005

    A. Krause, C. Guestrin. " Optimal Value of Information in Graphical Models - Efficient Algorithms and Theoretical Limits", IJCAI 2005

    Anderson, B. and Moore, A.; Fast Information Value for Graphical Models; NIPS 2005

15. Active/Budgeted Learning (**)
    Most learning tasks implicitly assume the learner begins with a body of labeled data -- typically a matrix M = (m_ij)  where m_ij is the value of the i-th attribute of the j-th instance. Now consider the challenge of dynamically gathering a subset of such data  --- ie, sequentially determining which attributes, of which training instances, to "purchase", where the goal is obtaining the subset of data that will allow the learner to produce the best performance system. In particular, the standard active learner sequentially purchases a sequence of k class-labels, for some currently unlabeled (but otherwise completely specified) instances, with the goal of producing the best possible "based on any k labels".  The budgeted learning model similarly seeks the best classifier, but here each instance is given only the class-label but NOT the attribute values.  Here, the budgeted learner can sequentially purchase attribute values, subject to spending at most a given fixed total budget.
    

    Possible projects include
    

    • Compare optimization criteria (eg, experimental design criteria) [CITE]
    • PAC-learning analysis: how many "attribute-instance" probes are necessary to produce a classifier that is (with probability at least 1-delta) within epsilon of optimal
      
    • Extend this analysis to the task of learning a generative model -- ie, where the goal is learning a probabilistic model that is close to the true model (in terms of KL-divergence)
      
    • Active learning that incorporate parameter uncertainty.
References
Simon Tong, Active Learning: Theory and Applications, PhD thesis, Stanford University 2001.

D Lizotte, O Madani and R Greiner; Budgeted Learning of Probabilistic Classifiers, UAI'04.

?PAC-learning - Valiant



16. >Segmentation A segmentation of an image maps each voxel to a label -- eg, to "healthy" vs "tumor", within an Magnetic Resonance image of a brain. To be effective, a segmentator should base its decision about the label of pixel x on information about the neighboring pixels (inclding perhaps their respective labels) as well as information on that location. There are a variety of technologies for this task, including generative models like Markov Random Fields, and their descriminative cousins, Conditional Random Fields (CRF), and recently Support Vector Random Fields (SVRF).

    Possible projects

    
    References
    CRF

    DRF

    C-H.~Lee, R.~Greiner and M.~Schmidt, ``Support Vector Random Fields for Spatial Classification'', PKDD 2005.

17. Modeling Text and Images
    Images are oftened annotated with text, such as captions or tags, which can be viewed as an additional source of information when clustering images or building topic models. For example a green patch might indicate that there is a plant in the image, until one reads the caption "man in a green shirt". A related problem (Carbonetto et al. 2004) is data association, linking words to segmented objects in an image. For example, if the caption contains the words boat and sea, we would like to be able to associate these words with the segment(s) of the image corresponding to boat and sea.

    References

    D. Blei and M. Jordan. Modeling annotated data. In Proceedings of the 26th annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 127–134
    http://www.cs.princeton.edu/~blei/papers/BleiJordan2003.pdf

    Peter Carbonetto, Nando de Freitas and Kobus Barnard. 
    A Statistical Model for General Contextual Object Recognition. ECCV 2004
    http://www.cs.ubc.ca/~nando/papers/mrftranstwo.pdf

18. 2D CRFs for Visual Texture Classification
    

    Discriminative Fields for Modeling Spatial Dependencies in Natural Images is about applying 2D conditional random fields (CRFs) for classifying image regions as containing "man-made building" or not, on the basis of texture. The goal of this project is to reproduce the results in the NIPS 2003 paper. Useful links:

    • labeled training data.
    • C++ graphcuts code for approximate inference
    • Kevin Murphys Matlab CRF code
    • Carl Rasmussen's matlab conjugate gradient minimizer (better than using netlab or matlab optimization toolbox)
    • Intro to CRFs by Hanna Wallach
    • Maxent page, includes code
    • Steerable pyramid matlab code, possibly useful set of image features
    • Matlab wavelet toolbox, possibly useful set of image features .
    • Paper of CRFs for sign detection, J. Weinman, 2004
    • Markov Random Field Modeling in Computer Vision, S. Z. Li, 1995. (I have a hardcopy of the 2001 edition.)
    • G. Winkler, "Image Analysis, Random Fields, and MCMC Methods", 2nd edition, 2003.
    • Markov random fields and images, P. Perez. CWI Quarterly, 11(4):413-437, 1998. Review article.
19. 2D CRFs for satellite image classification
    The goal of this project is to classify pixels in satellite image data into classes like field vs road vs forest, using MRFs/CRFs (see above), or some other technique. Some possibly useful links:
    • Fully Bayesian Image Segmentation -- an Engineering Perspective, Morris et al, 1996.
    • A binary tree-structured MRF model for multispectral satellite image segmentation ,2003
      

Data Sets

Functional fMRI measures brain activation over time, which allows one to measure changes as an activity is performed (eg, looking at a picture of a cat vs. looking at a picture of a chair). Tasks using this data are typically of the form "predict cognitive state given fMRI data". fMRI data is both temporal and spatial: each voxel contains a time series, each voxel is correlated to voxels near it. 

http://multivac.ml.cmu.edu/10708

Data B: Corel Image Data

Images featurized by color histogram, color histogram layout, color moments, and co-occurence texture. Useful for projects on image segementation, especially since there is a large benchmark repository available.

Most segmentation algorithms have focused on segmentation based on edges or based on discontinuity of color and texture.  The ground-truth in this dataset, however, allows supervised learning algorithms to segment the images based on statistics calculated over regions.  One way to do this is to "oversegment" the image into superpixels (Felzenszwalb 2004, code available) and merge the superpixels into larger segments.  Graphical models can be used to represent smoothness in clusters, by adding appropriate potentials between neighboring pixels. In this project, you can address, for example, learning of such potentials, and inference in models with very large tree-width.

http://kdd.ics.uci.edu//databases/CorelFeatures/CorelFeatures.html
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/

Data C: Twenty Newsgroups

This data set contains 1000 text articles posted to each of 20 online newgroups, for a total of 20,000 articles. This data is useful for a variety of text classification and/or clustering projects.  The "label" of each article is which of the 20 newsgroups it belongs to.  The newsgroups (labels) are hierarchically organized (e.g., "sports", "hockey").

http://www-2.cs.cmu.edu/afs/cs/project/theo-11/www/naive-bayes.html