One-hour Presentation about Cost Curves

Papers Describing Cost Curves

• Cost curves: An improved Method for visualizing classifier performance , Chris Drummond and Robert C. Holte (2006), Machine Learning, volume 65(1), pp. 95-130. Official Link to this paper on Springer's web pages
• What ROC Curves Can't Do (and Cost Curves Can) Chris Drummond and Robert C. Holte (2004). ECAI'2004 Workshop on ROC Analysis in AI.
• Explicitly Representing Expected Cost: An Alternative to ROC Representation. Chris Drummond, and Robert C. Holte (2000). Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 198-207.

Papers that Use or Refer to Cost Curves

I stopped maintaining this list in 2009 because an up-to-date list can be found via the Google Scholar entries for the above papers. These can be found on my Google Scholar profile ( click here)

• Information, Divergence and Risk for Binary Experiments, Mark D. Reid and Robert C. Williamson, http://arxiv.org/abs/0901.0356v1, January 2009.
• Cost curve evaluation of fault prediction models, Y. Jiang, B. Cukic, and T. Menzies (2008), in the 19th International Symposium on Software Reliability Engineering, pages 197-206, Nov. 2008.
• Application of cost matrices and cost curves to enhance diagnostic health management metrics for gas turbine engines. Craig Davison and Chris Drummond, Journal of Engineering for Gas Turbines and Power 132(4), 2010.
• Automatically countering imbalance and its empirical relationship to cost, Nitesh V. Chawla, David A. Cieslak, Lawrence O. Hall, and Ajay Joshi, Data Mining and Knowledge Discovery, Volume 17, Number 2 (October, 2008) Pages 225-252.
• A New Evaluation Measure for Imbalanced Datasets, Cheng G. Weng, Josiah Poon (2008), in AusDM 2008: 27-32
• "Modeling churn using customer lifetime value", Nicolas Gladya, Bart Baesensa, and Christophe Croux (2009), European Journal of Operational Research Volume 197, Issue 1, Pages 402-411.
• "Automatic Target Recognition and Fratricide Reduction", Bruce Fowler and William C. McCorkle (March, 2006), Report TR-AMR-IN-05-01, US Army Research, Development, and Engineering Command.
• "Evaluation of Classifiers: Practical Considerations for Security Applications", Alvaro A. Cardenas and John S. Baras (2006), Proceedings of the AAAI Workshop on Evaluation Methods for Machine Learning, Boston, Massachusetts, July 16-20, 2006.
• "Exact Bootstrap Distributions of Cost Curves", Charles Dugas and David Gadoury (2008), ICML'08.
• "Active Learning to Maximize Area Under the ROC Curve", Matt Culver, Deng Kun, and Stephen Scott (2006), pp.149-158, Sixth IEEE International Conference on Data Mining (ICDM'06).
• "Automatically countering imbalance and its empirical relationship to cost" Nitesh V. Chawla, David A. Cieslak, Lawrence O. Hall and Ajay Joshi (2008), Data Mining and Knowledge Discovery, DOI 10.1007/s10618-008-0087-0.
• "Capturing Heuristics and Intelligent Methods for Improving Micro-array Data Classification", Andrea Bosin, Nicoletta Dessi and Barbara Pes (2007), Proceedings of Intelligent Data Engineering and Automated Learning - IDEAL 2007, LNCS 4881, pp.790-799.
• "Rejoinder: The Skill Plot", William M. Briggs and Russell Zaretzki (2007), Biometrics
• "Discriminative vs. Generative Classifiers for Cost Sensitive Learning", Chris Drummond (2006), Proceedings of the Nineteenth Canadian Conference on Artificial Intelligence, LNAI 4013, pp.479-490.
• "Evaluation of classifiers for an uneven class distribution problem", Sophia Daskalaki, Ioannis Kopanas, Nikolaos Avouris, Applied Artificial Intelligence, 20, pp. 381-417, 2006.
• Cost Curves for Abstaining Classifiers, Caroline C. Friedel, 2006 ROCML workshop.
• Elaboration on Two Points Raised in "Classifier Technology and the Illusion of Progress" , Robert C. Holte (2006), Statistical Science, Vol. 21, Number 1, pp. 24-26.
• Maximizing Classifier Utility when Training Data is Costly, Gary Weiss and Ye Tian (2006), Second Workshop on Utility-Based Data Mining, held in conjunction with The 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2006).
• Training Cost-Sensitive Neural Networks with Methods Addressing the Class Imbalance Problem, Zhi-Hua Zhou, Xu-Ling Liu (2006), IEEE Transactions on Knowledge and Data Engineering, vol. 18(1), pp. 63-77.
• "Cost-sensitive learning and decision making revisited", Stijn Viaene and Guido Dedene (2005), European Journal of Operational Research, Volume 166, Issue 1, October 2005, Pages 212-220. on Artificial Intelligence, LNAI 4013, pp.479-490.
• Scalable and robust group discovery on large transactional data Patrick Pakyan Choi, Andrew Moore, and Jeremy Kubica, technical report CMU-RI-TR-05-60, December 2005.
• Severe Class Imbalance: Why Better Algorithms Aren't the Answer , Chris Drummond and Robert Holte (2005), Proceedings of the Sixteenth European Conference of Machine Learning, LNAI 3720, pp.539-546
• Wrapper-based Computation and Evaluation of Sampling Methods for Imbalanced Datasets, Nitesh V. Chawla, Lawrence O. Hall, Ajay Joshi (2005), workshop on Utility-Based Data Mining held in conjunction with the 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2005).
• On Abstaining Classifiers, Caroline Friedel (2005), Masterarbeit (Master's Thesis), Ludwig-Maximilians Universitat/Technische Universitat Munchen, Institut fur Informatik XII.
• Learning to Live with False Alarms, Chris Drummond and Robert C. Holte (2005). "Data Mining Methods for Anomaly Detection" workshop at the 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2005). pp. 21-24.
• The many faces of ROC analysis in machine learning, Peter Flach (2004), tutorial at ICML.
• "In vivo" spam filtering: A challenge problem for data mining, Tom Fawcett (2003), SIGKDD Explorations 5(2): 140-148.
• C4.5, Class Imbalance, and Cost Sensitivity: Why Under-Sampling beats Over-Sampling Chris Drummond and Robert C. Holte (2003). ICML'2003 Workshop on Learning from Imbalanced Datasets II.
• Learning When Data Sets are Imbalanced and When Costs are Unequal and Unknown, Marcus A. Maloof (2003), ICML Workshop on Learning from Imbalanced Data Sets II.
• Model Stability: A key factor in determining whether an algorithm produces an optimal model from a matching distribution, Kai Ming Ting, Regina Jing Ying Quek (2003), Third IEEE International Conference on Data Mining (ICDM'03), pp. 653-656.
• ROC Graphs: Notes and Practical Considerations for Data Mining Researchers, Tom Fawcett (2003), HP Labs Tech Report HPL-2003-4.
• Issues in Classifier Evaluation using Optimal Cost Curves, Kai Ming Ting (2002), ICML, pp. 642-649.
• Robust Classification for Imprecise Environments, Foster Provost, Tom Fawcett (2001), Machine Learning 42(3): 203-231.
• Exploiting the Cost (In)sensitivity of Decision Tree Splitting Criteria. Chris Drummond, and Robert C. Holte (2000). Proceedings of the 17th International Conference on Machine Learning (ICML'2000), pp. 239-246.