Preprints and Working Papers

  • Goal-Space Planning with Subgoal Models. Chunlok Lo, Kevin Roice, Parham Mohammad Panahi, Scott Jordan, Adam White, Gabor Mihucz, Farzane Aminmansour and Martha White. In Submission, 2024. [pdf]

  • Empirical Design in Reinforcement Learning. Andrew Patterson, Samuel Neumann, Martha White and Adam White. In Submission, 2023. [pdf]

Publications


  • The Cross-environment Hyperparameter Setting Benchmark for Reinforcement Learning. Andrew Patterson, Samuel Neumann, Raksha Kumaraswamy, Adam White, and Martha White. Reinforcement Learning Conference (RLC), 2024.

  • Investigating the Interplay of Prioritized Replay and Generalization. Parham Panahi, Andrew Patterson, Martha White, and Adam White. Reinforcement Learning Conference (RLC), 2024. [pdf]

  • Demystifying the Recency Heuristic in Temporal-Difference Learning. Brett Daley, Marlos Machado, and Martha White. Reinforcement Learning Conference (RLC), 2024. [pdf]

  • Multistep Predecessor Models and Mitigating Errors due to Hallucinated Value in Dyna-Style Planning. Farzane Aminmansour, Taher Jafferjee, Ehsan Imani, Erin Talvitie, Michael Bowling, and Martha White. Journal of AI Research (JAIR), 2024. [pdf]

  • Compound Returns Reduce Variance in Reinforcement Learning. Brett Daley, Martha White, and Marlos Machado. International Conference on Machine Learning (ICML), 2024. [pdf]

  • Position Paper: Limitations of and Alternatives to Benchmarking in Reinforcement Learning Research. Scott M. Jordan, Bruno Castro da Silva, Adam White, Martha White, and Philip S. Thomas. International Conference on Machine Learning (ICML), 2024. [pdf]

  • Offline Reinforcement Learning via Tsallis Regularization. Lingwei Zhu, Matthew Schlegel, Han Wang and Martha White. Transactions on Machine Learning Research (TMLR), 2024. [pdf]

  • Investigating the Properties of Neural Network Representations in Reinforcement Learning. Han Wang, Erfan Miahi, Martha White, Marlos C. Machado, Zaheer Abbas, Raksha Kumaraswamy, Vincent Liu and Adam White. Artificial Intelligence Journal (AIJ), 2023. [pdf]

  • GVFs in the Real World: Making Predictions Online for Water Treatment. Kamran Janjua, Haseeb Shah, Martha White, Erfan Miahi, Marlos Machado, and Adam White. Machine Learning (MLJ), 2023. [pdf]

  • General Munchausen Reinforcement Learning with Tsallis Kullback-Leibler Divergence. Lingwei Zhu, Zheng Chen, Matthew Schlegel and Martha White. Advances in Neural Information Processing Systems (NeurIPS), 2023. [pdf]

  • Resmax: An Alternative Soft-Greedy Operator for Reinforcement Learning. Erfan Miahi, Revan MacQueen, Alex Ayoub, Abbas Masoumzadeh and Martha White. Transactions on Machine Learning Research (TMLR), 2023. [pdf]

  • Scalable Real-Time Recurrent Learning Using Columnar-Constructive Networks. Khurram Javed, Haseeb Shah, Richard Sutton and Martha White. Journal of Machine Learning Research (JMLR), 2023. [pdf]

  • Trajectory-Aware Eligibility Traces for Off-Policy Reinforcement Learning. Brett Daley, Martha White, Chris Amato and Marlos Machado. International Conference on Machine Learning (ICML), 2023. [pdf]

  • Measuring and Mitigating Interference in Reinforcement Learning. Vincent Liu, Han Wang, Ruo Yu Tao, Khurram Javed, Adam White and Martha White. Conference on Lifelong Learning Agents (CoLLAs), 2023. [pdf]

  • Exploiting Action Impact Regularity and Exogenous State Variables for Offline Reinforcement Learning. Vincent Liu, James Wright and Martha White. Journal of Artificial Intelligence Research, 2023. [pdf]

  • Off-Policy Actor-Critic with Emphatic Weightings. Eric Graves, Ehsan Imani, Raksha Kumaraswamy and Martha White. Journal of Machine Learning Research (JMLR), 2023. [pdf]

  • Greedy Actor-Critic: A New Conditional Cross-Entropy Method for Policy Improvement. Samuel Neumann, Sungsu Lim, Ajin George Joseph, Yangchen Pan, Adam White and Martha White. International Conference on Representation Learning (ICLR), 2023. [pdf]

  • The In-Sample Softmax for Offline Reinforcement Learning. Chenjun Xiao, Han Wang, Yangchen Pan, Adam White and Martha White. International Conference on Representation Learning (ICLR), 2023. [pdf]

  • Asymptotically Unbiased Off-Policy Policy Evaluation when Reusing Old Data in Nonstationary Environments. Vincent Liu, Yash Chandak, Phillip Thomas and Martha White. International Conference on AI and Statistics (AISTATS), 2023.

  • Representation Alignment in Neural Networks . Ehsan Imani, Wei Hu and Martha White. Transactions on Machine Learning Research (TMLR), 2022. [pdf]

  • Robust Losses for Learning Value Functions. Andrew Patterson, Victor Liao and Martha White. Transactions on Pattern Analysis and Machine Learning (TPAMI), 2022. [pdf]

  • Greedification Operators for Policy Optimization: Investigating Forward and Reverse KL Divergences. Alan Chan, Hugo Silva, Sungsu Lim, Tadashi Kozuno, A. Rupam Mahmood and Martha White. Journal of Machine Learning Research (JMLR), 2022. [pdf]

  • No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RL. Han Wang, Archit Sakhadeo, Adam M White, James M Bell, Vincent Liu, Xutong Zhao, Puer Liu, Tadashi Kozuno, Alona Fyshe and Martha White. Transactions on Machine Learning Research (TMLR), 2022.
  • Understanding and Mitigating the Limitations of Prioritized Replay. Jincheng Mei, Yangchen Pan, Amir-massoud Farahmand, Hengshuai Yao and Martha White. Uncertainty in AI (UAI) , 2022. [pdf]

  • A Temporal-Difference Approach to Policy Gradient Estimation. Samuele Tosatto, Andrew Patterson, Martha White and A. Rupam Mahmood. International Conference on Machine Learning (ICML), 2022. [pdf]

  • A Generalized Projected Bellman Error for Off-policy Value Estimation in Reinforcement Learning. Andrew Patterson, Adam White and Martha White. Journal of Machine Learning Research (JMLR), 2022. [pdf]

  • An Alternate Policy Gradient Estimator for Softmax Policies. Shivam Garg, Yangchen Pan, Martha White and A. Rupam Mahmood. International Conference on on Artificial Intelligence and Statistics (AISTATS), 2022. [pdf]

  • Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentum. Kirby Banman, Liam Peet-Pare, Nidhi Hegde, Alona Fyshe and Martha White. International Conference on Learning Representations (ICLR), 2022. [pdf]

  • Sim2Real in Robotics and Automation: Applications and Challenges. S. Hofer, K. Bekris, A. Handa, J.C. Gamboa, M. Mozifian, F. Golemo, C. Atkeson, D. Fox, K. Goldberg, J. Leonard, C, Karen Liu, J. Peters, S. Song, P. Welinder, Peter and M. White. IEEE Transactions on Automation Science and Engineering, 2021. [pdf]

  • Continual Auxiliary Task Learning . Matthew McLeod, Chunlok Lo, Matthew Schlegel, Andrew Jacobsen, Raksha Kumaraswamy, Martha White and Adam White. Advances in Neural Information Processing Systems (NeurIPS), 2021. [pdf]

  • Structural Credit Assignment in Neural Networks using Reinforcement Learning. Dhawal Gupta, Gabor Mihucz, Matthew Schlegel, James Kostas, Philip Thomas, and Martha White. Advances in Neural Information Processing Systems (NeurIPS), 2021. [pdf]

  • Fuzzy Tiling Activations: A Simple Approach to Learning Sparse Representations Online. Yangchen Pan, Kirby Banman and Martha White. International Conference on Learning Representations (ICLR), 2021. [pdf]

  • General Value Function Networks. Matthew Schlegel, Andrew Jacobsen, Zaheer Abbas, Andrew Patterson, Adam White, and Martha White. Journal of AI Research (JAIR), 2021. [pdf]

  • An implicit function learning approach for parametric modal regression. Yancghen Pan, Ehsan Imani and Martha White. Advances in Neural Information Processing Systems (NeurIPS), 2020. [pdf]

  • Towards Safe Policy Improvement for Non-Stationary MDPs. Yash Chandak, Scott Jordan, Georgios Theocharous, Martha White, Philip S. Thomas. Advances in Neural Information Processing Systems (NeurIPS), 2020. [pdf]

  • Adapting Behaviour via Intrinsic Reward: A Survey and Empirical Study. Cam Linke, Nadia M. Ady, Martha White, Thomas Degris, and Adam White. Journal of AI Research (JAIR), 2020. [pdf]

  • Gradient Temporal-Difference Learning with Regularized Corrections. Sina Ghiassian, Andrew Patterson, Shivam Garg, Dhawal Gupta, Adam White and Martha White. International Conference on Machine Learning (ICML), 2020. [pdf]

  • Selective Dyna-style Planning Under Limited Model Capacity. Zaheer Abbas, Sam Sokota, Erin Talvitie and Martha White. International Conference on Machine Learning (ICML), 2020. [pdf]

  • Optimizing for the Future in Non-Stationary MDPs. Yash Chandak, Georgios Theocharous, Shiv Shankar, Martha White, Sridhar Mahadevan, Philip S. Thomas International Conference on Machine Learning (ICML), 2020. [pdf]

  • Training Recurrent Neural Networks Online by Learning Explicit State Variables. Somjit Nath, Vincent Liu, Alan Chan, Adam White and Martha White. International Conference on Learning Representations (ICLR), 2020. [pdf]

  • Maxmin Q-learning: Controlling the Estimation Bias of Q-learning. Qingfeng Lan, Yangchen Pan, Alona Fyshe and Martha White. International Conference on Learning Representations (ICLR), 2020. [pdf]

  • Maximizing Information Gain in Partially Observable Environments via Prediction Rewards. Yash Satsangi, Sungsu Lim, Shimon Whiteson, Frans Oliehoek and Martha White. International Conference on Autonomous Agents and Multi-agent Systems (AAMAS), 2020. [pdf]

  • Meta-Learning Representations for Continual Learning. Khurram Javed and Martha White. Advances in Neural Information Processing Systems (NeurIPS), 2019. [pdf]

  • Importance Resampling for Off-policy Prediction. Matthew Schlegel, Wesley Chung, Daniel Graves, Jian Qian and Martha White. Advances in Neural Information Processing Systems (NeurIPS), 2019. [pdf]

  • Learning Macroscopic Brain Connectomes via Group-Sparse Factorization. Farzane Aminmansour, Andrew Patterson, Lei Le, Yisu Peng, Daniel Mitchell, Franco Pestilli, Cesar Caiafa, Russell Greiner and Martha White. Advances in Neural Information Processing Systems (NeurIPS), 2019.

  • Planning with Expectation Models. Yi Wan, Muhammad Zaheer, Adam White, Martha White and Richard S. Sutton. International Joint Conference on Artificial Intelligence (IJCAI), 2019. [pdf]

  • Hill Climbing on Value Estimates for Search-control in Dyna. Yangchen Pan, Hengshuai Yao, Amir-massoud Farahmand and Martha White. International Joint Conference on Artificial Intelligence (IJCAI), 2019. [pdf]

  • Two-Timescale Networks for Nonlinear Value Function Approximation . Wesley Chung, Somjit Nath, Ajin Joseph and Martha White. International Conference on Learning Representations (ICLR), 2019. [pdf]

  • The Utility of Sparse Representations for Control in Reinforcement Learning. Vincent Liu, Raksha Kumaraswamy, Lei Le and Martha White. AAAI Conference on Artificial Intelligence, 2019. [pdf]

  • Meta-descent for online, continual prediction. Andrew Jacobsen, Matthew Schlegel, Cam Linke, Thomas Degris, Adam White and Martha White. AAAI Conference on Artificial Intelligence, 2019. [pdf]

  • An Off-policy Policy Gradient Theorem Using Emphatic Weightings. Ehsan Imani, Eric Graves and Martha White. Advances in Neural Information Processing Systems (NIPS), 2018. [pdf]

  • Supervised autoencoders: Improving generalization performance with unsupervised regularizers. Lei Le, Andrew Patterson and Martha White. Advances in Neural Information Processing Systems (NIPS), 2018. [pdf]

  • Context-dependent upper-confidence bounds for directed exploration. Raksha Kumaraswamy, Matthew Schlegel, Adam White and Martha White. Advances in Neural Information Processing Systems (NIPS), 2018. [pdf]

  • Improving Regression Performance with Distributional Losses. Ehsan Imani and Martha White. International Conference on Machine Learning (ICML), 2018. [pdf]

  • Reinforcement Learning with Function-Valued Action Spaces for Partial Differential Equation Control. Yangchen Pan, Amir-massoud Farahmand, Martha White, Saleh Nabi, Piyush Grover, Daniel Nikovski. International Conference on Machine Learning (ICML), 2018. [pdf]

  • Organizing experience: a deeper look at replay mechanisms for sample-based planning in continuous state domains. Yangchen Pan, Muhammad Zaheer, Adam White, Andrew Patterson, Martha White International Joint Conference on Artificial Intelligence (IJCAI), 2018. [pdf]

  • High-confidence error estimates for learned value functions. Touqir Sajed, Wesley Chung and Martha White. Uncertainty in Artificial Intelligence (UAI), 2018. [pdf]

  • Comparing Direct and Indirect Temporal-Difference Methods for Estimating the Variance of the Return. Craig Sherstan, Dylan Ashley, Brendan Bennet, Kenny Young, Adam White, Martha White, Richard Sutton. Uncertainty in Artificial Intelligence (UAI), 2018. [pdf]

  • Multi-view Matrix Factorization for Linear Dynamical System Estimation. Mahdi Karami, Martha White, Dale Schuurmans and Csaba Szepesvari. Advances in Neural Information Processing Systems (NIPS), 2017. [pdf] 

  • Unifying task specification in reinforcement learning. Martha White. International Conference on Machine Learning (ICML), 2017. [pdf]

  • Adapting kernel representations online using submodular maximization. Matthew Schlegel, Yangchen Pan and Martha White. International Conference on Machine Learning (ICML), 2017. [pdf]

  • Effective sketching methods for value function approximation. Yangchen Pan, Erfan Sadeqi Azer and Martha White. Uncertainty in Artificial Intelligence (UAI), 2017. [pdf]

  • Learning sparse representations in reinforcement learning with sparse coding. Lei Le, Raksha Kumaraswamy and Martha White. International Joint Conference on Artificial Intelligence (IJCAI), 2017. [pdf]

  • Accelerated Gradient Temporal Difference Learning. Yangchen Pan, Adam White and Martha White. AAAI Conference on Artificial Intelligence, 2017. [pdf]

  • Recovering true classifier performance in positive-unlabeled learning. Shantanu Jain, Martha White and Predrag Radivojac. [pdf]
    AAAI Conference on Artificial Intelligence, 2017.

  • Estimating the class prior and posterior from noisy positives and unlabeled data . Shantanu Jain, Martha White and Predrag Radivojac. Advances in Neural Information Processing Systems (NIPS), 2016. [pdf] 

  • Identifying global optimality for dictionary learning. Lei Le and Martha White. In submission to JMLR, 2016. [pdf] 

  • Nonparametric semi-supervised learning of class proportions. Shantanu Jain, Martha White, Michael W. Trosset and Predrag Radivojac. In submission to JMLR, 2016. [pdf] 

  • Incremental Truncated LSTD. Clement Gehring, Yangchen Pan and Martha White. International Joint Conference on Artificial Intelligence (IJCAI), 2016. [pdf]

  • Investigating practical, linear temporal difference learning. Adam White and Martha White. Autonomous Agents and Multi-agent Systems (AAMAS), 2016. [pdf]

  • A Greedy Approach to Adapting the Trace Parameter for Temporal Difference Learning . Adam White and Martha White. Autonomous Agents and Multi-agent Systems (AAMAS), 2016. [pdf] 

  • Scalable Metric Learning for Co-embedding. Farzaneh Mirzazadeh, Martha White, Andras Gyorgy and Dale Schuurmans. ECML PKDD, 2015. [pdf]

  • An Emphatic Approach to the Problem of Off-policy Temporal-Difference Learning. Richard S. Sutton, A Rupam Mahmood and Martha White. Journal of Machine Learning Research (JMLR), 2016. [pdf] 

  • Optimal Estimation of Multivariate ARMA Models. Martha White, Junfeng Wen, Michael Bowling and Dale Schuurmans. AAAI Conference on Artificial Intelligence, 2015. [pdf] 

  • Partition Tree Weighting. Joel Veness, Martha White, Michael Bowling and Andras Gyorgy. Data Compression Conference (DCC), 2013. [pdf] 

  • Convex Multi-view Subspace Learning. Martha White, Yaoliang Yu, Xinhua Zhang, and Dale Schuurmans. Advances in Neural Information Processing Systems (NIPS), 2012. [pdf] 

  • Off-Policy Actor-Critic. Thomas Degris, Martha White, Richard S. Sutton. In International Conference on Machine Learning (ICML), 2012. [pdf] 

  • Generalized Optimal Reverse Prediction. Martha White and Dale Schuurmans. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2012. [pdf] 

  • Convex Sparse Coding, Subspace Learning, and Semi-Supervised Extensions. Xinhua Zhang, Yaoliang Yu, Martha White, Ruitong Huang and Dale Schuurmans. In AAAI Conference on Artificial Intelligence (AAAI), 2011. [pdf] 

  • Interval Estimation for Reinforcement-Learning Algorithms in Continuous-State Domains. Martha White and Adam White. In Advances in Neural Information Processing Systems (NIPS), 2010. [pdf] 

  • Relaxed Clipping: A Global Training Method for Robust Regression and Classification. Yaoliang Yu, Min Yang, Linli Xu, Martha White, Dale Schuurmans. In Advances in Neural Information Processing Systems (NIPS), 2010. [pdf] 

  • Learning a Value Analysis Tool For Agent Evaluation. Martha White and Michael Bowling. In International Joint Conference on Artificial Intelligence (IJCAI), 2009. [pdf] 

  • Optimal Reverse Prediction: A Unified Perspective on Supervised, Unsupervised and Semi-supervised Learning. Linli Xu, Martha White, and Dale Schuurmans. In International Conference on Machine Learning (ICML), 2009. [pdf] 

Theses

  • Martha White. Regularized factor models. PhD thesis,
    Details     BibTeX     Download: [pdf] 
  • Martha White. A General Framework for Reducing Variance in Agent Evaluation. Master's thesis,
    Details     BibTeX     Download: [pdf]