# Awesome Monte Carlo Tree Search Papers.

A curated list of Monte Carlo tree search papers with implementations from the following conferences/journals:

- Machine learning
  - NeurIPS
  - ICML
- Computer vision
  - CVPR
  - ICCV
- Natural language processing
  - ACL
- Data
  - KDD
- Artificial intelligence
  - AAAI
  - AISTATS
  - IJCAI
  - UAI
- Robotics
  - RAS
- Games
  - CIG

Similar collections about graph classification, gradient boosting, classification/regression trees, fraud detection, and community detection papers with implementations.

## 2023

- Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree search (ICLR 2023)
  - Fangzheng Sun, Yang Liu, Jian-Xun Wang, Hao Sun
  - [Paper]

## 2022

- Finding Backdoors to Integer Programs: A Monte Carlo Tree Search Framework (AAAI 2022)
  - Elias B. Khalil, Pashootan Vaezipoor, Bistra Dilkina
  - [Paper]
- NSGZero: Efficiently Learning Non-exploitable Policy in Large-Scale Network Security Games with Neural Monte Carlo Tree Search (AAAI 2022)
  - Wanqi Xue, Bo An, Chai Kiat Yeo
  - [Paper]
- Solving Disjunctive Temporal Networks with Uncertainty under Restricted Time-Based Controllability Using Tree Search and Graph Neural Networks (AAAI 2022)
  - Kevin Osanlou, Jeremy Frank, Andrei Bursuc, Tristan Cazenave, Eric Jacopin, Christophe Guettier, J. Benton
  - [Paper]
- Qubit Routing Using Graph Neural Network Aided Monte Carlo Tree Search (AAAI 2022)
  - Animesh Sinha, Utkarsh Azad, Harjinder Singh
  - [Paper]
- Split Moves for Monte-Carlo Tree Search (AAAI 2022)
  - Jakub Kowalski, Maksymilian Mika, Wojciech Pawlik, Jakub Sutowicz, Marek Szykula, Mark H. M. Winands
  - [Paper]
- Procrastinated Tree Search: Black-Box Optimization with Delayed%2C Noisy and Multi-Fidelity Feedback (AAAI 2022)
  - Junxiong Wang, Debabrota Basu, Immanuel Trummer
  - [Paper]
- Enabling Arbitrary Translation Objectives with Adaptive Tree Search (ICLR 2022)
  - Wang Ling, Wojciech Stokowiec, Domenic Donato, Chris Dyer, Lei Yu, Laurent Sartran, Austin Matthews
  - [Paper]
- What’s Wrong with Deep Learning in Tree Search for Combinatorial Optimization (ICLR 2022)
  - Maximili1an Böther, Otto Kißig, Martin Taraz, Sarel Cohen, Karen Seidel, Tobias Friedrich
  - [Paper]
- Anytime Capacity Expansion in Medical Residency Match by Monte Carlo Tree Search (IJCAI 2022)
  - Kenshi Abe, Junpei Komiyama, Atsushi Iwasaki
  - [Paper]
- Fast and Accurate User Cold-Start Learning Using Monte Carlo Tree Search (RECSYS 2022)
  - Dilina Chandika Rajapakse, Douglas Leith
  - [Paper]

## 2021

- Learning to Stop: Dynamic Simulation Monte-Carlo Tree Search (AAAI 2021)
  - Li-Cheng Lan, Ti-Rong Wu, I-Chen Wu, Cho-Jui Hsieh
  - [Paper]
- Dec-SGTS: Decentralized Sub-Goal Tree Search for Multi-Agent Coordination (AAAI 2021)
  - Minglong Li, Zhongxuan Cai, Wenjing Yang, Lixia Wu, Yinghui Xu, Ji Wang
  - [Paper]
- Improved POMDP Tree Search Planning with Prioritized Action Branching (AAAI 2021)
  - John Mern, Anil Yildiz, Lawrence Bush, Tapan Mukerji, Mykel J. Kochenderfer
  - [Paper]
- Dynamic Automaton-Guided Reward Shaping for Monte Carlo Tree Search (AAAI 2021)
  - Alvaro Velasquez, Brett Bissey, Lior Barak, Andre Beckus, Ismail Alkhouri, Daniel Melcer, George K. Atia
  - [Paper]
- Single Player Monte-Carlo Tree Search Based on the Plackett-Luce Model (AAAI 2021)
  - Felix Mohr, Viktor Bengs, Eyke Hüllermeier
  - [Paper]
- Learning to Pack: A Data-Driven Tree Search Algorithm for Large-Scale 3D Bin Packing Problem (CIKM 2021)
  - Qianwen Zhu, Xihan Li, Zihan Zhang, Zhixing Luo, Xialiang Tong, Mingxuan Yuan, Jia Zeng
  - [Paper]
- Practical Massively Parallel Monte-Carlo Tree Search Applied to Molecular Design (ICLR 2021)
  - Xiufeng Yang, Tanuj Kr Aasawat, Kazuki Yoshizoe
  - [Paper]
- Convex Regularization in Monte-Carlo Tree Search (ICML 2021)
  - Tuan Dam, Carlo D’Eramo, Jan Peters, Joni Pajarinen
  - [Paper]
- Combining Tree Search and Action Prediction for State-of-the-Art Performance in DouDiZhu (IJCAI 2021)
  - Yunsheng Zhang, Dong Yan, Bei Shi, Haobo Fu, Qiang Fu, Hang Su, Jun Zhu, Ning Chen
  - [Paper]

## 2020

- Monte Carlo Tree Search in Continuous Spaces Using Voronoi Optimistic Optimization with Regret Bounds (AAAI 2020)
  - Beomjoon Kim, Kyungjae Lee, Sungbin Lim, Leslie Pack Kaelbling, Tomás Lozano-Pérez
  - [Paper]
- Neural Architecture Search Using Deep Neural Networks and Monte Carlo Tree Search (AAAI 2020)
  - Linnan Wang, Yiyang Zhao, Yuu Jinnai, Yuandong Tian, Rodrigo Fonseca
  - [Paper]
  - [Code]
- Monte-Carlo Tree Search in Continuous Action Spaces with Value Gradients (AAAI 2020)
  - Jongmin Lee, Wonseok Jeon, Geon-Hyeong Kim, Kee-Eung Kim
  - [Paper]
  - [Code]
- Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions (AISTATS 2020)
  - Lars Buesing, Nicolas Heess, Theophane Weber
  - [Paper]
- Watch the Unobserved: A Simple Approach to Parallelizing Monte Carlo Tree Search (ICLR 2020)
  - Anji Liu, Jianshu Chen, Mingze Yu, Yu Zhai, Xuewen Zhou, Ji Liu
  - [Paper]
  - [Code]
- Information Particle Filter Tree: An Online Algorithm for POMDPs with Belief-Based Rewards on Continuous Domains (ICML 2020)
  - Johannes Fischer, Ömer Sahin Tas
  - [Paper]
  - [Code]
- Sub-Goal Trees a Framework for Goal-Based Reinforcement Learning (ICML 2020)
  - Tom Jurgenson, Or Avner, Edward Groshev, Aviv Tamar
  - [Paper]
- Monte-Carlo Tree Search for Scalable Coalition Formation (IJCAI 2020)
  - Feng Wu, Sarvapali D. Ramchurn
  - [Paper]
- Generalized Mean Estimation in Monte-Carlo Tree Search (IJCAI 2020)
  - Tuan Dam, Pascal Klink, Carlo D’Eramo, Jan Peters, Joni Pajarinen
  - [Paper]
- Sparse Tree Search Optimality Guarantees in POMDPs with Continuous Observation Spaces (IJCAI 2020)
  - Michael H. Lim, Claire Tomlin, Zachary N. Sunberg
  - [Paper]
- Mix and Match: An Optimistic Tree-Search Approach for Learning Models from Mixture Distributions (NeurIPS 2020)
  - Matthew Faw, Rajat Sen, Karthikeyan Shanmugam, Constantine Caramanis, Sanjay Shakkottai
  - [Paper]
- Extracting Knowledge from Web Text with Monte Carlo Tree Search (WWW 2020)
  - Guiliang Liu, Xu Li, Jiakang Wang, Mingming Sun, Ping Li
  - [Paper]

## 2019

- ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search (AAAI 2019)
  - Shangtong Zhang, Hengshuai Yao
  - [Paper]
  - [Code]
- A Monte Carlo Tree Search Player for Birds of a Feather Solitaire (AAAI 2019)
  - Christian Roberson, Katarina Sperduto
  - [Paper]
  - [Code]
- Vine Copula Structure Learning via Monte Carlo Tree Search (AISTATS 2019)
  - Bo Chang, Shenyi Pan, Harry Joe
  - [Paper]
  - [Code]
- Noisy Blackbox Optimization using Multi-fidelity Queries: A Tree Search Approach (AISTATS 2019)
  - Rajat Sen, Kirthevasan Kandasamy, Sanjay Shakkottai
  - [Paper]
  - [Code]
- Reinforcement Learning Based Monte Carlo Tree Search for Temporal Path Discovery (ICDM 2019)
  - Pengfei Ding, Guanfeng Liu, Pengpeng Zhao, An Liu, Zhixu Li, Kai Zheng
  - [Paper]
- Monte Carlo Tree Search for Policy Optimization (IJCAI 2019)
  - Xiaobai Ma, Katherine Rose Driggs-Campbell, Zongzhang Zhang, Mykel J. Kochenderfer
  - [Paper]
- Subgoal-Based Temporal Abstraction in Monte-Carlo Tree Search (IJCAI 2019)
  - Thomas Gabor, Jan Peter, Thomy Phan, Christian Meyer, Claudia Linnhoff-Popien
  - [Paper]
  - [Code]
- Automated Machine Learning with Monte-Carlo Tree Search (IJCAI 2019)
  - Herilalaina Rakotoarison, Marc Schoenauer, Michèle Sebag
  - [Paper]
  - [Code]
- Multiple Policy Value Monte Carlo Tree Search (IJCAI 2019)
  - Li-Cheng Lan, Wei Li, Ting-Han Wei, I-Chen Wu
  - [Paper]
- Learning Compositional Neural Programs with Recursive Tree Search and Planning (NeurIPS 2019)
  - Thomas Pierrot, Guillaume Ligner, Scott E. Reed, Olivier Sigaud, Nicolas Perrin, Alexandre Laterre, David Kas, Karim Beguir, Nando de Freitas
  - [Paper]

## 2018

- Monte Carlo Methods for the Game Kingdomino (CIG 2018)
  - Magnus Gedda, Mikael Z. Lagerkvist, Martin Butler
  - [Paper]
  - [Code]
  - [Game Server]
- Reset-free Trial-and-Error Learning for Robot Damage Recovery (RAS 2018)
  - Konstantinos Chatzilygeroudis, Vassilis Vassiliades, Jean-Baptiste Mouret
  - [Paper]
  - [Code]
  - [MCTS C++ Library]
- Memory-Augmented Monte Carlo Tree Search (AAAI 2018)
  - Chenjun Xiao, Jincheng Mei, Martin Müller
  - [Paper]
- Feedback-Based Tree Search for Reinforcement Learning (ICML 2018)
  - Daniel R. Jiang, Emmanuel Ekwedike, Han Liu
  - [Paper]
- Extended Increasing Cost Tree Search for Non-Unit Cost Domains (IJCAI 2018)
  - Thayne T. Walker, Nathan R. Sturtevant, Ariel Felner
  - [Paper]
- Three-Head Neural Network Architecture for Monte Carlo Tree Search (IJCAI 2018)
  - Chao Gao, Martin Müller, Ryan Hayward
  - [Paper]
- Bidding in Periodic Double Auctions Using Heuristics and Dynamic Monte Carlo Tree Search (IJCAI 2018)
  - Moinul Morshed Porag Chowdhury, Christopher Kiekintveld, Son Tran, William Yeoh
  - [Paper]
- Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search (NIPS 2018)
  - Zhuwen Li, Qifeng Chen, Vladlen Koltun
  - [Paper]
- M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search (NIPS 2018)
  - Yelong Shen, Jianshu Chen, Po-Sen Huang, Yuqing Guo, Jianfeng Gao
  - [Paper]
- Single-Agent Policy Tree Search With Guarantees (NIPS 2018)
  - Laurent Orseau, Levi Lelis, Tor Lattimore, Theophane Weber
  - [Paper]
- Monte-Carlo Tree Search for Constrained POMDPs (NIPS 2018)
  - Jongmin Lee, Geon-hyeong Kim, Pascal Poupart, Kee-Eung Kim
  - [Paper]

## 2017

- An Analysis of Monte Carlo Tree Search (AAAI 2017)
  - Steven James, George Dimitri Konidaris, Benjamin Rosman
  - [Paper]
- Beyond Monte Carlo Tree Search: Playing Go with Deep Alternative Neural Network and Long-Term Evaluation (AAAI 2017)
  - Jinzhuo Wang, Wenmin Wang, Ronggang Wang, Wen Gao
  - [Paper]
- Designing Better Playlists with Monte Carlo Tree Search (AAAI 2017)
  - Elad Liebman, Piyush Khandelwal, Maytal Saar-Tsechansky, Peter Stone
  - [Paper]
- Learning in POMDPs with Monte Carlo Tree Search (ICML 2017)
  - Sammie Katt, Frans A. Oliehoek, Christopher Amato
  - [Paper]
- Learning to Run Heuristics in Tree Search (IJCAI 2017)
  - Elias B. Khalil, Bistra Dilkina, George L. Nemhauser, Shabbir Ahmed, Yufen Shao
  - [Paper]
- Estimating the Size of Search Trees by Sampling with Domain Knowledge (IJCAI 2017)
  - Gleb Belov, Samuel Esler, Dylan Fernando, Pierre Le Bodic, George L. Nemhauser
  - [Paper]
- A Monte Carlo Tree Search Approach to Active Malware Analysis (IJCAI 2017)
  - Riccardo Sartea, Alessandro Farinelli
  - [Paper]
- Monte-Carlo Tree Search by Best Arm Identification (NIPS 2017)
  - Emilie Kaufmann, Wouter M. Koolen
  - [Paper]
- Thinking Fast and Slow with Deep Learning and Tree Search (NIPS 2017)
  - Thomas Anthony, Zheng Tian, David Barber
  - [Paper]
- Monte-Carlo Tree Search using Batch Value of Perfect Information (UAI 2017)
  - Shahaf S. Shperberg, Solomon Eyal Shimony, Ariel Felner
  - [Paper]

## 2016

- Using Domain Knowledge to Improve Monte-Carlo Tree Search Performance in Parameterized Poker Squares (AAAI 2016)
  - Robert Arrington, Clay Langley, Steven Bogaerts
  - [Paper]
- Monte Carlo Tree Search for Multi-Robot Task Allocation (AAAI 2016)
  - Bilal Kartal, Ernesto Nunes, Julio Godoy, Maria L. Gini
  - [Paper]
- Large Scale Hard Sample Mining with Monte Carlo Tree Search (CVPR 2016)
  - Olivier Canévet, François Fleuret
  - [Paper]
- On the Analysis of Complex Backup Strategies in Monte Carlo Tree Search (ICML 2016)
  - Piyush Khandelwal, Elad Liebman, Scott Niekum, Peter Stone
  - [Paper]
- Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Games (IJCAI 2016)
  - Xiaoxiao Guo, Satinder P. Singh, Richard L. Lewis, Honglak Lee
  - [Paper]
- Monte Carlo Tree Search in Continuous Action Spaces with Execution Uncertainty (IJCAI 2016)
  - Timothy Yee, Viliam Lisý, Michael H. Bowling
  - [Paper]
- Learning Predictive State Representations via Monte-Carlo Tree Search (IJCAI 2016)
  - Yunlong Liu, Hexing Zhu, Yifeng Zeng, Zongxiong Dai
  - [Paper]

## 2015

- Efficient Globally Optimal Consensus Maximisation with Tree Search (CVPR 2015)
  - Tat-Jun Chin, Pulak Purkait, Anders P. Eriksson, David Suter
  - [Paper]
- Interplanetary Trajectory Planning with Monte Carlo Tree Search (IJCAI 2015)
  - Daniel Hennes, Dario Izzo
  - [Paper]

## 2014

- State Aggregation in Monte Carlo Tree Search (AAAI 2014)
  - Jesse Hostetler, Alan Fern, Tom Dietterich
  - [Paper]
- Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning (NIPS 2014)
  - Xiaoxiao Guo, Satinder P. Singh, Honglak Lee, Richard L. Lewis, Xiaoshi Wang
  - [Paper]
- Learning Partial Policies to Speedup MDP Tree Search (UAI 2014)
  - Jervis Pinto, Alan Fern
  - [Paper]

## 2013

- Monte Carlo Tree Search for Scheduling Activity Recognition (ICCV 2013)
  - Mohamed R. Amer, Sinisa Todorovic, Alan Fern, Song-Chun Zhu
  - [Paper]
- Convergence of Monte Carlo Tree Search in Simultaneous Move Games (NIPS 2013)
  - Viliam Lisý, Vojtech Kovarík, Marc Lanctot, Branislav Bosanský
  - [Paper]
- Bayesian Mixture Modelling and Inference based Thompson Sampling in Monte-Carlo Tree Search (NIPS 2013)
  - Aijun Bai, Feng Wu, Xiaoping Chen
  - [Paper]

## 2012

- Generalized Monte-Carlo Tree Search Extensions for General Game Playing (AAAI 2012)
  - Hilmar Finnsson
  - [Paper]

## 2011

- A Local Monte Carlo Tree Search Approach in Deterministic Planning (AAAI 2011)
  - Fan Xie, Hootan Nakhost, Martin Müller
  - [Paper]
- Real-Time Solving of Quantified CSPs Based on Monte-Carlo Game Tree Search (IJCAI 2011)
  - Satomi Baba, Yongjoon Joe, Atsushi Iwasaki, Makoto Yokoo
  - [Paper]
- Nested Rollout Policy Adaptation for Monte Carlo Tree Search (IJCAI 2011)
  - Christopher D. Rosin
  - [Paper]
- Variance Reduction in Monte-Carlo Tree Search (NIPS 2011)
  - Joel Veness, Marc Lanctot, Michael H. Bowling
  - [Paper]
- Learning Is Planning: Near Bayes-Optimal Reinforcement Learning via Monte-Carlo Tree Search (UAI 2011)
  - John Asmuth, Michael L. Littman
  - [Paper]

## 2010

- Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search (AAAI 2010)
  - Jeffrey Richard Long, Nathan R. Sturtevant, Michael Buro, Timothy Furtak
  - [Paper]
- Bayesian Inference in Monte-Carlo Tree Search (UAI 2010)
  - Gerald Tesauro, V. T. Rajan, Richard Segal
  - [Paper]

## 2009

- Monte Carlo Tree Search Techniques in the Game of Kriegspiel (IJCAI 2009)
  - Paolo Ciancarini, Gian Piero Favini
  - [Paper]
- Bootstrapping from Game Tree Search (NIPS 2009)
  - Joel Veness, David Silver, William T. B. Uther, Alan Blair
  - [Paper]

## 2008

- Direct Mining of Discriminative and Essential Frequent Patterns via Model-Based Search Tree (KDD 2008)
  - Wei Fan, Kun Zhang, Hong Cheng, Jing Gao, Xifeng Yan, Jiawei Han, Philip S. Yu, Olivier Verscheure
  - [Paper]

## 2007

- Bandit Algorithms for Tree Search (UAI 2007)
  - Pierre-Arnaud Coquelin, Rémi Munos
  - [Paper]

## 2006

- Properties of Forward Pruning in Game-Tree Search (AAAI 2006)
  - Yew Jin Lim, Wee Sun Lee
  - [Paper]
- Graph Branch Algorithm: An Optimum Tree Search Method for Scored Dependency Graph with Arc Co-Occurrence Constraints (ACL 2006)
  - Hideki Hirakawa
  - [Paper]

## 2005

- Game-Tree Search with Combinatorially Large Belief States (IJCAI 2005)
  - Austin Parker, Dana S. Nau, V. S. Subrahmanian
  - [Paper]

## 2003

- Solving Finite Domain Constraint Hierarchies by Local Consistency and Tree Search (IJCAI 2003)
  - Stefano Bistarelli, Philippe Codognet, Kin Chuen Hui, Jimmy Ho-Man Lee
  - [Paper]

## 2001

- Incomplete Tree Search using Adaptive Probing (IJCAI 2001)
  - Wheeler Ruml
  - [Paper]

## 1998

- KnightCap: A Chess Programm That Learns by Combining TD with Game-Tree Search (ICML 1998)
  - Jonathan Baxter, Andrew Tridgell, Lex Weaver
  - [Paper]

## 1988

- A Tree Search Algorithm for Target Detection in Image Sequences (CVPR 1988)
  - Steven D. Blostein, Thomas S. Huang
  - [Paper]

License

- CC0 Universal

montecarlotreesearchpapers.md Github