CTIT University of Twente
Research Business & Innovation About CTIT Research Calls Looking for a job? Intranet

Multi-armed bandit problems with underlying discrete choice models

Project Number: 639.031.448
Project Manager: Dr. A.V. den Boer
Faculty of Electrical Engineering, Mathematics and Computer Science
Tel.: +31-53-4893461
Email: a.v.denboer@utwente.nl


A fundamental question in the Information Age is how data - which nowadays is often abundantly available - can guide decision makers in complex optimization problems. These are often dynamic problems, where decisions are not only based on data, but also generate new data themselves which influences future decisions. The standard framework to model and analyze such data-driven optimization problems is the so-called multi-armed bandit (MAB), which is subject of extensive research and finds application in areas ranging from finance and business-optimization to clinical trials and public health policy.

Many of these decision problems under uncertainty are determined by human choices: choices about travel modes, products to buy, websites to visit, et cetera. The theory of discrete choice provides several models that can estimate and predict such human choice behavior. Incorporating these models in the MAB framework offers a huge potential, because it makes several decision problems tractable that otherwise would be computationally untractable.

Despite these benefits, MAB problems in conjunction with choice models have hardly been considered thus far. The reason is that the statistical aspects of choice models are not well understood. In particular, it is not clear how the finite-sample behavior of statistical estimates is affected by the alternatives (and their attributes) from which people choose. Such knowledge is essential for successfully designing and analysing decision policies in MAB problems.

Project duration: 1-11-2014 / 1-11-2017
Project budget: 250 k-€ funding
Number of person/months: 36 person months
Involved groups: Discrete Mathematics and Mathematical Programming (DMMP)