Vacant taxi routing in Markov Decision Process (MDP)

Bc. Nurbulat Shektbayev

Master's thesis

Abstract:

This thesis focuses on optimizing taxi routes by using the Markov Decision Processes (MDP) with an aim to enhance taxi driver performance and profitability. It begins with a literature review on reinforcement learning and the MDP to establish a good theoretical background and understanding. This is then followed by defining the optimization problem within the MDP framework. To construct the model, state (each possible pickup and drop-off location with the city) and action (moving to each possible location) spaces are developed by using and processing the yellow taxi data for 2022 year from the New York City Open Data website. Once actions and states were defined, the research continued with building transition probabilities (the probabilities of moving from one state to another state) and reward (reward or benefit which the taxi drive gets by moving from one location to another by taking certain action) matrices and setting discount factor (a rate at which taxi would desire future or immediate reward). After construction of the model, the study then implements and tests four reinforcement learning algorithms such as: Value Iteration, Policy Iteration, Q-learning and SARSA in Python code. These algorithms are used to develop optimal policies for taxi drivers based on the MDP model constructed earlier. Then reward matrix was adjusted to fit more real-life scenario and each of the algorithm was implemented again to examine possible adjustments. After that each of the optimal policies was evaluated, visualized and discussed. At the end of thesis, it switches to possible limitations and obstacles related to data, business and model itself as well as future implementations of the performed work and further possible improvements.

Keywords

MDP, machine learning, data processing, algorithms, markov decision process, dynamic programming, data

Language used: English

Date on which the thesis was submitted / produced: 2. 12. 2023

Thesis defence

• Date of defence: 8. 2. 2024
• Supervisor: Ing. Robert Hlavatý, Ph.D.
• Reader: Martin Horák, externi

Czech University of Life Sciences Prague

Faculty of Economics and Management

Master programme / field:
Systems Engineering and Informatics / Informatics