Optimizing cloud-based machine learning models for low-latency applications

Tom Nelson Tembo

Bakalářská práce

Abstract:

Cloud-based machine learning (ML) models play a critical role in modern applications, ranging from real-time financial trading to autonomous systems and interactive web services. However, latency remains a significant challenge, particularly for applications requiring near-instantaneous response times. This thesis explores strategies for optimizing cloud-based ML models to achieve low-latency performance, ensuring efficient and responsive deployments across diverse environments. The practical evaluation of the thesis focuses on benchmarking an MNIST classification models performance across different deployment configurations. By measuring runtime latency and throughput, we assess the impact of optimization techniques such as pruning and quantization on computational efficiency. Runtime latency isolates inference performance, while request latency provides a comprehensive view of the model’s end-to-end processing time. These measurements help identify bottlenecks and evaluate the trade-offs between efficiency and accuracy, guiding improvements for real-world deployment.

Keywords

optimizing, real-world datasets, cloud-based machine learning models, performance metrics, cloud platforms, latency, neural-networks, monitoring, inference

Jazyk práce: angličtina

Datum vytvoření / odevzdání či podání práce: 26. 3. 2025

Obhajoba závěrečné práce

• Obhajoba proběhla 23. 5. 2025
• Vedoucí: Ing. Miroslav Brabec
• Oponent: Václav Lohr, Ph.D.

Česká zemědělská univerzita v Praze

Provozně ekonomická fakulta

Bakalářský studijní program:
Informatics