Pretraining and Evaluation of Czech ALBERT Language Model – Petr Zelina
Petr Zelina
Bachelor's thesis
Pretraining and Evaluation of Czech ALBERT Language Model
Pretraining and Evaluation of Czech ALBERT Language Model
Abstract:
Tato práce se zabývá novým jazykovým modelem nazvaným ALBERT, který byl vydaný společností Google Research v roce 2019. Architektura ALBERT je velmi úspěšná ve zpracování anglických jazykových problémů a tato práce testuje možnosti jejího využití pro český jazyk. Práce nejdříve rozebírá vývoj a strukturu modelu ALBERT. Dále se zabývá návrhem a trénováním několika českých modelů. Nakonec porovnává jejich …moreAbstract:
This thesis explores a new language model called ALBERT, released by Google Research in 2019. The ALBERT architecture has been very successful in English NLP tasks, and this thesis tests its applicability for the Czech language. The work gives an overview of the models leading to ALBERT and their design. Further, the creation process of several Czech ALBERT models is described. Finally, their performance …more
Language used: English
Date on which the thesis was submitted / produced: 26. 5. 2020
Identifier:
https://is.muni.cz/th/t946b/
Thesis defence
- Date of defence: 24. 6. 2020
- Supervisor: doc. RNDr. Aleš Horák, Ph.D.
- Reader: RNDr. Zuzana Nevěřilová, Ph.D.
Citation record
Full text of thesis
Contents of on-line thesis archive
Published in Theses:- světu
Other ways of accessing the text
Institution archiving the thesis and making it accessible: Masarykova univerzita, Fakulta informatikyMasaryk University
Faculty of InformaticsBachelor programme / field:
Informatics / Artificial Intelligence and Natural Language Processing
Theses on a related topic
-
Visualization of Digital Pathology Images and Results of Their Analyses Using Deep Neural Networks
Nikoleta Češeková -
Interpretation techniques for deep neural networks in digital histopathology
Martin Krebs -
Urban Change Monitoring with Neural Networks and Deep-Temporal Remote Sensing Data
Georg Zitzlsberger -
Image super sampling using deep neural networks
Ivan Gorbatenko -
Alzheimer's dementia recognition from spontaneous speech using deep neural networks
Mariia Buntovskikh -
Detection of IoT Cyberattacks in Smart Cities using Deep Neural Networks
Zeru Kifle Kebede -
Text classification of customer inquiries via e-mail
Armin WEIGOLD -
Czech Question Answer Selection using Recurrent Neural Networks
Radoslav Sabol