개발 일기

CNN (Convolutional Neural Network) Model Khaiii used the CNN Model as an algorithm for machine learning. Syllable-Based model When it comes to morphological analysis of Korean, the analysis result often displays a different form and length than the input due to syllable restoration, irregular eogan (parts of word like ‘빌’ of ‘빌리다’) and eomi (parts of word that may change like ‘리다’, ‘렸다’ of ‘빌리다’..

Analysis Error Patch There can be errors in any machine learning model’s analysis. There cannot be a 100% accurate morpheme analyzer for any input. The analysis error patch is a user dictionary which can correct the model’s analysis errors. Pre- Analysis Dictionary vs Analysis Error Patch The difference between the pre-analysis dictionary and the Analysis Error Patch is written below. Pre- Analy..

CNN Model Training Process Learning Corpus Khaiii used the Sejong Corpus for the model’s learning. The corpus format is as seen below. BTAA0001-00000001 1993/06/08 1993/SN + //SP + 06/SN + //SP + 08/SN BTAA0001-00000002 19 19/SN BTAA0001-00000012 프랑스의 프랑스/NNP + 의/JKG BTAA0001-00000013 세계적인 세계/NNG + 적/XSN + 이/VCP + ㄴ/ETM BTAA0001-00000014 의상 의상/NNG BTAA0001-00000015 디자이너 디자이너/NNG BTAA0001-0000001..

Test for Specialized Spacing Error Model.** Overfitting: If a model is overfitted, it would have a high accuracy to the training data, but it does not work properly on the verification or test data. An overfitted model struggles to evaluate new data. This means the model has been adapted too closely to the training data and even learned its noise. A user often forgets to put a space (especially ..

This post contains key terms that appear frequently in Korean morphological analysis and natural language processing (NLP). It encompasses concepts that are used in Khaiii project and will be helpful for those are not familiar with the Korean structure. I would recommend to review this glossary first before reading the translations. Terms (1) Korean Unit A. Jaso: a Korean Character ( e.g. ㄱ,ㄴ,ㄷ,..

Words (1) Define new or unfamiliar terms • If the term already exists, link to a good existing explanation. • If your document is introducing the term, define the term. If your document is introducing many terms, collect the definitions into a glossary. (2) Use Acronyms Properly On the initial use of an unfamiliar acronym within a document or a section, spell out the full term, and then put the ..

Are you new to technical writing? If you feel lost and try to figure out how to get started, I highly recommend to take Google's Technical Writing course. What is this course about? Google's Technical Writing is a 3 hour online course developed by Google team. The course teaches you how to write clearer technical documentation. IT engineers are the first target audience for this course, but it i..

Pre-analyzed Dictionary Pre-analyzed dictionary is used when a word’s analysis shows a consistent result regardless of its context. Type of Dictionary Entry There are two types of pre-analyzed dictionaries. Exact Match: When two words are fully matched Partial Match: When two words have the same beginning in common but are not exact matches Below are the examples of pre-analyzed dictionaries Num..