Abstract:
Spelling correction is a crucial task in natural language processing, enabling
improved understanding and processing of text by correcting phonetic errors, context
errors and typographical errors. The proposed system uses Noisy Channel Model that
provides the probabilistic framework for spelling correction by treating the process of
typing as a transmission over a noisy channel in which the intended word is distorted.
According to the Bayesian inferences, this model consists of two main components: the
language model, which captures the prior probability of word sequences, and the error
model, which estimates the probability of spelling errors given the intended word. The
data are extracted from UCSY Monolingual Corpus. The Spell-error Corpus is
manually created for training the error model. The language model is trained, using the
Spelling Correction Corpus, which is manually segmented and corrected from the
UCSY Monolingual Corpus. Damerau-Levenshtein Distance Algorithm is used for
candidate generation. Experimental results demonstrate that the system performs
effectively on both closed and open test sets. On the closed test set, the system achieves
high accuracy of over 95% across all error categories and the values of precision, recall
and F1-score are also strong. On the other hand, it maintains robust performance with
accuracy exceeding 86% and the values of precision, recall and F1-score are slightly
lower on the open test set. These results underscore the system adaptability and
precision in addressing diverse error types, positioning it as a transformative tool in the
field of Myanmar language processing. The proposed system was developed entirely in
Python programming language for further advancements in automated text correction.