Abstract:
Nowadays web pages are implemented in various kinds of languages on Web and web crawlers
are important for search engine. Language specific crawlers are crawlers that traverse and
collect the relative web pages using the successive URls of web page. There is very little
research area in crawling for Myanmar Language web sites. Most of the language specific
crawlers are based on n-gram character sequences which require training documents, the
proposed crawler differ from those crawlers. The proposed system focused on only part of
crawler to search and retrieve Myanmar web pages for Myanmar Language search engine. The
proposed crawler detects the Myanmar character and rule based syllable threshold is used to
judgment the relevant of the pages. According to experimental results, the proposed crawler has
better performance, achieves successful accuracy and storage space for search engines are
lesser since it only crawls the relevant documents for Myanmar web sites.
