Omorfi–Open morphology of Finnish
Omorfi is a free and open source project containing various tools and data for natural language processing of Finnish based on knowledge driven paradigm. The main components of this repository are:
- a lexical database containing hundreds of thousands of words (c.f. lexical statistics)
- a collection of scripts to convert lexical database into formats used by upstream NLP tools (c.f. lexical processing)
- an autotools setup to build and install (or package, or deploy): the scripts, the database, and simple APIs / convenience processing tools
- a collection of relatively simple APIs with bindings for a selection of programming languages and scripts to apply the NLP tools and access the database
The formats we produce are (links to free open source implementations included):
- lexc, as processed by HFST and foma, to be used for morphological analysis, stemming, segmentation, natural language generation, hyphenation and as a basis for language models,
- apertium, to be used for machine translation
- voikko, to be used for spell-checking and correction
- kotus-sanalista, lexical markup framework, tab-separated values, etc. for long and short term storage, intermediate formats.
The main point of up-to-date documentation is these webpages. You should find list of all pages on the left.
Everyone should read at least versioning information and readme:
If you wish to use omorfi in a serious application you probably found out from the README that a python or java API is the way to go:
There’s a bunch of semi-automatically generated statistics from each version of omorfi:
- Lexical statistics
- Coverage tests
- Missing word-forms by corpora
- Faithfulness tests
- Automata sizes
The design principles of morphological analysis have been changed a dozen of times to accommodate various applications:
More internal documentations:
And more… Perhaps see the automatically generated link list
Automatically generated stuff
Eventually some more documentation will be neatly generated by github and gh-pages.
- Words, particularly those that are problematic (a FAQ for word entries, in a way)
- Paradigms, i.e. inflection patterns
- Internal keys and codes
If you want to discuss about omorfi in Finnish or English, the IRC channels #omorfi and #hfst on Freenode are available for immediate chats (Freenode webchat here). The google group discussion list email@example.com (Google groups web interface here) can also be used, it may require subscription but is very low volume. Suggestions, bug reports, corrections and new lexical data can be sent using github’s omorfi issue tracker.
If, for some reason, you wish to discuss in private, authors’ private emails can be used as contact, but prefer public chats for general usage etc., questions as the archive of frequently asked questions will surely benefit everyone. For bug reports use the issue functionality on this site, or even pull requests.
Alternatives of omorfi
If omorfi doesn’t suit your needs, you may want to try other similar products: suomi-malaga of voikko fame is another morphological analyser of Finnish. Grammatical Framework also has NLP components for Finnish, and it’s written in haskell.
If you want to use commercial products, there are surely some available somewhere.