Słownik fleksyjny został opracowany we współpracy z Katedrą Informatyki AGH (Grupa Lingwistyki Komputerowej) i jest od kilkunastu lat używany w zajęciach dydaktycznych przeznaczonych dla studentów Informatyki na AGH. Słownik fleksyjny jest także narzędziem stosowanym w zespołowych pracach badawczych, ostatnio w projekcie INDECT – 7 Program Ramowy.

Polish word representation

Inflection of primary word

Polish is a highly inflected language. Each primary word has a number of inflectional forms: verbs have 47 (if we exclude participles),  adjectives 44, numerals up to 49, nouns and pronouns 14, and adverbs 3. These figures, and the fact that many words have irregular stem alternations, show that  Polish inflection presents real problems for the computational linguist.

If we ask how to inflect properly the Polish word, eg. personal masculine noun aktor (‘actor’):

the grammarian’s answer is that one must first learn Polish lexical grammar and then apply that grammar to particular lexical items

But, in fact, if one wants to inflect a particular word properly, one must first select the proper inflection ending, eg. -0, -a, -o, -e, -e, -i or -y  to form Masc. Pers. Nom. Sing., -a, -e, or -ego to form Gen. Sing., -owi, -u, or -emu to form Dat. Sing., etc. and then must apply the proper stem alternation rule.

As we can see, the inflectional stem of the word aktor changes from aktor- to aktorz- before ending -e and -y. which is the result of the palatalization process. Let’s compare behaviour of the final stem consonant before ending -y, which can occur in Nom. Sing. and Nom. Plur. of nouns, eg. aktor-0 : aktorz-y, senior-0 : seniorz-y,  amor -0: amor-y, gbur-0 : gbur-y,  traktor-0 : traktor-y and adjectives, eg.  któr-y : którz-y and  stary : starz-y.  It is clear  that  the global phonological rule which says that a front vowel causes consonant palatalization is not appropriate; here, as in aktorz-y, seniorz-y the palatalization takes place before -y in Nom. Plur., but cf. the co-existence of amor-y, gbur-y, traktor-y and któr-y, star-y alongside którz-y, starz-y respectively Nom.Sing. and Plur. of adjective. This shows that there is a need for a new approach to the stem alternation process.

The data show that the belief that it is possible to develop efficient stemming algorithm for Polish seems to be naïve one. We argue, that if one wants to create algorithm, which recognize a particular word properly, one must store all inflection forms in the dictionary – word by word.

Polish Inflection Dictionary

There exist the dictionary of 120.000 primary words with more than 3.300.000 inflection forms,  see.

W. Lubaszewski, H. Wróbel, M. Gajęcki, B. Moskal, A. Orzechowska, P. Pietras, P. Pisarek, T. Rokicka,  Słownik fleksyjny języka polskiego, Kraków 2001.

The dictionary was tested in academic education for more that 6 years (Natural Languge Processing courses at AGH), and 5 years in commercial legal information system LexPolonica, see www.lexpolonica.pl

The dictionary exits:

• in text form
• as lexical grammar
• as C/C++ library

Parts of speech

• A – noun
  • AA – male personal
  • AB – male animate
  • AC – male inanimate
  • AD – female
  • AE – neutral
  • AF – plurale tantum personal
  • AG – plurale tantum impersonal
  • AH – uninflected
• B – verb
  • BA – conjugation ę/esz
  • BB – conjugation ę/isz
  • BC – conjugation ę/ysz
  • BD – conjugation m/sz
    • P – perfective
    • I – imperfective
    • I/P – neutral
  • BE – verb ‘be’
  • C – adjective
    • AA – y/y
    • AB – y/i
    • AC – i/y
    • AD – 0/i
  • D – numeral
    • DA – multi gender
    • DB – dual gender
    • DC – single gender
    • DD – gender free
    • DE – indefinite
    • DF – uninflected
  • E – pronoun
    • EA – male
    • EB – female
    • EC – neutral
    • ED – gender free
  • F – adverb
    • S – synthetic grade dorm
    • A – analytic grade form
    • F – grade free
  • G – uninflected
    • HA – conjunction
    • HB – preposition
    • HC – particle
    • HD – operator
    • HE – predicative

The dictionary in text format

AAADAA: abonent:abonenta:abonentowi:abonenta:abonentem:abonencie:abonencie:abonenci:abonentów:abonentom:abonentów:abonentami:abonentach:abonenci:abonentka

 ADAB: abonentka:abonentki:abonentce:abonentkę:abonentką:abonentce:abonentko:abonentki:abonentek:abonentkom:abonentki:abonentkami:abonentkach:abonentki

BAAA: abonować:abonuję:abonujesz:abonuje:abonujemy:abonujecie:abonują:abonuj:##:abonujmy:abonujcie:##:abonując:abonujący:abonowałem:abonowałeś:abonował:abonowałam:abonowałaś:abonowała:abonowałom:abonowałoś:abonowało:abonowaliśmy:abonowaliście:abonowali:abonowałyśmy:abonowałyście:abonowały:abonowałbym:abonowałbyś:abonowałby:abonowałabym:abonowałabyś:abonowałyby:abonowałobym:abonowałobyś:abonowałoby:abonowalibyśmy:abonowalibyście:abonowaliby:abonowałybyśmy:abonowałybyście:abonowałyby:abonowano:abonowany:abonowawszy:abonowanie

AEBAA: abonowanie:abonowania:abonowaniu:abonowanie:abonowaniem:abonowaniu:abonowanie:abonowania:abonowań:abonowaniom:abonowania:abonowaniami:abonowaniach:abonowania:abonowany

CAB: abonowany:abonowanego:abonowanemu:abonowanego:abonowanym:abonowanym:abonowany:abonowany:abonowanego:abonowanemu:abonowany:abonowanym:abonowanym:abonowany:abonowana:abonowanej:abonowanej:abonowaną:abonowaną:abonowanej:abonowana:abonowane:abonowanego:abonowanemu:abonowane:abonowanym:abonowanym:abonowane:abonowani:abonowanych:abonowanym:abonowanych:abonowanymi:abonowanych:abonowani:abonowane:abonowanych:abonowanym:abonowane:abonowanymi:abonowanych:abonowane:abonowanszy:abonowanszy:abonując

CAA: abonujący:abonującego:abonującemu:abonującego:abonującym:abonującym:abonujący:abonujący:abonującego:abonującemu:abonujący:abonującym:abonującym:abonujący:abonująca:abonującej:abonującej:abonującą:abonującą:abonującej:abonująca:abonujące:abonującego:abonującemu:abonujące:abonującym:abonującym:abonujące:abonujący:abonujących:abonującym:abonujących:abonującymi:abonujących:abonujący:abonujące:abonujących:abonującym:abonujące:abonującymi:abonujących:abonujące:abonujący:abonujący

The lexical grammar – dictionary generating system

The lexical grammar of Polish should be recognized as the composition of pattern recognition rules and local grammars. This makes it possible to use our grammar in processing  of any Polish word, even a potetntial one.

The inflectional pattern

Inflectional pattern is the key concept. We shall define it in terms of three related elements:

F – which is the unique set of inflectional endings, unique in that it differs from the rest of the F’s at least by one ending, eg. {0, a, owi, a, em, e, e, y, ów, om, ów, ami, ach, y}

O – which is the set of descriptions, one description for one ending, eg. {Nom.Sing., Gen.Sing., Dat.Sing., Acc.Sing. Instr.Sing. Loc.Sing. Voc.Sing., Nom.Plur., Gen.Plur., Dat.Plur., Acc.Plur. Instr.Plur. Loc.Plur. Voc.Plur.},  

T –  which is the set of stem alternation, eg. {á# : #ñ, ár : rzñ, át : ciñ, ás : siñ, … }, where # : #  stands for the epty alternation.

Our example forms the inflectional pattern appropriate for  masculine personal nouns like aktor, senior, drapichrust, obdartus:

We argue that this it is easy to transform such pattern into the local grammar, which uses two-level like bidirectional rules, see. local  grammars for Polish adjective p. 62- 64.

Pattern Recognition 

We shall present the set of possible patterns in the form of a labelled tree, in which each node subordinate to the root selects O and each terminal node selects F. Both O and F select T.

                                           __________|______________

                                      A|                           |B                 |C

                                   ____|____________

                               A|               |B             |C

                        _____|__

                   A |               |B

                                ___|____

                             A|              |B

That is to say that if the node labelled A takes the value „noun”, it will select O = {Nom.Sing., Gen.Sing., Dat.Sing., … ,Voc.Plur.}. If then AA takes the value „masculine-personal”, AAB takes the value „nominative singular = 0”, and AABA takes the value „nominative plural = y” then the terminal node AABA selects F = {0, a, owi, a, em, e, e, y, ôw, om, ôw, ami, ach, y}. Both O and F select  the T, eg. {<# : #>, <r : rz>, <t : ci>, <s : si>, …}. The path AABA selects the pattern  proper, for example, for the word aktor.

                It is clear that if one wants to build a pattern recognition rule one must first find the equivalence between a word-form orthographic shape and the proper path. We argue that it is possible to find in the word-form the right-bounded segment, which  points to the path, eg. ¹cy  = {Adj., Mom.Sing.-y, Nom.Plur.-y}. If we label {Adj., Mom.Sing.-y, Nom.Plur.-y} as CAA we can build the pattern recognition rule: ¹cy => CAA.

C/C++ library function list

void clp_init()                                – library initialization;

void clp_info(char *out)                       – library info;

void clp_rec(char *inp, int *out, int *num)    – word ID;

void clp_bform(int id, char *wy)               – base word form;

void clp_forms(int id, char *wy)               – all word forms;

int clp_class(int id)                          – part of speech;

void clp_label(int id, char *wy)               – full description;

void clp_form(int id, int nr, char *wy)        – text form description;

void clp_vec(int id, char *we, char *wy)       – syncretic forms;

Other problems

Multiple forms for one meaning

In general Polish verb creates form system to express particular meaning. This means:

– most verbs have imperfective and perfective form, eg. verb drink  has imperfective form pić and perfective wypić, each of which consist of 47 inflection forms to express person, gender, time, mood and infinitive, participle and impersonal form. Unfortunately  we can not associate perfective form with particular prefix,

– some verbs has in addition the iterative form, eg. verb write: pisać (imperfective), napisać (perfective and pisywać (iterative),

– many verbs have reflexive form, eg, verb wash: myć and reflexive myć się. Unfortunaltely there is not a rule that się forms reflexive verb form, eg. upić ‘drink a bit’ and upić się ‘get drunk’,

– every verb has at least one inflected participle, which inflects like adjective, ie. consists of 44 inflection forms.

We argue, that if one wants to recognize particular verb properly one must link multiple verb forms in the dictionary.

Multipart-word inflection

Multi-part words are composed of primary words, each of which can inflect. But in fact some components can inflect fully and some can reduce their inflection to a single form. And there is no simple way to recognize that in panna młoda ‘bride’ both components are fully inflected, but in pies ogrodnika ‘a person who prevents others from having things that they do not need themselves’ fully inflected is only first component.

We argue, that if one wants to recognize particular multipart-word properly one must introduce to the dictionary form of multipart-word and has to mark fully inflected components.

Disambiguation

The dictionary of Polish which consists of 120 000 primary words contains nor than 3.300.000 inflection forms, 80.000 of which belongs to more then one word, and each such form in the text can represent more than one meaning. Therefore one must develop the meaning description, which provide valuable data for the form disambiguation algorithm. Fortunately, to develop reasonable meaning description one can use language-free technique, which was elaborated by structural semantics, ie. one can describe the meaning of the word by use of semantic relations which exist between described meaning and meanings of related words. There are two relation types. The first show hierarchic dependencies between meanings, eg. dog vs. terrier, spaniel etc., which were for example  applied in WordNet. The second describe semantics connections in the text – if connections are proper the text is meaningful, eg. Captain and the first lady entered the deck, if not the text is senseless, eg. Captain drank the deck. 

We argue, that if one wants to disambiguate word form in the text one must introduce to the meaning description stored in the dictionary all proper semantic connections for each word with ambiguous form.