Looking for:
Windows 10 1703 download iso itar regulations synonyme - |
Windows 10 1703 download iso itar regulations synonyme. ONE OF THE LARGEST LAND WARFARE SHOWS IN THE WORLD…
StackOverFlowTagPredictor/ at master · ChokshiUtsav/StackOverFlowTagPredictor · GitHub.
The problem of the unit of measurement. In other words: There can be no a priori decision as to what a word is, or in what units word length can be measured. Meanwhile, in contemporary theories of science, linguistics is no exception to the rule: there is hardly any science which would not acknowledge, to one degree or another, that it has to define its object, first, and that constructive processes are at work in doing so.
The relevant thing here is that measuring is made possible, as an important thing in the construction of theory. What has not yet been studied is whether there are particular dependencies between the results obtained on the basis of different measurement units; it goes without saying that, if they exist, they are highly likely to be language- specific. Also, it should be noted that this problem does not only concern the unit of measurement, but also the object under study: the word.
It is not even the problem of compound words, abbreviation and acronyms, or numbers and digits, which comes into play here, or the distinction between word forms and lexemes lemmas — rather it is the decision whether a word is to be defined on a graphemic, orthographic-graphemic, or phonological level.
The population problem. Again, as to these questions, there are hardly any systematic studies which would aim at a comparison of results obtained on an empirical basis. However, there are some dozens of different types of letters, which can be proven to follow different rules, and which even more clearly differ from other text types. The goodness-of-fit problem. Rather, the question is, what is a small text, and where does a large text start?
History and Methodology of Word Length Studies 75 d. The problem of the interrelationship of linguistic properties. What they have in mind are in- tralinguistic factors which concern the synergetic organization of language, and thus the interrelationship between word length factors such as size of the dictionary, or the phoneme inventory of the given language, word frequency, or sentence length in a given text to name but a few examples.
As soon as the interest shifts from language, as a more or less abstract system, to the object of some real, fictitious, imagined, or even virtual communicative act, between some producer and some recipient, we are not concerned with language, any more, but with text. Consequently, there are more factors to be taken into account forming the boundary conditions, factors such as author- specific, or genre-dependent conditions.
Ultimately, we are on the borderline here, between quantitative linguistics and quantitative text analysis, and the additional factors are, indeed, more language-related than intralinguistic in the strict sense of the word. It should be mentioned, however, that very little is known about such factors, and systematic work on this problem has only just begun. The modelling problem. As can be seen, the aim may be different with regard to the particular research object, and it may change from case to case; what is of crucial relevance, then, is rather the question of interpretability and explanation of data and their theoretical modelling.
The problem of explanation. Consequently, in order to obtain an explanation of the nature of word length, one must discover the mechanism generating it, hereby taking into account the necessary boundary conditions. Thus far, we cannot directly concentrate on the study of particular boundary conditions, since we do not know enough about the general system mechanism at work.
Consequently, contemporary research involves three different kinds of orientation: first, we have many bottom-up oriented, partly in the form of ad-hoc solutions for particular problems, partly in the form of inductive research; second, we have top-down oriented, deductive research, aiming at the formulation of general laws and models; and finally, we have much exploratory work, which may be called abductive by nature, since it is characterized by constant hypothesis testing, possibly resulting in a modification of higher-level hypotheses.
In this framework, it is not necessary to know the probabilities of all individual frequency classes; rather, it is sufficient to know the relative difference between two neighboring classes, e. Ultimately, this line of research has in fact provided the most important research impulses in the s, which shall be discussed in detail below.
In their search for relevant regularities in the organization of word length, Wimmer et al. Wimmer et al. This model was already discussed above, in its 1-displaced form 2. It has also been found to be an adequate model for word length frequencies from a Slovenian frequency dictionary Grzybek After corresponding re-parametrizations, these modifications result in well-known distribution models.
In , Wimmer et al. The set of word length classes is organized as a whole, i. Now, different distributions may be inserted for j. Thus, inserting the Borel distribution cf. The parameters a and b of the GPD are independent of each other; there are a number of theoretical restrictions for them, which need not be discussed here in detail cf. Irrespective of these restrictions, already Wimmer et al. These observations are supported by recent studies in which Stadlober analyzed this distribution in detail and tested its adequacy for linguistic data.
Stadlober As can be seen, the results are good or even excellent in all cases; in fact, as opposed to all other distributions discussed above, the Consul-Jain GPD is able to model all data samples given by Fucks.
It can also be seen from Table 2. In this respect, i. As to this problem, it seems however important to state that this is not a problem specifically related to the GPD; rather, any mixture of distributions will cause the very same problems.
In this respect, it is important that other distributions which imply no mixtures can also be derived from 2. It would go beyond the frame of the present article to discuss the various extensions and modifications in detail here. As a result, there seems to be increasing reason to assume that there is in- deed no unique overall distribution which might cover all linguistic phenom- ena; rather, different distributions may be adequate with regard to the material studied. This assumption has been corroborated by a lot of empirical work on word length studies from the second half of the s onwards.
Best More often than not, the relevant analyses have been made with specialized software, usually the Altmann Fitter. This is an interactive computer pro- gram for fitting theoretical univariate discrete probability functions to empirical frequency distributions; fitting starts with the common point estimates and is optimized by way of iterative procedures.
There can be no doubt about the merits of such a program. Now, the door is open for inductive research, too, and the danger of arriving at ad-hoc solutions is more virulent than ever before. What is important, therefore, at present, is an abductive approach which, on the one hand, has theory-driven hypotheses at its background, but which is open for empirical findings which might make it necessary to modify the theoretical assumptions.
In addition to the C values of the discrepancy coefficient, the values for parameters a and b as a result of the fitting are given. As can be seen, fitting results are really good in all cases. As to the data analyzed, at least, the hyper-Poisson distribution should be taken into account as an alternative model, in addition to the GDP, suggested by Stadlober Comparing these two models, a great advantage of the GPD is the fact that its reference value can be very easily calculated — this is not so convenient in the case of the hyper-Poisson distribution.
On the other hand, the generation of the hyper-Poisson distribution does not involve any secondary distribution to come into play; rather, it can be directly derived from equation 2. In its 1-displaced form, equation 2. To summarize, we can thus state that the synergetic approach as developed by Wimmer et al. Generally speaking, the authors understand their contribution to be a logical extension of their synergetic approach, unifying previous assumptions and empirical findings.
The individual hypotheses belonging to the proposed system have been set up earlier; they are well-known from empirical research of the last decades, and they are partly derived from different approaches. Specifically, Wimmer et al. History and Methodology of Word Length Studies 85 it is confined to the first four terms of formula 2.
Many distributions can be derived from 2. It can thus be said that the general theoretical assumptions implied in the synergetic approach has experienced strong empirical support. One may object that this is only one of possible alternative models, only one theory among others.
However, thus far, we do not have any other, which is as theoretically sound, and as empirically supported, as the one presented. On the other hand, hardly any systematic studies have been undertaken to empirically study pos- sible influencing factors, neither as to the data basis in general i. Ultimately, the question, what may influence word length frequencies, may be a bottomless pit — after all, any text production is an historically unique event, the boundary conditions of which may never be reproduced, at least not completely.
Still, the question remains open if particular factors may be detected, the relevance of which for the distribution of word length frequencies may be proven. This point definitely goes beyond a historical survey of word length studies; rather, it directs our attention to research desires, as a result of the methodolog- ical discussion above.
A, ; — Best, Karl-Heinz ed. Brainerd, Barron Weighing evidence in language and literature: A statistical approach. Chebanow Chebanow, S. Dewey, G. Cambridge; Mass. Elderton, William P. London, Fucks, Wilhelm Nach allen Regeln der Kunst.
Leningrad, Nauka: — Dordrecht, NL. Grzybek, Peter ed. Ljubljana etc. The Impact of Word Length. Kromer, Victor V. Materialy konferencii. Ma- terialy konferencii. Markov, Andrej A. Mendenhall, Thomas C. Studien zum 1. Internationalen Bulgaristikkongress Sofia Piotrovskij, Rajmond G. Williams, Carrington B. Wimmer, Gejza; Altmann, Gabriel Thesaurus of univariate discrete probability distributions.
Zerzwadse, G. In: Grundlagenstudien aus Kybernetik und Geisteswissenschaft 4, — The idea is derived from the Fitts—Garner controversy in mathematical psychology cf. Fitts et al. Obviously, the problem is quite old but has not penetrated into linguistics as yet. A word in a text can be thought of as a realization of a number of different alternative possibilities, see Fig.
They can even be understood in different ways, e. What is neglected when correlating the lengths and the frequencies of words in real texts is the fact that for the text producer there is not at all free choice of all existing words at every moment.
Trying to fill in the blank is a model for determining the uncertainty of the missing word. It must be noted that SIC or HIC are associated not only with words but also with whole phrases or clauses, so that they represent rather polystratic structures and sequences.
The present approach is the first approximation at the word level. Preparation In order to illustrate the variables which will be treated later on, let us first define some quantities. The cardinality of the set X will be symbolized as X. P the set of positions in a text, whatever the counting unit. The elements of this set are tokens tijk , i.
If the type and its token are known, the indices i and j can be left out. The elements of this set, aij , are not necessarily synonyms but in the given context they are admissible alternatives of the given token. The index k can be omitted Aij the number of elements in the set Aij , i. This entity can be called tokeme. By defining Mij , we are able to distinguish between tokens of the same type but with different alternatives and different number a i — so they are different tokemes.
Example Using Table 9 cf. The text is reproduced word for word in the second column of Table 9 p. The length is measured in terms of the number of syllables of the word. Thus, e. We can define it for types too: then it is the mean of all LLs of all tokens of this type in the text. LL is usually a positive real number. The errors compensate each other in the long run, so the distribution of L equals that of LL.
It can be ascertained for any text. We can set up the hypothesis that Hypothesis 1 The longer the token, the longer the tokeme at the given position. This hypothesis can be tested in different ways. As an empirical consequence of hypothesis 1 it can be expected that the distribution of L and LL is approximately equal. A token of length L has alternatives which are on average the same length, i.
Since LL is a positive real number it is an average we divide the range of lengths in the text in continuous intervals and ascertain the number of cases the frequency in the particular intervals.
This can easily be made using the third and the sixth column of Table 9 p. The result is presented in Table 3. It can easily be shown that the frequencies differ non-significantly. Since the distributions are equal, they must abide by the same theoretical distribution. Using the well corroborated theory of word length cf. Wim- mer et al. As a matter of fact, for the distribution of LL we take the middles of the intervals as variable. It would, perhaps, be more correct to use for both data the continuous equivalent of the geometric distribution, namely the exponential distribution — however, again not quite correct.
Thus we adhere to discreteness without loss of validity. The result of fitting the geometric distribution to the data from Table 3. Length range in tokemes In each tokeme the lengths of words local latent lengths are distributed them- selves in a special way.
It is not fertile to study them individually since the majority of them is deterministic i. It is more prolific to consider the ranges of latent lengths for the whole text. For this phenomenon we set up the hypothesis Hypothesis 2 The range of latent lengths within the tokemes is geometric-Poisson.
Since the latent length distribution LLx is geometric and each LLx is al- most identical on average with that of Lx the alternatives tend to keep the length of the token , the range of the latent lengths in the tokeme is very restricted. The deviations seem to be distributed at random, i. Evidently, the fitting is very good and corroborates in addition hypothesis 1, too. Thus latent length is a kind of latent mechanism controlling the token length at the given position.
Latent length is not directly measurable, it is an invisible result of the complex flow of information. Nevertheless, it can be made visible — as we tried to do above — or it can be approximately deduced on the basis of token lengths.
Information Content of Words in Texts 99 Table 3. Stable latent length Consider the deviations of the individual token lengths from those of the re- spective tokeme lengths as shown in Table 9 p.
This encourages us to set up the hypothesis that Hypothesis 3 There is no tendency to choose the smallest possible alternative at the given position in text. The hypothesis can easily be tested. SIC of the text Above, we defined SIC of a type as the dual logarithm of the mean size of all tokeme sizes of the given type, as shown in formula 3.
Two possibilities can be proposed. We shall use here 3. For the given text it can be computed directly using the fifth column of Table 9 p. We suppose that it is the smaller the more formal the text. We can build about it a confidence interval. Here the tokeme sizes build a sequence of a 1, 16, 3, 2, 1, 8, 2, 1,. Taking the dual logarithms we obtain a new sequence b 0, 4, 1.
In order to control the information flow and at the same time to allow licentia poetica, zeros and non-zeroes must display some pattern which is characteristic of different text types. Thus we obtain the two state sequence c 0, 1, 1, 1, 0, 1, 1, 0,. We begin with the examination of runs of 0 and 1 and set up the hypothesis that Hypothesis 4 The building of text blocks with zero uncertainty 0 and those with selection possibilities 1 is random i.
In practice it means that the runs of zeroes and ones are distributed at random. In our text see Table 9 , p. Another possibility is to consider sequence c as a two-state Markov chain or sequences a and b as multi-state Markov chains. In the first approx- imation we consider case c as a dynamical system and compute the transition matrix between zeroes and ones. Taking the powers of the above matrix we can easily see that the probabilities are stable to four decimal places with P 4 yielding a matrix with equal rows [0.
Since P n represents the n-step transition prob- ability matrix, the exponent n is also a characteristic of the text. Alternatives, length and frequency Since SIC has not been imbedded in the network of synergetic linguistics as yet, it is quite natural to ask whether it is somehow associated with basic language properties such as length and frequency.
In the present paper all other properties e. The data for testing can easily be prepared using Table 9 p. Below we show merely lengths 4 and 5 because the full Table is very extensive cf. Table 3. This results in Table 3. In such cases they must be taken into account explicitly.
In our case this leads to partial differential equations. Let us assume that length has a constant effect, i. Fitting this curve to the data in Table 3. This is, of course, merely the first approximation using data smoothing because the text was rather a short one. Interpretation and outlook Looking at Tables 3.
But we recognize that the influence of frequency is considerably weaker than that of length. If we regard 3. The direction of this influence is even more astonishing: with increasing length the number of alternatives is increasing too, longer words are more often freely chosen, while one perhaps would expect a preference for choosing shorter words. Since the e-function plays an important role in psychology, for example in cognitive tasks like decision making, we suppose that word length is a variable which is connected with some basic cognitive psychological processes.
Andersen, S. Attneave, F. New York. Berlyne, D. Coombs, C. Englewood Cliffs, N. Evans, T. Fitts, P. Garner, W. Hartley, R. Piotrowski, R. Wimmer, G. June 21—23, , Graz University. Bahnhof 2 - 1 2. Altona 3 - 1 3. Kinderbuch 3 - 1 3.
Stiftung 2 - 1 2. Deutschland 2 - 1 2. Kinder- 2 - 1 2. Krimi 2 Kriminalroman, Thriller 3 3. Kinderbuchautor 5 Autor, Schriftsteller, Kinderbuchschriftsteller 4 4. Andreas 3 - 1 3. Anfang 2 Beginn, Start 3 1. Jungen 2 - 1 2. Weltuhr 2 Uhr 2 1.
Seiten 2 - 1 2. Aktion 3 Leistung, Tat, Sache 4 2. Guinness 2 - 1 2. Rekorde 3 - 1 3. Altona 3 Hamburg 2 2. Bahnhofshalle 4 Halle, Vorhalle, Wandelhalle 4 3. Szenen 2 Bilder, Teile, Partien 4 2. Detektive 4 - 1 4. Introduction This paper concentrates on the question of zero-syllable words i. As an essential result of these studies it turned out that, due to the specific structure of syllable and word in Slavic languages a several probability distribution models have to be taken into account, and this depends b on the fact if zero-syllable words are considered as a separate word class in its own right or not.
Predominantly putting a particular accent on zero-syllable words, we examine if and how the major statistical measures are influenced by the theoretical definition of the above- mentioned units. We do not, of course, generally neglect the question if and how the choice of an adequate frequency model is modified depending on these pre-conditions — it is simply not pursued in this paper which has a different accent.
Basing our analysis on Slovenian texts, we are mainly concerned with the following two questions: a How can word length reasonably be defined for automatical analyses, and b what influence has the determination of the measure unit i.
Thus, subsequent to the discussion of a , it will be necessary to test how the decision to consider zero-syllable words as a specific word length class in its own right influences the major statistical measures. Any answer to the problem outlined should lead to the solution of specific prob- lems: among others, it should be possible to see to what extent the proportion of x-syllable words can be interpreted as a discriminating factor in text typology — to give but one example.
In a way, the scope of this study may be understood to be more far-reaching, however, insofar as it focuses relevant pre-conditions which are of general methodological importance. For these ends, we will empirically test, on a basis of Slovenian texts, which effects can be observed in dependence of diverging definitions of these units. Word Definition Without a doubt, a consistent definition of the basic linguistic units is of utmost importance for the study of word length.
Zero-syllable Words in Determining Word Length Irrespective of the theoretical problems of defining the word, there can be no doubt that the latter is one of the main formal textual and perceptive units in linguistics, which has to be determined in one way or another. Knowing that there is no uniquely accepted, general definition, which we can accept as a standardized definition and use for our purposes, it seems reasonable to discuss relevant available definitions. As a result, we should then choose one intersubjectively acceptable definition, adequate for dealing with the concrete questions we are pursuing.
Taking into consideration syntactic qualities, and differentiating autosemantic vs. Subsequent to this discussion of three different theoretical definitions, we will try to work with one of these definitions, of which we demand that it is acceptable on an intersubjective level. The decisive criterium in this next step will be a sufficient degree of formalization, allowing for an automatic text processing and analysis.
Rather, what can be realized, is an attempt to show which consequences arise if one makes a decision in favor of one of the described options. Since this, too, cannot be done in detail for all of the above-mentioned alternatives, within the framework of this article, there remains only one reasonable way to go: We will tentatively make a decision for one of the options, and then, in a number of comparative studies, empirically test which consequences result from this decision as compared to the theoretical alternatives.
This will be briefly analyzed in the following work and only in the Slovenian language, but under special circumstances, and with specific modifications. In the previous discussion, we already pointed out the weaknesses of this defini- tion; therefore, we will now have to explain that we regard it to be reasonable to take just the graphematic-orthographic definition as a starting point. It can therefore be expected that the results allow for some intersubjective comparability, at least to a particular degree.
Zero-syllable Words in Determining Word Length b Second, since the definition of the units involves complex problems of quantifying linguistic data, this question can be solved only by way of the assumption that any quantification is some kind of a process which needs to be operationally defined.
The word thus being defined according to purely formal criteria — i. This, in turn, can serve as a guarantee that an analysis on all other levels of language i. The definition chosen above is, of course, of relevance for the automatic pro- cessing and quantitative analysis of text s. In detail, a number of concrete textual modifications result from the above-mentioned definition. In case of single elements, they are processed according to their syllabic structure. Particularly with regard to foreign language elements and passages, attention must be paid to syllabic and non-syllabic elements which, for the two languages under consideration, differ in func- tion: cf.
It should be noted here that irrespective of these secondary manipulations the original text structure remains fully recognizable to a researcher; in other words, the text remains open for further examinations e. Altmann et al. Unuk 3. In order to automatically measure word length it is therefore not primarily necessary to define the syllable boundaries; rather, it is sufficient to determine all those units phonemes which are characterized by an increased sonority and thus have syllabic function.
On the other hand, empirical sonographic studies show that there are no bilabial fricatives in Slovenian standard language cf. Srebot-Rejec Of course, 2 For further discussions on this topic see: Tivadar , Srebot Rejec , Slovenski pravopis ; cf. On the Question of zero-syllabic Words The question whether there is a class of zero-syllabic words in its own right, is of utmost importance for any quantitative study on word length. With regard to this question, two different approaches can be found in the research on the frequency of x-syllabic words.
In this context, it will be important to see if consideration or neglect of this special word class results in statistical differences, and how much information consideration of them offers for quantitative studies.
As can be seen, we are concerned with two zero-syllable prepositions and with corresponding orthographical-graphematic variants for their phonetic realiza- tions. As opposed to this, these prepositions are treated as zero-syllable words in modern Slovenian; they thus exemplify the following general trend: original one-syllable words have been transformed into zero-syllable words.
Obviously, there are economic reasons for this reduction tendency. From a phonematic point of view, one might add the argument that these prepositions do not display any suprasegmental properties, i. Following this diachronic line of thinking might lead one to assume that zero-syllable words should or need not be considered as a specific class in linguo-statistic studies.
Incidently, the depicted trend i. Yet, as was said above, it is not our aim to provide a theo- retical solution to this open question. Nor do we have to make a decision, here, whether zero-syllable words should or should not be treated as a specific class, i. Rather, we will leave this question open and shift our attention to the empirical part of our study, testing what importance such a decision might have for particular statistical models.
Descriptive Statistics The statistical analyses are based on Slovenian texts, which are considered to represent the text corpus of the present study. The whole number of texts is divided into the following groups4 : literary prose, poetry, journalism. The detailed reference for the prose and poetic texts are given in Tables 4. Table 4. Based on these considerations, and taking into account that the text data basis is heterogeneous both with regard to content and text types, statistical measures, such as mean, standard deviation, skewness, kurtosis, etc.
Level I The whole corpus is analyzed under two conditions, once considering zero- syllable words to be a separate class in their own right, and once not doing so. One can thus, for example, calculate relevant statistical measures or analyze the distribution of word length within one of the two corpora.
Level II Corresponding groups of texts in each of the two corpora can be compared to each other: one can, for example, compare the poetic texts, taken as a group, in the corpus with zero-syllable words, with the corresponding text group in corpus without zero-syllable words. Level III Individual texts are compared to each other. Here, one has to distinguish different possibilities: the two texts under consideration may be from one and the same text group, or from different text groups; additionally, they may be part of the corpus with zero-syllable words or the corpus without zero-syllable words.
Level IV An individual text is studied without comparison to any other text. A larger positive skewness implies a right skewed distribution. In the next step, we analyze which percentage of the whole text corpus is represented by x-syllable words. Text no. Three Text Types Figure 4. It should be noted that many poetic texts do not contain any 0-syllable words at all. Of the 51 poetic texts, only 26 contain such words.
Analysis of Mean Word Length in Texts The statistical analysis is carried out twice, once considering the class of zero- syllable words as a separate category, and once considering them to be proclitics. Our aim is to answer the question, whether the influence of the zero-syllable words on the mean word length is significant.
In the next step concentrating on the mean word length value of all texts Level I , two vector variables are introduced, each of them with components: W C 0 and W C. The i-th component of the vector variable W C 0 defines the mean word length of the i-th text including zero-syllable words. In analogy to this, the i-th com- ponent of the vector variable W C gives the mean word length of the i-th text excluding zero-syllable words see Table 4.
In order to obtain a more precise structure of the word length mean values, the analyses will be run both over all texts of the whole corpus Level I , and over the given number of texts belonging to one of the following three text types, only Level II : i literary prose L , ii poetry P , iii journalistic prose J. A scatterplot is a graph which uses a coordinate plane to show the relation correlation between two variables X and Y.
Each point in the scatterplot represents one case of the data set. In such a graph, one can see if the data follow a particular trend: If both variables tend in the same direction that is, if one variable increases as the other increases, or if one variable decreases as the other decreases , the relation is positive.
There is a negative relationship, if one variable increases, whereas the other decreases. The more tightly data points are arranged around a negatively or positively sloped line, the stronger is the relation. If the data points appear to be a cloud, there is no relation between the two variables. In the following graphical representations of Figure 4. In our case, the scatterplot shows a clear positive, linear dependence between mean word length in the texts both with and without zero-syllable words , for each pair of variables.
This result is corroborated by a correlation analysis. W C 0 b Scatterplot W L vs. W P 0 d Scatterplot W J vs. W J 0 Figure 4. As to our data, a strong dependence at the 0. Let us therefore take a look at the histograms of each of the eight new variables. The first pair of histograms cf. Figure 4. Still, we have to test these assumptions. Usually, either the Kolmogorov-Smirnov test or the Shapiro-Wilk test are ap- plied in order to test if the data follow the normal distribution.
Since, in our case, the parameters of the distribution must be estimated from the sample data, we use the Shapiro-Wilk test, instead. This test is specifically designed to detect deviations from normality, without requiring that the mean or variance of the hypothesized normal distribution are specified in advance. To determine whether the null hypothesis of normality has to be rejected, the prob- ability associated with the test statistic i. If this value is less than the chosen level of significance such as 0.
The obtained p-values support our assumptions, i. In the following analyses, we shall focus on the second analytical level, i. In order to test this, we can apply the t-test for paired samples.
This test compares the means of two variables; it computes the difference between the two vari- ables for each case, and tests if the average difference is significantly different from zero. This means that we test the following hypothesis: H0 : There is no significant difference between the theoretical means i. Before applying the t-test, we have to test if the variables d L , dP , dJ are also normally distributed. As they are linear combinations of normally dis- tributed variables, there is sound reason to assume that this is the case.
The Shapiro-Wilk test yields the p-values given in Table 4. The histogram of the variable dP shows the same result cf. In spite of the result of the Shapiro- Wilk test, we therefore apply a one sample t-test assuming that d P is normally distributed.
Two distribution functions for variables which denote mean word length of texts with and without zero-syllable words have the same shape, but they are shifted, since their expected values differ. The following Figures 4. It should be noted that this conclusion can not be generalized. As long as the variables dL , dP , dJ are normally distributed, our statement is true.
Yet, normality has to be tested in advance and we can not generally assume normally distributed variables. In the next step we show the box plots and error bars of the variables d L , dP , dJ. A box plot is a graphical display which shows a measure of location the median-center of the data , a measure of dispersion the interquartile range, i. Horizontal lines are drawn both at the median — the 50th percentile q0.
The horizontal lines are joined by vertical lines to produce the box. A vertical line is drawn up from the upper quartile to the most extreme data point i. The most extreme data point thus is min x n , q0. Short horizontal lines are added in order to mark the ends of these vertical lines. The difference in the mean values of the three samples is obvious; also it can clearly be seen that all three samples produce symmetric distributions, variable dJ displaying the largest variability.
As can be seen, the confidence intervals do not overlap; we can therefore conclude that the percentage of zero-syllable words possibly may allow for a distinction between different text types. It turns out that the number of sylla- bles per word i. This class of words may either be considered to be a separate word-length class in its own right, or as clitics.
Without making an a priori decision as to this question, the mean word length of Slovenian texts is analyzed in the present study, under these two conditions, in order to test the statistical effect of the theoretical decision.
In the present study, the material is analyzed from two perspectives, only: mean word length is calculated both in the whole text corpus Level I , and in three different groups of text types, representing Level II: literary, journalistic, poetic. These empirical analyses are run under two conditions, either including the zero-syllable words as a separate word length class in its own right, or not doing so.
Zero-syllable Words in Determining Word Length Based on these definitions and conditions, the major results of the present study may be summarized as follows: 1 As a first result, the proportion of zero-syllable words turned out to be relatively small i. Furthermore, it can be shown that the mean word length in texts under both conditions are highly correlated with each other; the positive linear trend, which is statistically tested in the form of a correlation analysis and graphically represented in Figure 4.
As a result, it turns out that mean word length is normally distributed in the three text groups analyzed Level II , but, interestingly enough, not in the whole corpus Level II.
Based on this finding, further analyses concentrate on Level II, only. Therefore, t-tests are run, in order to compare the mean lengths between the three groups of texts on the basis of the differences between the mean lengths under both conditions.
As a result, the expected values of mean word length significantly differ between all three groups. To summarize, we thus obtain a further hint at the well-organized structure of word length in texts. Altmann, G. Figge zum Stutt- gart. Bajec, A. Predlogi in predpone. Best, K. Genzor; S. Wimmer; G. Altmann; R. Girzig, P. Grotjahn, R. Grzybek, P. Jachnow, H. Lehfeldt, W. Jachnow ed. Lekomceva, M. Tom 1. Rottmann, Otto A. Royston, P. Schaeder, B.
Srebot-Rejec, T. Tivadar, H. Unuk, D. Doktorska disertacija. Figure 5. Many of the relevant psychological findings seem to be interesting for linguistics as well: the results reported e.
Of particular linguistic interest is the question whether the serial position effects shown in the recall of lists of unconnected words show in the recall of real sentences as well.
Are the underlying processes also efficient in real sentence processing and in connected discourse? Having failed to disprove the charges, Taylor was later fired by the president" p.
The serial position curve reported by Fenk 25 shows a marked recency effect only in auditory presentation of the sentence. But these results originate from only two different sentences presented simultaneously in two different sense modalities.
Subjects were instructed to write down as much as they could remember from the last sentence before the test pause. Nevertheless the family of curves shows a rather weak primacy effect and a marked recency effect.
Data from this experiment were re-analyzed in order to investigate further questions. Wordclass-specific effects on the serial position curve? In brief: The relevant division here is between context-specific content words and rather context-independent function words. Ad b : A widely accepted model concerning our memory says: After having ex- tracted the meaning of an actual clause, its verbatim form words and syntax is rapidly lost from memory, while the meaning is preserved and affects e.
This conception is strongly influ- enced by Sachs On the other hand, recall of previous sentences indicates that they had received a relatively thorough semantic interpretation. The first quarter I was defined as the primacy part of the sentence, II and III taken together as the medium part, and the last 25 percent of the words IV as the recency part.
But the alternative — to define the primacy part and the recency part in terms of a fixed number of words — would again be arbitrary: How many words should be fixed? Our operationalization, however, offers a wide range of applications and es- tablishes a firm proportion between, on the one hand, the primacy and recency part, and, on the other hand, the part in between and the sentence as a whole.
And it has proved to bring about significant results. Thus, a quantification in absolute terms did not make much sense, and the recall scores had to be related to the number of words presented. Table 5. Actually there is, as can be seen from the values in Table 5. But in both cases this convergence is far from significant. Three more or less hypothetical regularities The formulation of the first of the following assumptions is motivated by the occasional observation that our test sentences taken from a Glasersfeld text showed a tendency of an increase of content words and a decrease of function words during a sentence.
Results strongly indicate that this is a general tendency at least in German texts. And if our tentative explanation section 4 of this regularity holds, its scope should not be restricted to German texts. Regularities 3. Regularity 3. This statistical regular- ity has proven to be the most powerful one in the explanation of word order in frozen conjoined expressions Fenk-Oczlon , and it seems that its range of validity can be extended on clauses in general.
In this present paper we will state this generalized rule mainly as an inferential step to our third regularity 3. Despite the small sample of only ten sentences, the relevant differences proved to be significant in the Wilcoxon test Table 5. These differences in the distribution of the instances of the two word classes were, as already mentioned in section 2. A pilot study was conducted in order to find some indications of possible generalizations of this tendency.
The sample of authors was increased — nine more German text passages, four of them from scientific books, five from literary books. Taken to- gether with the already analysed text passages from Glasersfeld this is a sample of ten five scientific, five literary text passages, and a sample of ten sentences from each of these passages, i.
Source texts are listed at the end of the paper. She was instructed not to collect ten successive sentences from each passage into the sample, but — where possible — each third sentence. Sometimes she had to overleap more than two sentences, e.
As already suggested by Niehaus , a colon was accepted as the end of the sentence when the following word started with a capital letter. These results suggest that the tendency of function words to decrease and of content words to increase in the course of a sentence is a general tendency at least in German texts. From all the rules examined e. Our regularity 3.
Behaghel illustrates this law with many examples from classical texts in a variety of languages such as ancient Greek, Latin, Old High German and German. They were instructed to form a sentence from these fragments, and the result was always the same: sie besitzt Gold und edles Geschmeide.
Behaghel Behaghel f. At present we cannot offer results of empirical tests of this lawlike assumption. But we can contribute two new perspectives: 1. An interpretation specifying a concrete factor that might at least contribute to the rhythmic pattern described by Behaghel. This factor is the concentration or accumulation of function words in the first parts of clauses sentences, subordinate clauses. And since function words are generally extremely fre- quent and frequent words tend — for economic reasons — to be rather short Zipf , , the concentration of these rather short units in the first part of clauses results in an increase of the mean word length in the course of a sentence.
This hypothesized tendency will of course depend on the re- spective language type and is expected to be more pronounced in languages with a tendency to agglutinative morphology and a tendency to OV order. This reference is — most probably not only in German texts — first of all brought about by function words e.
If this is an appropriate explanation of our regularity 3. As a consequence, one may expect an increase of word length in the course of a sentence. Frankfurt: Suhrkamp Verlag suhrkamp taschenbuch Das Reich des Zufalls. Konstruktivismus statt Erkenntnistheorie.
In: W. Mitterer eds. Klagenfurt: Drava Verlag. Der Steppenwolf. Doktor Faustus. Frankfurt a. Der Mann ohne Eigenschaften.
In: Best, K. Glot- tometrika 16, Quantitative Linguistics 58, — Unsicheres Wissen. Das Wahrheitsproblem und die Idee der Semantik. Wien: Springer-Verlag.
Behaghel, O. Fenk, A. Fenk-Oczlon, G. Jarvella, R. Luther, P. Murdock, B. Niehaus, B. Sachs, J. Zipf, G. Introduction From the first beginnings in the mids, availability of electronic text corpora in Slovenian, all with an Internet user interface, has grown to a level compara- ble to many European languages with a long history of quantitative linguistic research.
There are two established corpora with million running words, an academic one which is freely accessible and a commercial one, prepared by industrial and academic partners.
The two are complemented by a sizeable collection of works of fiction, available for reading in a free virtual library and several specialized corpora, compiled for the needs of particular institutions. The majority of Slovenian newspapers are also accessible online, at least in the form of selected articles. The basic infrastructure for word-length analysis is in place and in the fol- lowing chapters these topics are discussed in some more detail. Online Text Corpora There are two online text corpora in the narrow sense of this word, each million running words in size and each equipped with an Internet user interface including a concordancer and some other searching facilities.
Other text col- lections have been built with different uses in mind and they complement the Slovenian corpus scene.
Nova beseda was upgraded to 48 million words in September , to 76 million words in October , to 93 million words in April and to million words of text in Slovenian in July The current corpus contents can be classified as: DELO daily newspaper — — All texts have undergone an extensive word form check-up and correction process and so the level of noise is kept to a minimum over 45, errors, mostly typ- ing errors, but also other errors which usually appear during the preparation of electronic publications or its transfer from one format or platform to another, have been detected and corrected.
The corpus web pages are accessed over times a day and an overview of the referring URLs in the first three months of are shown in Table 6. The domain. DZS was also the coordinator and leading partner. Amebis, the main Slovenian en- terprise in the field of language resources, mostly spell-checkers, provides the A in FIDA. Gorjanc , the corpus contains million running words of mostly newspaper text, it went operational in and was completed in the first half of ; the corpus has remained unchanged since that time.
The project, aiming at a reference corpus of modern Slovenian, has been financed by the two commercial partners and so is not freely available. Free use is restricted to 10 concordance lines per search and the number of concurrent free users is also limited; full use requires the signing of a contract which regulates eventual publications based on the use of the FIDA corpus and a yearly fee in the vicinity of e per user.
Words from around 1. An automatic pro- cedure based on n-gram frequencies, is used to identify the page language — it is usually successful after two or three lines of text. The distribution of languages represented in March can be seen in Table 6. Nevertheless, it is an excellent source of new words in Slovenian.
The search engine does not yet include a lemmatizer; a simple stemmer is used instead and it usually performs remarkably well. Slovenian Polish Norwegian 82 2. English Danish Bulgarian 20 3. German Finnish Albanian 18 4. Croatian 4. Czech Korean 17 5. Serbian 2. The individual hypotheses belonging to the proposed system have been set up earlier; they are well-known from empirical research of the last decades, and they are partly derived from different approaches.
Specifically, Wimmer et al. History and Methodology of Word Length Studies 85 it is confined to the first four terms of formula 2. Many distributions can be derived from 2. It can thus be said that the general theoretical assumptions implied in the synergetic approach has experienced strong empirical support. One may object that this is only one of possible alternative models, only one theory among others. However, thus far, we do not have any other, which is as theoretically sound, and as empirically supported, as the one presented.
On the other hand, hardly any systematic studies have been undertaken to empirically study pos- sible influencing factors, neither as to the data basis in general i. Ultimately, the question, what may influence word length frequencies, may be a bottomless pit — after all, any text production is an historically unique event, the boundary conditions of which may never be reproduced, at least not completely. Still, the question remains open if particular factors may be detected, the relevance of which for the distribution of word length frequencies may be proven.
This point definitely goes beyond a historical survey of word length studies; rather, it directs our attention to research desires, as a result of the methodolog- ical discussion above. A, ; — Best, Karl-Heinz ed. Brainerd, Barron Weighing evidence in language and literature: A statistical approach. Chebanow Chebanow, S.
Dewey, G. Cambridge; Mass. Elderton, William P. London, Fucks, Wilhelm Nach allen Regeln der Kunst. Leningrad, Nauka: — Dordrecht, NL. Grzybek, Peter ed. Ljubljana etc. The Impact of Word Length.
Kromer, Victor V. Materialy konferencii. Ma- terialy konferencii. Markov, Andrej A. Mendenhall, Thomas C. Studien zum 1. Internationalen Bulgaristikkongress Sofia Piotrovskij, Rajmond G.
Williams, Carrington B. Wimmer, Gejza; Altmann, Gabriel Thesaurus of univariate discrete probability distributions. Zerzwadse, G. In: Grundlagenstudien aus Kybernetik und Geisteswissenschaft 4, — The idea is derived from the Fitts—Garner controversy in mathematical psychology cf. Fitts et al. Obviously, the problem is quite old but has not penetrated into linguistics as yet.
A word in a text can be thought of as a realization of a number of different alternative possibilities, see Fig. They can even be understood in different ways, e. What is neglected when correlating the lengths and the frequencies of words in real texts is the fact that for the text producer there is not at all free choice of all existing words at every moment.
Trying to fill in the blank is a model for determining the uncertainty of the missing word. It must be noted that SIC or HIC are associated not only with words but also with whole phrases or clauses, so that they represent rather polystratic structures and sequences. The present approach is the first approximation at the word level. Preparation In order to illustrate the variables which will be treated later on, let us first define some quantities.
The cardinality of the set X will be symbolized as X. P the set of positions in a text, whatever the counting unit. The elements of this set are tokens tijk , i. If the type and its token are known, the indices i and j can be left out. The elements of this set, aij , are not necessarily synonyms but in the given context they are admissible alternatives of the given token.
The index k can be omitted Aij the number of elements in the set Aij , i. This entity can be called tokeme. By defining Mij , we are able to distinguish between tokens of the same type but with different alternatives and different number ai — so they are different tokemes.
Example Using Table 9 cf. The text is reproduced word for word in the second column of Table 9 p. The length is measured in terms of the number of syllables of the word. Thus, e.
We can define it for types too: then it is the mean of all LLs of all tokens of this type in the text. LL is usually a positive real number. The errors compensate each other in the long run, so the distribution of L equals that of LL. It can be ascertained for any text.
We can set up the hypothesis that Hypothesis 1 The longer the token, the longer the tokeme at the given position. This hypothesis can be tested in different ways. As an empirical consequence of hypothesis 1 it can be expected that the distribution of L and LL is approximately equal. A token of length L has alternatives which are on average the same length, i.
Since LL is a positive real number it is an average we divide the range of lengths in the text in continuous intervals and ascertain the number of cases the frequency in the particular intervals. This can easily be made using the third and the sixth column of Table 9 p.
The result is presented in Table 3. It can easily be shown that the frequencies differ non-significantly. Since the distributions are equal, they must abide by the same theoretical distribution.
Using the well corroborated theory of word length cf. Wim- mer et al. As a matter of fact, for the distribution of LL we take the middles of the intervals as variable. It would, perhaps, be more correct to use for both data the continuous equivalent of the geometric distribution, namely the exponential distribution — however, again not quite correct.
Thus we adhere to discreteness without loss of validity. The result of fitting the geometric distribution to the data from Table 3. Length range in tokemes In each tokeme the lengths of words local latent lengths are distributed them- selves in a special way.
It is not fertile to study them individually since the majority of them is deterministic i. It is more prolific to consider the ranges of latent lengths for the whole text. For this phenomenon we set up the hypothesis Hypothesis 2 The range of latent lengths within the tokemes is geometric-Poisson.
Since the latent length distribution LLx is geometric and each LLx is al- most identical on average with that of Lx the alternatives tend to keep the length of the token , the range of the latent lengths in the tokeme is very restricted.
The deviations seem to be distributed at random, i. Evidently, the fitting is very good and corroborates in addition hypothesis 1, too. Thus latent length is a kind of latent mechanism controlling the token length at the given position.
Latent length is not directly measurable, it is an invisible result of the complex flow of information. Nevertheless, it can be made visible — as we tried to do above — or it can be approximately deduced on the basis of token lengths.
Information Content of Words in Texts 99 Table 3. Stable latent length Consider the deviations of the individual token lengths from those of the re- spective tokeme lengths as shown in Table 9 p. This encourages us to set up the hypothesis that Hypothesis 3 There is no tendency to choose the smallest possible alternative at the given position in text.
The hypothesis can easily be tested. SIC of the text Above, we defined SIC of a type as the dual logarithm of the mean size of all tokeme sizes of the given type, as shown in formula 3. Two possibilities can be proposed. We shall use here 3. For the given text it can be computed directly using the fifth column of Table 9 p.
We suppose that it is the smaller the more formal the text. We can build about it a confidence interval. Here the tokeme sizes build a sequence of a 1, 16, 3, 2, 1, 8, 2, 1,.
Taking the dual logarithms we obtain a new sequence b 0, 4, 1. In order to control the information flow and at the same time to allow licentia poetica, zeros and non-zeroes must display some pattern which is characteristic of different text types. Thus we obtain the two state sequence c 0, 1, 1, 1, 0, 1, 1, 0,. We begin with the examination of runs of 0 and 1 and set up the hypothesis that Hypothesis 4 The building of text blocks with zero uncertainty 0 and those with selection possibilities 1 is random i.
In practice it means that the runs of zeroes and ones are distributed at random. In our text see Table 9 , p. Another possibility is to consider sequence c as a two-state Markov chain or sequences a and b as multi-state Markov chains.
In the first approx- imation we consider case c as a dynamical system and compute the transition matrix between zeroes and ones. Taking the powers of the above matrix we can easily see that the probabilities are stable to four decimal places with P 4 yielding a matrix with equal rows [0. Since P n represents the n-step transition prob- ability matrix, the exponent n is also a characteristic of the text. Alternatives, length and frequency Since SIC has not been imbedded in the network of synergetic linguistics as yet, it is quite natural to ask whether it is somehow associated with basic language properties such as length and frequency.
In the present paper all other properties e. The data for testing can easily be prepared using Table 9 p. Below we show merely lengths 4 and 5 because the full Table is very extensive cf.
Table 3. This results in Table 3. In such cases they must be taken into account explicitly. In our case this leads to partial differential equations.
Let us assume that length has a constant effect, i. Fitting this curve to the data in Table 3. This is, of course, merely the first approximation using data smoothing because the text was rather a short one. Interpretation and outlook Looking at Tables 3. But we recognize that the influence of frequency is considerably weaker than that of length. If we regard 3. The direction of this influence is even more astonishing: with increasing length the number of alternatives is increasing too, longer words are more often freely chosen, while one perhaps would expect a preference for choosing shorter words.
Since the e-function plays an important role in psychology, for example in cognitive tasks like decision making, we suppose that word length is a variable which is connected with some basic cognitive psychological processes. Andersen, S. Attneave, F. New York. Berlyne, D. Coombs, C. Englewood Cliffs, N. Evans, T.
Fitts, P. Garner, W. Hartley, R. Piotrowski, R. Wimmer, G. June 21—23, , Graz University. Bahnhof 2 - 1 2. Altona 3 - 1 3. Kinderbuch 3 - 1 3. Stiftung 2 - 1 2. Deutschland 2 - 1 2. Kinder- 2 - 1 2. Krimi 2 Kriminalroman, Thriller 3 3. Kinderbuchautor 5 Autor, Schriftsteller, Kinderbuchschriftsteller 4 4.
Andreas 3 - 1 3. Anfang 2 Beginn, Start 3 1. Jungen 2 - 1 2. Weltuhr 2 Uhr 2 1. Seiten 2 - 1 2. Aktion 3 Leistung, Tat, Sache 4 2.
Guinness 2 - 1 2. Rekorde 3 - 1 3. Altona 3 Hamburg 2 2. Bahnhofshalle 4 Halle, Vorhalle, Wandelhalle 4 3. Szenen 2 Bilder, Teile, Partien 4 2. Detektive 4 - 1 4. Introduction This paper concentrates on the question of zero-syllable words i. As an essential result of these studies it turned out that, due to the specific structure of syllable and word in Slavic languages a several probability distribution models have to be taken into account, and this depends b on the fact if zero-syllable words are considered as a separate word class in its own right or not.
Predominantly putting a particular accent on zero-syllable words, we examine if and how the major statistical measures are influenced by the theoretical definition of the above- mentioned units. We do not, of course, generally neglect the question if and how the choice of an adequate frequency model is modified depending on these pre-conditions — it is simply not pursued in this paper which has a different accent. Basing our analysis on Slovenian texts, we are mainly concerned with the following two questions: a How can word length reasonably be defined for automatical analyses, and b what influence has the determination of the measure unit i.
Thus, subsequent to the discussion of a , it will be necessary to test how the decision to consider zero-syllable words as a specific word length class in its own right influences the major statistical measures. Any answer to the problem outlined should lead to the solution of specific prob- lems: among others, it should be possible to see to what extent the proportion of x-syllable words can be interpreted as a discriminating factor in text typology — to give but one example.
In a way, the scope of this study may be understood to be more far-reaching, however, insofar as it focuses relevant pre-conditions which are of general methodological importance. For these ends, we will empirically test, on a basis of Slovenian texts, which effects can be observed in dependence of diverging definitions of these units.
Word Definition Without a doubt, a consistent definition of the basic linguistic units is of utmost importance for the study of word length. Zero-syllable Words in Determining Word Length Irrespective of the theoretical problems of defining the word, there can be no doubt that the latter is one of the main formal textual and perceptive units in linguistics, which has to be determined in one way or another.
Knowing that there is no uniquely accepted, general definition, which we can accept as a standardized definition and use for our purposes, it seems reasonable to discuss relevant available definitions.
As a result, we should then choose one intersubjectively acceptable definition, adequate for dealing with the concrete questions we are pursuing. Taking into consideration syntactic qualities, and differentiating autosemantic vs. Subsequent to this discussion of three different theoretical definitions, we will try to work with one of these definitions, of which we demand that it is acceptable on an intersubjective level. The decisive criterium in this next step will be a sufficient degree of formalization, allowing for an automatic text processing and analysis.
Rather, what can be realized, is an attempt to show which consequences arise if one makes a decision in favor of one of the described options.
Since this, too, cannot be done in detail for all of the above-mentioned alternatives, within the framework of this article, there remains only one reasonable way to go: We will tentatively make a decision for one of the options, and then, in a number of comparative studies, empirically test which consequences result from this decision as compared to the theoretical alternatives.
This will be briefly analyzed in the following work and only in the Slovenian language, but under special circumstances, and with specific modifications. In the previous discussion, we already pointed out the weaknesses of this defini- tion; therefore, we will now have to explain that we regard it to be reasonable to take just the graphematic-orthographic definition as a starting point.
It can therefore be expected that the results allow for some intersubjective comparability, at least to a particular degree.
Zero-syllable Words in Determining Word Length b Second, since the definition of the units involves complex problems of quantifying linguistic data, this question can be solved only by way of the assumption that any quantification is some kind of a process which needs to be operationally defined. The word thus being defined according to purely formal criteria — i. This, in turn, can serve as a guarantee that an analysis on all other levels of language i.
The definition chosen above is, of course, of relevance for the automatic pro- cessing and quantitative analysis of text s. In detail, a number of concrete textual modifications result from the above-mentioned definition. In case of single elements, they are processed according to their syllabic structure. Particularly with regard to foreign language elements and passages, attention must be paid to syllabic and non-syllabic elements which, for the two languages under consideration, differ in func- tion: cf.
It should be noted here that irrespective of these secondary manipulations the original text structure remains fully recognizable to a researcher; in other words, the text remains open for further examinations e. Altmann et al. Unuk 3. In order to automatically measure word length it is therefore not primarily necessary to define the syllable boundaries; rather, it is sufficient to determine all those units phonemes which are characterized by an increased sonority and thus have syllabic function.
On the other hand, empirical sonographic studies show that there are no bilabial fricatives in Slovenian standard language cf. Srebot-Rejec Of course, 2 For further discussions on this topic see: Tivadar , Srebot Rejec , Slovenski pravopis ; cf. On the Question of zero-syllabic Words The question whether there is a class of zero-syllabic words in its own right, is of utmost importance for any quantitative study on word length.
With regard to this question, two different approaches can be found in the research on the frequency of x-syllabic words. In this context, it will be important to see if consideration or neglect of this special word class results in statistical differences, and how much information consideration of them offers for quantitative studies. As can be seen, we are concerned with two zero-syllable prepositions and with corresponding orthographical-graphematic variants for their phonetic realiza- tions.
As opposed to this, these prepositions are treated as zero-syllable words in modern Slovenian; they thus exemplify the following general trend: original one-syllable words have been transformed into zero-syllable words. Obviously, there are economic reasons for this reduction tendency. From a phonematic point of view, one might add the argument that these prepositions do not display any suprasegmental properties, i.
Following this diachronic line of thinking might lead one to assume that zero-syllable words should or need not be considered as a specific class in linguo-statistic studies. Incidently, the depicted trend i.
Yet, as was said above, it is not our aim to provide a theo- retical solution to this open question. Nor do we have to make a decision, here, whether zero-syllable words should or should not be treated as a specific class, i.
Rather, we will leave this question open and shift our attention to the empirical part of our study, testing what importance such a decision might have for particular statistical models.
Descriptive Statistics The statistical analyses are based on Slovenian texts, which are considered to represent the text corpus of the present study. The whole number of texts is divided into the following groups4 : literary prose, poetry, journalism.
The detailed reference for the prose and poetic texts are given in Tables 4. Table 4. Based on these considerations, and taking into account that the text data basis is heterogeneous both with regard to content and text types, statistical measures, such as mean, standard deviation, skewness, kurtosis, etc.
Level I The whole corpus is analyzed under two conditions, once considering zero- syllable words to be a separate class in their own right, and once not doing so. One can thus, for example, calculate relevant statistical measures or analyze the distribution of word length within one of the two corpora. Level II Corresponding groups of texts in each of the two corpora can be compared to each other: one can, for example, compare the poetic texts, taken as a group, in the corpus with zero-syllable words, with the corresponding text group in corpus without zero-syllable words.
Level III Individual texts are compared to each other. Here, one has to distinguish different possibilities: the two texts under consideration may be from one and the same text group, or from different text groups; additionally, they may be part of the corpus with zero-syllable words or the corpus without zero-syllable words.
Level IV An individual text is studied without comparison to any other text. A larger positive skewness implies a right skewed distribution. In the next step, we analyze which percentage of the whole text corpus is represented by x-syllable words. Text no. Three Text Types Figure 4.
It should be noted that many poetic texts do not contain any 0-syllable words at all. Of the 51 poetic texts, only 26 contain such words. Analysis of Mean Word Length in Texts The statistical analysis is carried out twice, once considering the class of zero- syllable words as a separate category, and once considering them to be proclitics.
Our aim is to answer the question, whether the influence of the zero-syllable words on the mean word length is significant. In the next step concentrating on the mean word length value of all texts Level I , two vector variables are introduced, each of them with components: W C 0 and W C. The i-th component of the vector variable W C 0 defines the mean word length of the i-th text including zero-syllable words. In analogy to this, the i-th com- ponent of the vector variable W C gives the mean word length of the i-th text excluding zero-syllable words see Table 4.
In order to obtain a more precise structure of the word length mean values, the analyses will be run both over all texts of the whole corpus Level I , and over the given number of texts belonging to one of the following three text types, only Level II : i literary prose L , ii poetry P , iii journalistic prose J. A scatterplot is a graph which uses a coordinate plane to show the relation correlation between two variables X and Y.
Each point in the scatterplot represents one case of the data set. In such a graph, one can see if the data follow a particular trend: If both variables tend in the same direction that is, if one variable increases as the other increases, or if one variable decreases as the other decreases , the relation is positive.
There is a negative relationship, if one variable increases, whereas the other decreases. The more tightly data points are arranged around a negatively or positively sloped line, the stronger is the relation. If the data points appear to be a cloud, there is no relation between the two variables. In the following graphical representations of Figure 4.
In our case, the scatterplot shows a clear positive, linear dependence between mean word length in the texts both with and without zero-syllable words , for each pair of variables. This result is corroborated by a correlation analysis. W C 0 b Scatterplot W L vs. W P 0 d Scatterplot W J vs. W J 0 Figure 4. As to our data, a strong dependence at the 0. Let us therefore take a look at the histograms of each of the eight new variables.
The first pair of histograms cf. Figure 4. Still, we have to test these assumptions. Usually, either the Kolmogorov-Smirnov test or the Shapiro-Wilk test are ap- plied in order to test if the data follow the normal distribution. Since, in our case, the parameters of the distribution must be estimated from the sample data, we use the Shapiro-Wilk test, instead. This test is specifically designed to detect deviations from normality, without requiring that the mean or variance of the hypothesized normal distribution are specified in advance.
To determine whether the null hypothesis of normality has to be rejected, the prob- ability associated with the test statistic i. If this value is less than the chosen level of significance such as 0. The obtained p-values support our assumptions, i. In the following analyses, we shall focus on the second analytical level, i. In order to test this, we can apply the t-test for paired samples.
This test compares the means of two variables; it computes the difference between the two vari- ables for each case, and tests if the average difference is significantly different from zero. This means that we test the following hypothesis: H0 : There is no significant difference between the theoretical means i.
Before applying the t-test, we have to test if the variables d L , dP , dJ are also normally distributed. As they are linear combinations of normally dis- tributed variables, there is sound reason to assume that this is the case. The Shapiro-Wilk test yields the p-values given in Table 4. The histogram of the variable dP shows the same result cf. In spite of the result of the Shapiro- Wilk test, we therefore apply a one sample t-test assuming that d P is normally distributed.
Two distribution functions for variables which denote mean word length of texts with and without zero-syllable words have the same shape, but they are shifted, since their expected values differ. The following Figures 4. It should be noted that this conclusion can not be generalized. As long as the variables dL , dP , dJ are normally distributed, our statement is true.
Yet, normality has to be tested in advance and we can not generally assume normally distributed variables. In the next step we show the box plots and error bars of the variables d L , dP , dJ. A box plot is a graphical display which shows a measure of location the median-center of the data , a measure of dispersion the interquartile range, i.
Horizontal lines are drawn both at the median — the 50th percentile q0. The horizontal lines are joined by vertical lines to produce the box.
A vertical line is drawn up from the upper quartile to the most extreme data point i. The most extreme data point thus is min x n , q0. Short horizontal lines are added in order to mark the ends of these vertical lines. The difference in the mean values of the three samples is obvious; also it can clearly be seen that all three samples produce symmetric distributions, variable dJ displaying the largest variability.
As can be seen, the confidence intervals do not overlap; we can therefore conclude that the percentage of zero-syllable words possibly may allow for a distinction between different text types. It turns out that the number of sylla- bles per word i.
This class of words may either be considered to be a separate word-length class in its own right, or as clitics. Without making an a priori decision as to this question, the mean word length of Slovenian texts is analyzed in the present study, under these two conditions, in order to test the statistical effect of the theoretical decision.
In the present study, the material is analyzed from two perspectives, only: mean word length is calculated both in the whole text corpus Level I , and in three different groups of text types, representing Level II: literary, journalistic, poetic.
These empirical analyses are run under two conditions, either including the zero-syllable words as a separate word length class in its own right, or not doing so. Zero-syllable Words in Determining Word Length Based on these definitions and conditions, the major results of the present study may be summarized as follows: 1 As a first result, the proportion of zero-syllable words turned out to be relatively small i.
Furthermore, it can be shown that the mean word length in texts under both conditions are highly correlated with each other; the positive linear trend, which is statistically tested in the form of a correlation analysis and graphically represented in Figure 4. As a result, it turns out that mean word length is normally distributed in the three text groups analyzed Level II , but, interestingly enough, not in the whole corpus Level II.
Based on this finding, further analyses concentrate on Level II, only. Therefore, t-tests are run, in order to compare the mean lengths between the three groups of texts on the basis of the differences between the mean lengths under both conditions. As a result, the expected values of mean word length significantly differ between all three groups.
To summarize, we thus obtain a further hint at the well-organized structure of word length in texts. Altmann, G. Figge zum Stutt- gart. Bajec, A. Predlogi in predpone. Best, K. Genzor; S. Wimmer; G. Altmann; R.
Girzig, P. Grotjahn, R. Grzybek, P. Jachnow, H. Lehfeldt, W. Jachnow ed. Lekomceva, M. Tom 1. Rottmann, Otto A. Royston, P. Schaeder, B. Srebot-Rejec, T. Tivadar, H. Unuk, D. Doktorska disertacija. Figure 5.
Many of the relevant psychological findings seem to be interesting for linguistics as well: the results reported e. Of particular linguistic interest is the question whether the serial position effects shown in the recall of lists of unconnected words show in the recall of real sentences as well.
Are the underlying processes also efficient in real sentence processing and in connected discourse? Having failed to disprove the charges, Taylor was later fired by the president" p. The serial position curve reported by Fenk 25 shows a marked recency effect only in auditory presentation of the sentence.
But these results originate from only two different sentences presented simultaneously in two different sense modalities. Subjects were instructed to write down as much as they could remember from the last sentence before the test pause. Nevertheless the family of curves shows a rather weak primacy effect and a marked recency effect.
Data from this experiment were re-analyzed in order to investigate further questions. Wordclass-specific effects on the serial position curve? In brief: The relevant division here is between context-specific content words and rather context-independent function words.
Ad b : A widely accepted model concerning our memory says: After having ex- tracted the meaning of an actual clause, its verbatim form words and syntax is rapidly lost from memory, while the meaning is preserved and affects e. This conception is strongly influ- enced by Sachs On the other hand, recall of previous sentences indicates that they had received a relatively thorough semantic interpretation.
The first quarter I was defined as the primacy part of the sentence, II and III taken together as the medium part, and the last 25 percent of the words IV as the recency part. But the alternative — to define the primacy part and the recency part in terms of a fixed number of words — would again be arbitrary: How many words should be fixed?
Our operationalization, however, offers a wide range of applications and es- tablishes a firm proportion between, on the one hand, the primacy and recency part, and, on the other hand, the part in between and the sentence as a whole. And it has proved to bring about significant results.
Thus, a quantification in absolute terms did not make much sense, and the recall scores had to be related to the number of words presented. Table 5. Actually there is, as can be seen from the values in Table 5. But in both cases this convergence is far from significant. Three more or less hypothetical regularities The formulation of the first of the following assumptions is motivated by the occasional observation that our test sentences taken from a Glasersfeld text showed a tendency of an increase of content words and a decrease of function words during a sentence.
Results strongly indicate that this is a general tendency at least in German texts. And if our tentative explanation section 4 of this regularity holds, its scope should not be restricted to German texts. Regularities 3. Regularity 3.
This statistical regular- ity has proven to be the most powerful one in the explanation of word order in frozen conjoined expressions Fenk-Oczlon , and it seems that its range of validity can be extended on clauses in general.
In this present paper we will state this generalized rule mainly as an inferential step to our third regularity 3. Despite the small sample of only ten sentences, the relevant differences proved to be significant in the Wilcoxon test Table 5. These differences in the distribution of the instances of the two word classes were, as already mentioned in section 2.
A pilot study was conducted in order to find some indications of possible generalizations of this tendency. The sample of authors was increased — nine more German text passages, four of them from scientific books, five from literary books. Taken to- gether with the already analysed text passages from Glasersfeld this is a sample of ten five scientific, five literary text passages, and a sample of ten sentences from each of these passages, i.
Source texts are listed at the end of the paper. She was instructed not to collect ten successive sentences from each passage into the sample, but — where possible — each third sentence.
Sometimes she had to overleap more than two sentences, e. As already suggested by Niehaus , a colon was accepted as the end of the sentence when the following word started with a capital letter. These results suggest that the tendency of function words to decrease and of content words to increase in the course of a sentence is a general tendency at least in German texts.
From all the rules examined e. Our regularity 3. Behaghel illustrates this law with many examples from classical texts in a variety of languages such as ancient Greek, Latin, Old High German and German. They were instructed to form a sentence from these fragments, and the result was always the same: sie besitzt Gold und edles Geschmeide.
Behaghel Behaghel f. At present we cannot offer results of empirical tests of this lawlike assumption. But we can contribute two new perspectives: 1. An interpretation specifying a concrete factor that might at least contribute to the rhythmic pattern described by Behaghel. This factor is the concentration or accumulation of function words in the first parts of clauses sentences, subordinate clauses.
And since function words are generally extremely fre- quent and frequent words tend — for economic reasons — to be rather short Zipf , , the concentration of these rather short units in the first part of clauses results in an increase of the mean word length in the course of a sentence. This hypothesized tendency will of course depend on the re- spective language type and is expected to be more pronounced in languages with a tendency to agglutinative morphology and a tendency to OV order.
This reference is — most probably not only in German texts — first of all brought about by function words e. If this is an appropriate explanation of our regularity 3. As a consequence, one may expect an increase of word length in the course of a sentence. Frankfurt: Suhrkamp Verlag suhrkamp taschenbuch Das Reich des Zufalls. Konstruktivismus statt Erkenntnistheorie. In: W. Mitterer eds. Klagenfurt: Drava Verlag. Der Steppenwolf. Doktor Faustus. Frankfurt a. Der Mann ohne Eigenschaften.
In: Best, K. Glot- tometrika 16, Quantitative Linguistics 58, — Unsicheres Wissen. Das Wahrheitsproblem und die Idee der Semantik. Wien: Springer-Verlag. Osterreichischen Linguistiktagung in Klagenfurt. Behaghel, O.
Fenk, A. Fenk-Oczlon, G. Jarvella, R. Luther, P. Murdock, B. Niehaus, B. Sachs, J. Zipf, G. Introduction From the first beginnings in the mids, availability of electronic text corpora in Slovenian, all with an Internet user interface, has grown to a level compara- ble to many European languages with a long history of quantitative linguistic research.
There are two established corpora with million running words, an academic one which is freely accessible and a commercial one, prepared by industrial and academic partners. The two are complemented by a sizeable collection of works of fiction, available for reading in a free virtual library and several specialized corpora, compiled for the needs of particular institutions.
The majority of Slovenian newspapers are also accessible online, at least in the form of selected articles. The basic infrastructure for word-length analysis is in place and in the fol- lowing chapters these topics are discussed in some more detail.
Online Text Corpora There are two online text corpora in the narrow sense of this word, each million running words in size and each equipped with an Internet user interface including a concordancer and some other searching facilities.
Other text col- lections have been built with different uses in mind and they complement the Slovenian corpus scene. Nova beseda was upgraded to 48 million words in September , to 76 million words in October , to 93 million words in April and to million words of text in Slovenian in July The current corpus contents can be classified as: DELO daily newspaper — — All texts have undergone an extensive word form check-up and correction process and so the level of noise is kept to a minimum over 45, errors, mostly typ- ing errors, but also other errors which usually appear during the preparation of electronic publications or its transfer from one format or platform to another, have been detected and corrected.
The corpus web pages are accessed over times a day and an overview of the referring URLs in the first three months of are shown in Table 6. The domain. DZS was also the coordinator and leading partner. Amebis, the main Slovenian en- terprise in the field of language resources, mostly spell-checkers, provides the A in FIDA. Gorjanc , the corpus contains million running words of mostly newspaper text, it went operational in and was completed in the first half of ; the corpus has remained unchanged since that time.
The project, aiming at a reference corpus of modern Slovenian, has been financed by the two commercial partners and so is not freely available. Free use is restricted to 10 concordance lines per search and the number of concurrent free users is also limited; full use requires the signing of a contract which regulates eventual publications based on the use of the FIDA corpus and a yearly fee in the vicinity of e per user.
Words from around 1. An automatic pro- cedure based on n-gram frequencies, is used to identify the page language — it is usually successful after two or three lines of text. The distribution of languages represented in March can be seen in Table 6. Nevertheless, it is an excellent source of new words in Slovenian. The search engine does not yet include a lemmatizer; a simple stemmer is used instead and it usually performs remarkably well.
Slovenian Polish Norwegian 82 2. English Danish Bulgarian 20 3. German Finnish Albanian 18 4. Croatian 4. Czech Korean 17 5. Serbian 2. Portuguese Ukrainian 10 6. Italian 2. Japanese Icelandic 4 7. French 2. Latin Arab 3 8. Russian 1. Dutch Macedonian 3 9. Spanish 1. Slovak Chinese 1 Hungarian Swedish Greek 1 Romanian Bosnian The entry prod Engl. Over the past three years the collection of books, mainly fiction, all in well-designed, attractive and legible PDF format clipboard copy is disabled , has grown to the current titles with over 40, pages.
Besides many classic works from late 19th and early 20th century, mainly scanned in by Mr. Evrokorpus is accompanied by Evroterm, which is not a standard web dictionary with terms in two or more languages, but a terminology database of the translated acquis communautaire.
It contains more than 40, entries and in April alone there were , queries, which makes Evroterm the second most popular web page on the Slovenian government server www. An inter-faculty project with much wider ambitions, involving electronic theses and dissertations, supported by a grant from the Ministry of Information Society, was initiated at the end of and at the beginning of Democracy can be chaos but it is, however, also the most effective way of doing things.
The more important five have a free online pres- ence — not with complete coverage but with a selection of articles available in full text. There are many more weekly , biweekly and monthly magazines in Slovenian and every year a larger number is available online, at least with a selection of articles. Yearly growth has been estimated to roughly 1. A copy of every printed publication is collected and stored by the national and university library NUK , under an instrument of legal deposit.
As virtually every publication nowadays is printed from a computer file, i. Words, Word Lengths More often than not, words are the basic units of linguistic research, and word lengths in particular are a very welcome object in quantitative linguistics Grzy- bek , The definition of a word was of no particular importance in classic works, such as grammars, but in corpus construction, for instance, it can be a real problem. How far to go, what to treat as a basic word token of a frequency dictionary?
Most definitions are close to what one would intuitively expect — a sequence of letters that can be pronounced and has meaning. In corpus construction, large groups of tokens also emerge which do not fulfill the above criterion but which definitely have a meaning and which obviously should not be wasted.
The author of these lines described them as wordlike terms and as nonwords; they could be classified according to the following schemas examples and frequencies, where given, are taken from the DELO — subcorpus, 47 million running words. Wordlike terms from DELO — 1. Incomplete Words. Hyphen-connected terms can be quite long, the longest is 68 characters long, and in the ten longest there are five writer-invented multiword expressions, four adjectives, one noun, and a chemical formula.
Words with parentheses are either explained abbreviations of names or two words written as one. Incomplete words would often look very strange if written without dots, and besides terms such as prapra. Nonwords from DELO — 1. Nonwords, especially numbers, represent the bulk of what in corpus does not fit the standard definition for word and if not treated properly they would seriously pollute the word form dictionary; each full URL, for instance, contains at least four strings of letters.
In Table 6. There is a remarkable match between the two fiction corpora in the top six places je, in, se, v, da and na — Engl. In general there are four words from the first list which do not show in the DELO column five from the second and only three words ga, sem, tako from the C.
How various corpora can really be quite different and how it shows in the top list of nouns can be seen from the Table 6. In the lists of the two fiction subcorpora words from ordinary life, of communication in romantic circumstances, such as eyes, heart, head, hand or cheek are to be found, while in the newspaper subcorpus words related to politics, economy and sports are easily recognized. SI web index the origin of top nouns is more difficult to explain.
From the table it is also clear that fiction operates with a smaller noun apparatus of higher frequency than is the case in other corpora. Figure 6. SI black, from the index of March , million Slovenian words.
SI It is clearly evident that fiction has a much more fluent language, the share of function class words, most of them two letters long also see Table 6. It is also interesting that the curve tail peaks at 5-letter words for fiction, 6-letter words for DELO and 4-letter words for the web index.
The share of long words, 14 letters or more, is negligible. SI These trends are further illustrated in Figure 6. This fact may be attributed to the large share of names; the tail diminishes much more slowly to the length of In the web index the peak is very broad, it stretches from 5-letter to 8-letter words, and it remains to be further explored in the future.
Table 6.
No comments:
Post a Comment