This article examines the accuracy of the Gallup poll in a situation involving multiple political parties of different sizes. Using data collected by the Danish Gallup Institute 1957-1980, the article demonstrates that polling results, except those of small parties, are well within the sampling error. An explorative analysis of the data, showing that there is a normal distribution of the differences between election results and polling values, provides the possibility of a simple method of interpreting Gallup poll data at a desirable level of confidence.

Introduction

The purpose of this article is to present a new way of assessing the reliability of the Gallup poll as an indicator of voter support for political parties in a multi-party system. It will be shown that Gallup poll results for large and medium-sized, but not for small, parties are within the sampling error.

The accuracy of national polls in the predominantly two-party system of the United States is well documented (Kohut 1986). Denmark affords an opportunity to assess the reliability of the Gallup poll in situations involving multiple political parties of different sizes.¹ Twelve political parties are represented in consecutive Gallup poll figures since 1957. Six of these parties have less than 10 per cent of the electoral vote each, and the other six range in size from 10 per cent upwards, with only one party reaching the level of 30 to 40 per cent of the vote in the current polls.

In some ways the Danish system is similar to the Israeli political system that has recently been described by Shamir (1986) regarding the reliability of polling with respect to the large blocks of parties involved in two general elections. Like other researchers of polling confidence Shamir uses only

---

* This study is part of a project dealing with Danish political events and polls 1957-83. The project is supported by the NATO Research Grants Programme, Brussels. The author is grateful to Niels-Erik Jensen, Statistical Institute, University of Copenhagen, for helpful comments on this explorative analysis but is solely responsible for analysis and interpretation.

pre-election polis and election results; post-election polis tend to be regarded with suspicion and are not considered. By contrast, I have used post-election polis as well in the analysis, and I shall discuss the consequences.

The Data

In Denmark, monthly Gallup polls began more than ten years before those of other polling agencies. The research data base covers the period from June 1957 to June 1980. Eleven or twelve times per year during this period the Danish Gallup Institute (Gallup Markedsanalyse A/S) has asked 1,000 to 1,200 persons: 'To which party would you give your vote if there were a general election tomorrow?' The number of respondents (unweighted) is estimated on the basis of Danish Data Guide Update 1982.² The institute publishes its results in the newspaper Berlingske Tidende, but without information on the number of respondents and the non-response. The Danish Data Guide survey of the sample data for the year 1978 yielded the following information: the average number of persons polled was 1049; 70.1 per cent gave the name of their preferred party; 13.4 per cent did not know; 6.9 per cent refused to answer; 6.0 per cent were without franchise (15-17 years of age); 3.1 per cent had decided not to vote; and 0.6 per cent gave other answers.

However, it is not my intention to discuss the Gallup methods but rather to study the published 2,055 individual party polls for the years 1957-80 with respect to their reliability. I emphasize, it should be noted, that I am not using the special pre-election polls prepared by Gallup, but only the ordinary monthly Gallup polls.

Compared to the population of 2,055 cases there were 73 occasions where a political party has participated in a general election.³ Because the time interval between polls and elections is not constant, I have established - by linear interpolation from the last Gallup poll before an election to the first Gallup poll after the election - a poll figure corresponding to the week of election (i.e. the week in which the day of election occurs. The time of a poll is considered to be the middle week of the interview period).

Analysis

Election Results and Polls

The election results and the interpolated week of election figures are shown in Table 1. Also displayed are the relationships between the election results and the last Gallup poll before an election and the first Gallup poll after an election, respectively. On average the last poll was taken 4.7 weeks

before an election, but if the two unusually long periods in 1957 and 1968
are excluded the average time interval before an election is 2.6 weeks. This
is similar to the average time interval for the first post-election polls.

As can be seen in Table 1, the average numerical difference between election results and the first poll after an election is only 0.74 percentage points; the difference between election results and the last poll before an election is 1.64 percentage points. The latter figure is similar to the 1.5 percentage points average error per party for the major British parties in 37 polls made a few days before eleven general elections between 1945 and 1979 (Worchester 1980).

This indicates that polls are more accurate in the first part of an interelection period than in the last part. Two explanations seem possible. First, in some procedures Gallup uses a separate question regarding the respondent's vote in the previous general election. It is a fact that some voters do not remember how they voted and that voter memory decreases over time (Adamany & Shelley 1980). It has been reported that 87 per cent of British voters are able to recall a recent economic event, but only 58 per cent and 33 per cent are able to do so after one and two years, respectively. After five years the percentage is only six (Hibbs 1982). Even though it might be easier for voters to remember their own voting than an economic event, the value of answers in regard to the weighting of numbers of polling respondents must decline as time passes after a general election.

Second, it would also seem reasonable to assume that voters do not change preferences immediately after an election, which would lead to greater polling accuracy. During the campaign period prior to an election voter preferences are more changeable, making consistent polling results difficult to obtain. However, Felson and Sudman have shown that polls taken during the last week prior to a general election are more accurate than those taken earlier (Felson & Sudman 1975).

In addition to the comparison of the accuracy of post-election and preelection polls, Table 1 demonstrates the accuracy of interpolated polls. The average error per party is only 0.55 per cent if election results are compared to the interpolated week of election figures. If the last pre-election and first post-election values taken from the three- or five-point moving averages of polls are used in order to estimate the week of election figures, accuracy diminishes to 0.68 and 0.73 per cent, respectively. This indicates that there is no justification for using methods more complicated than simple linear interpolation between two polls.

Distributions of the Polls

I now turn to the matter of distribution. My point is that the 73 estimated
Gallup polls from the week of election can be regarded as a sample drawn
at random from the about 2,000 such estimates in the total series of Gallup

polls

polls 1957-80. General election day in Denmark is decided freely by the Prime Minister. With respect to the accuracy of the polls his decisions are made at random. Perhaps he thinks that his party will get a good result at that particular time but his purpose is not to check if polls are accurate compared with the result of a general election. Therefore it can be assumed that the 73 cases are drawn at random.

The next problem is the form of the distribution in question. This subject has been treated in Table 2. Here it is verified that there is an approximately normal distribution of the differences between the election results and the estimated week of election polls. And the mean of the distribution is close to the level of the election results which can be seen in Table 2.

As there is a normal distribution it is possible to find the variance,
standard deviation and the confidence limits (cf. Table 2). The table

concerns the whole sample of 73 cases and samples of the 11 large party cases, the 22 medium-size and the 40 small party cases, respectively. Large parties have more than 25 per cent, medium-size parties 10 to 25 per cent, and small parties less than 10 per cent of the votes in the polls.

Table 2 shows the normal distributions for the various party groups, the
standard deviations of the distributions, and, in the last column, the range
of 95 per cent confidence.

It will be noted that the mean of the differences is 0.1 percentage point or less when only two sets of polls are used in the interpolation process. As in Table 1, the best results are found by simple linear interpolation between two consecutive Gallup polls. Thus, by taking the last poll before an election and the first poll after an election and interpolating to find a corresponding figure for the week of election, the better estimate is obtained.

The 95 per cent confidence intervals for the mean in the distribution of these estimated polls are shown in Table 3. However, it has to be noted, as can also be seen in Table 3, that the confidence intervals for the estimated Gallup polls should be multiplied by V2 since this estimate is a linear combination of the two individual polls. In what follows these individual poll intervals from Table 3 are used.

The next question is whether this method of interpreting Gallup polling data produces results that are within the sampling error. In finding the respondents for interviews Gallup uses a four-stage stratified sampling technique that will usually give somewhat better results than simple random sampling. However,-since the difference can really be considered as diminutive, I find it acceptable to regard the sampling as if the respondents had been found by simple random sampling. Under this assumption Table 4 has been calculated using the formula for sampling error in the 95 per cent level of confidence. Thus the confidence interval is approximately given by

where P is the size of the party in per cent and N is the number of
unweighted respondents.

As can be seen in Table 4 the 95 per cent empirical confidence intervals for individual polls are very well within the sampling error for large parties. Medium-size parties are within the sampling error too, but only just. On the other hand, small parties are not within the sampling error even if Ihe difference is not of much significance.

These conclusions depend upon the assumption that sampling error
intervals are estimated by drawing at random 800 respondents out of a

large population. As previously mentioned, Gallup polls in Denmark 1957-80
usually have included 1,000-1,200 respondents and about 700-800 (70
per cent) appropriate, unweighted answers.

However, it is demonstrated in Fig. 1, where the empirical curve and the curves for the sampling error are shown for N-values of 700, 800. 1,000 and 1,200, that the conclusions hold true for all mentioned unweighted numbers in question. While the empirical 95 per cent curve for Gallup poll confidence intervals can be visually observed in Fig. 1, the roughly approximate values can be seen in Table 5. Notice that the three small circles in the mentioned empirical curve indicate values corresponding to average sizes for the groups of small, medium-size, and large political parties, respectively (cf. Table 3).

It is remarkable that the differences between ranges of certainty for large and small parties in the empirical study are so narrow. No doubt this is why practical errors in estimating polling results for small parties are relatively so predominant, while the technique of stratification favours the results for large parties.

However, it is possible to compare the final empirical results with results
from a Swedish investigation in which another method has been used. The

* P is the party size in per cent. N is the number of interviewed persons. The three small
circles in the empirical curve indicate values corresponding to average sizes for the groups
of small, medium-sized, and large political parties, respectively (cf. Table 3).

differences between small parties and large parties are here very narrow too. Svenska Institutet for Opinionsundersokningar (SIFO), Stockholm, has analysed one hundred monthly SIFO polls 1967-78 using the Henderson Trend Method, where nine months moving averages of polls are considered as a trend line. By taking the differences between the trend line and the polls, the variance and the standard deviation have been computed, and

the 95 per cent confidence intervals for the distribution of the differences appeared as two standard deviations. This holds for all kinds of uncertainty, e.g. sampling error, error of measurement, etc. (cf. Holmberg & Petersson 1980. 91-92). The Swedish confidence intervals can be seen in Table 6. The Swedish figures are lower than the Danish, (only about two thirds) but Holmberg and Petersson criticize the Henderson Method for giving too small standard deviations.

My conclusion concerning the comparison with the Swedish investigation is that it is better to use results from general elections than to take a trend line, when accuracy of polls is researched. As my estimated polls are contemporary to the elections and as they are generated, relatively, of very varying pre-election polls and rather constant post-election polls, I assume that my method is more realistic. The results of my analysis are such that I am confident in applying the confidence intervals to any Gallup polls of political parties in Denmark, even if they tend to be a little too wide just after elections and in politically quiet periods, and a little too narrow during a campaign or in the last part of an electoral period or in other politically changeable periods.

Conclusions

The empirically found 95 per cent confidence intervals for the monthly Danish Gallup polls on voter support for political parties can be used directly in historical studies 1957-80. For small parties this means intervals of about ±1.8 percentage points; for medium-size parties ±2.5 percentage points; and for large parties ±2.0 percentage points for individual polls.

It is not appropriate to estimate the level of a party at a particular time by using an average of three or more consecutive polls. It is better to strike an average of only two consecutive polls. In this case the 95 per cent confidence intervals are to be found by dividing the above intervals for individual polls by V2. That is, for small parties 1.3 percentage points; for medium-size parties 1.8 percentage points, and for large parties 1.4 percentage points (cf. Table 3).

It is remarkable that the empirical confidence intervals for large and medium-sized parties are within the sampling error. This is especially obvious for large parties. The intervals for small parties are not within the sampling error, but close to it. It is noticeable that the Danish Gallup estimates large parties better than small parties and that the confidence intervals for the three party sizes differ less than one half from ±2.0 percentage points.

It is reasonable to use the mentioned confidence intervals in regard to
the current polls as the Gallup method has not been basically changed since

1980. These intervals will make the polls more informative and improve
political debates and discussions.

---

NOTES

1. A very detailed description of the data and their chronology is found in Larsen (1978).

2. Danish Data Archives, Danish Data Guide Update 1982 (Odense; University of Odense, 1982). The Data Archives holds machine-readable Gallup poll data described in the Guide.

3. About 20 occasions were not adequate because of missing data for very small parties in the poll series.

REFERENCES

Adamany, D. & Shelley, M. C. 1980. 'Encore! The Forgetful Voter', Public Opinion
Quarterly, 44, 234-240.

Danish Data Guide Update 1982. Danish Data Archives, University of Odense.

Felson, M. & Sudman, S. 1975. 'The Accuracy of Presidential Preference Primary Polls',
Public Opinion Quarterly, 39, 232-236.

Hibbs, D. A. Jr. 1982. 'Economic Outcomes and Political Support for British Governments
among Occupational Classes: A Dynamic Analysis', The American Political Science Review,
June, 259-289.

Holmberg, S. & Petersson, O. 1980. Inom Felmarginalen. Stockholm: Publica. Liber Forlag.

Kohut, A. 1986. 'Rating the Polls: The Views of Media Elites and the General Public', Public
Opinion Quarterly, 50, 1-9.

Larsen, D. 1978. 'Partiernes bevægelser i Gallups politiske index'. Århus: Institut for
Presseforskning.

Shamir, J. 1986. 'Preelection Polls in Israel: Structural Constraints on Accuracy', Public
Opinion Quarterly, 50, 62-75.

Worchester, R. M. 1980. 'Polsters, the Press, and Political Polling in Britain', Public Opinion
Quarterly, 44, 548-588.