Resumé

Modern governments must increase their capacity for analysing and evaluating public
programmes. Evaluations, however, take place in a political context and many policies have
unstated political goals. Sweden appears to have a highly developed evaluation and
effectiveness auditing system and the Swedish case is instructive, not only because of its
experience in the evaluation field, but also because it illustrates the changing policy style.
Two main agencies are involved in the evaluation process and they both experience some
difficulties in their evaluation studies. These difficulties stem in part from the political
environment in which they both operate but also relate to their own organisational styles.
The resource squeeze could bring broad environmental charges which could be conducive to
a greater influence for the two agencies.

Introduction: Evaluation, Effectiveness Auditing and Rational Policy-Making

Daniel Tarschys has recently argued that modern governments must increase their capacity for analysing their own activities. Thus he suggests that "... we can no longer only be content with more effective methods for achieving objectives. The objectives themselves must be subjected to a strict priority grading, so that scarce resources are actually used for the purposes which are now considered most pressing" (Tarschys 1981, 17). Moreover, he suggests that this critical analysis cannot be left to the politicians. Civil servants who have personal experiences, and insight into the problems, must also play a key role. In short, "... we shall have to create procedures for analysis, debate and decision-making which give scope for the reconsideration of former decisions

---

* The research for this project was funded by the Nuffield Foundation. The authors wish to thank Gunnel Gustafsson for comments on an earlier draft, as well as acknowledging the helpful assistance from many officials in Sweden.

and which do not only result in demands for more and more resources'' (Tarschys 1981, 18). His analysis reflects a well known belief that evaluation research can increase the rationality of the decision-making and implementation processes. Thus, writing in 1975, Carol Weiss noted that "Evaluation research is a rational enterprise. It examines the effects of policies and programs on their targets (individuals, groups, institutions, communities) in terms of the goals they are meant to achieve. By objective and systematic methods, evaluation research assesses the extent to which goals are realised and looks at the factors associated with successful or unsuccessful outcomes. The assumption is that by providing "the facts" evaluation assists decision-makers to make wise choices among future courses of action. Careful and unbiased data on the consequences of programs should improve decision-making" (Weiss 1975). However, as she goes on to indicate, there are, of course, great problems in defining rationality. Moreover, evaluation always takes place in a political context and political considerations can intrude in several ways. Indeed, she suggests that one of the reasons why evaluations are so often disregarded is that they address only officially stated goals and ignore the political goals such as placating interest groups (Weiss 1975, 18). However, in situations where political considerations have been dominant in setting up a programme, but where an evaluation has been instituted, the evaluator can at least, as Weiss argues, point out the political trade-offs involved.

She is, however, generally pessimistic about the potential benefits of much evaluation research, noting that "as a matter of record, relatively few evaluation studies have had a noticeable effect on the making and remaking of public policy" (Weiss 1975, 18). The reasons for this relative failure are many, but are often essentially political in nature. We will be returned to the importance of political factors in our discussion of the results of evaluation studies conducted as part of the Swedish effectiveness auditing programme.

Despite the political problems in evaluation and effectiveness studies, most modern governments have begun to recognise the need to question existing programmes. This is particularly true, as Tarschys suggests, in a time of resource squeeze. The danger for the evaluator, as Weiss warns, is that he may risk becoming the political hatchet man — called in to discredit a programme which the politician wants to axe (Weiss 1975, 23). Though this clearly is a danger, it should not be overestimated, or be used as as excuse to prevent the development of effectiveness auditing. It is important to have independent studies of the functioning and effectiveness of public programmes if the public expenditure "debate" is not to be a collection of slogans and myths. If public money is being used for certain stated objectives, then some process is needed for assessing whether or not the objectives are being achieved, and ultimately whether the objectives are sensible in the light of changing circumstances. One of the difficulties which, for example, the Thatcher Government in Britain has

had in trying (unsuccessfully) to reduce the level of public expenditure is that it has lacked the necessary expertise to really question existing programmes (Richardson 1982). Unless central policy-makers have detailed evaluations of public programmes they will inevitably be making policy under a quite unnecessary degree of uncertainty.

It is no surprise that Sweden, which has traditionally stressed the importance of rationality and objectiveness in the policy process, should have what many believe is one of the most advanced systems of effectiveness auditing in the public sector. The Swedish case is instructive, not simply because effectiveness auditing has been developed to a far greater degree than elsewhere, but because the system has operated for the most part in a period of resource expansion in the public sector and is now operating in what is likely to be a decade of resource squeeze. The Swedish case is also instructive because it provides an important indication of more fundamental changes in the Swedish policy style. For example, there appears to be some modification of the principle that Swedish central ministries should remain small units, mainly concerned with policy. These ministries have tended to get larger, particularly since the arrival of coalition government (see Table 1). It also seems that there is a tendency for many ministries to get increasingly involved in what used to be considered administrative detail best left to the agencies, and a tendency for ministries to positively avoid making "policy" for fear of political conflict; policy "goals" are getting broader and less well defined; the agencies are in fact often determining policy (following the dictum "he who implements in fact decides policy") themselves; and significant changes have taken place in the central/local relationship (Gustafsson 1980). It is not the purpose of this paper to do more than draw attention to this "erosion" of rational policy-making in Sweden, but it does have important consequences for the nature and effects of the system of effectiveness auditing.

The Origins and Structure of Programme Evaluation and Effectiveness Auditing in Sweden

There are two main central agencies and an appointed body involved in some form of programme evaluation and effectiveness auditing in Sweden: the Parliamentary Auditors; Riksrevisionsverket (RRV or the Swedish National Audit Bureau); and Statskontoret (SAFAD or the Swedish Agency for Administrative Development). In addition, the Association of Swedish Local Authorities (Kommunförbundet) is concerned with the evaluation of the implementation of central government policies by local authorities. The "autonomous agencies" are also involved in many efforts to evaluate policies in their own areas. The main focus of this study is in fact the two agencies which are set up mainly

for the purposes of auditing: the Swedish National Audit Bureau and the Swedish Agency for Administrative Development. This is not to suggest, however, that the Parliamentary Auditors are of no significance and it may be useful to briefly outline their organisation and work before moving on to a detailed discussion of RRV and Statskontoret.

The Parliamentary Auditors

It can be argued that because of the political nature of the evaluation process, it is important to link the evaluation process with the legislature. Moreover, recent (1979) changes in the system of Parliamentary Auditing have strengthened the role of MPs in the work of the Parliamentary Auditors.

There are twelve Auditors and twelve Deputy Auditors, all members of the Riksdag. They have a staff of twenty-one. The Auditors operate through three sub-committees, which have the task of preparing reports for the Riksdag. In fact, the Auditors are not a Parliamentary Committee as such. They are an official "authority" (having a similar status to the Bank of Sweden) and have their own separate budget. The Reports (up to ten per year) themselves can fairly be described as effectiveness audits, rather than as traditional financial and regulatory audits.

The changes which were introduced in 1979 have directly affected the process by which topics are suggested for examination by the Authors. The individual Parliamentary Committees now have the power to make suggestions for audits and are invited to do so each year by the Auditors. The staff briefly examine the proposals and make suggestions to the Auditors after which some projects are subject to a pre-study, lasting up to a month. When the topics which are selected by the Auditors are finally reported on, the Reports are made public. They used to be sent to the government but are now sent to the relevant Parliamentary Committees. It is too early to gauge the effects of this change, but this might reduce the impact of the Auditors in the long run. In particular it may be difficult in future for the staff to follow up "their" Reports, as the responsibility for this rests with the individual Parliamentary Committees. The advantage of the recent changes is that the Audit process possibly stands a better chance of achieving publicity and, as a result, issues raised in the Audits may be more likely to reach the political agenda. It remains to be seen whether this trade-off outweighs the disadvantage of the weakening of more direct links with government.

Riksrevisionsverket: the National Audit Bureau

Riksrevisionsverket (RRV) lays claim to ancient origins. The ideas expressed in the current functions of RRV can be traced back to King Gustaf Wasa who set up the Exchequer in 1539 to control state revenues. RRV is today the main government agency with responsibility for accounting and auditing within the

central administration. In practice, RRV has four main duties (Berggren 1976,
7):

1. Audit of agencies and development of audit methods.

2. Development of methods for planning, budgeting and accounting.
3. Framing and supervision of the public accounting system.

4. Training of personnel in the fields of effective and efficient administration
such as planning, budgeting, accounting and auditing.

In terms of traditional financial auditing, RRV has played a decreasing role over recent years. Traditional financial auditing is now mainly the responsibility of the individual agencies themselves. Thus the 175 or so governmental agencies are now grouped into twenty-seven "accounting groups". Each "group" has an "accounting centre" for financial transactions and an auditing office for financial auditing. RRV's role in the financial auditing field is one of general oversight, and through information and training, the encouragement of good accounting and auditing practice.

The more important activity from our viewpoint is RRV's role in the field of effectiveness auditing as this, over the last decade, has gradually superseded its role as financial auditor. RRV was reorganized in 1967 and since then has been centrally concerned with the development of effectiveness auditing. By 1967 there was a widespread belief that traditional systems of budgeting and auditing were both inadequate (Iversson 1974; RRV 1975). The practical meaning of effectiveness auditing has, of course, changed over the decade or more during which it has been in operation. It is true to say, however, that the basic principles of effectiveness auditing which an RRV appointed Committee formulated in 1970, still hold true today. The Committee identified four main objectives for effectivess auditing:

1. To analyse the activities of government agencies.
2. To stimulate agencies to do this by themselves.
3. To offer them suggestions for improvements.

4. To supply governments with economic information in order to improve the
basis of future policies (RRV 1971, 13).

In practice RRV has developed a variety of detailed strategies for conducting effectiveness audits. The Director General of RRV, Rune Berggren, has, however, suggested that effectiveness auditing can be broadly divided into three stages. Firstly, the goals of an agency must be elucidated. Secondly, the activity of the agency must be examined. And thirdly, the agency's system of control needs to be examined (Berggren 1978).

Once an audit is complete and the Report published, it is formally up to the
agency concerned to put into effect (or not, as it chooses) any recommendation

which RRV makes or to consider any problem areas to which RRV has drawn attention. RRV has no legal authority to enforce any of its views resulting from an effectiveness audit. Indeed, RRV's lack of the power to enforce its Reports encourages it to adopt an organisation style which in some ways reduces its own effectiveness (see below).

Ultimately, as we shall argue, RRV has to persuade agencies to be more effective. In this sense, it is important to see RRV's activity as a total "package" of measures designed to improve efficiency and effectiveness at agency level. Indeed, the broad sweep of RRV's activity is usually referred to as the State Economic-Administrative System (SEA). SEA is intended to be an integrated approach to managing government activities in goal-oriented terms. It specifies how management systems within government shall be improved and aims to develop management tools and information systems for planning, budgeting, accounting, evaluation and auditing. The educative role within SEA is of central importance. The SEA "system" (it is in fact not a system as such, but an approach, using a range of management techniques) is designed to "spread the Gospel" of modern management techniques within the public sector. Indeed, it can be argued that some of the most significant achievements of the SEA programme, in increasing efficiency and effectiveness in the public sector, resulted from a specific publicity and educational campaign for the period 1974-76. As the Auditor General has argued "... we came to the realisation that the new demands being made on government agencies were concerned just as much with attitudes as with knowledge" (Berggren 1978, 3).

In practice, RRV's activity is organised under two programmes. Programme I covers RRV's effectiveness auditing work. The SEA approach is, of course, reflected in auditors' reports — for example, they may criticise a lack of long-range planning or poor information and control systems in an agency and may encourage the agency to use better management techniques. Programme I has been organised along sectoral lines, i.e. agencies are grouped according to policy sectors such as Education, Defence, Industry, etc. The organisational structure also reflects a need to audit certain special functions across agencies, e.g. tariffs and charges. More recently (1980) RRV has been trying to develop what they call a "matrix system" within Programme I. The concept behind the matrix system is that RRV should in future concentrate on problems which are common to several agencies. In practice this means that some projects involve the co-operation of auditors from several sections within RRV, and a number of difficulties have arisen in the early stages of the development of the matrix system (Storby 1981). For example, the projects tend to get delayed, and demand more resources. Also sectional loyalties are difficult to break down. On the other hand, if RRV can develop audits which are less agency-specific, then its actual impact on the agencies might be increased, as the agencies will be less defensive if the criticism is more widely spread. There is thus likely to be a shift

in emphasis, within RRV, in these directions, with slightly less emphasis on
sectoral based, individual agency specific audits.

Programme II is not directly concerned with effectiveness auditing (though it does include responsibility for the development and running of the System S public accounting system). The Programme includes a wide variety of activities such as collecting and analysing data for the public sector as a whole; helping the agencies to plan and account for their activities; forecasting state revenue and expenditure; following the development of public finance and producing extensive statistics on public finance. Work on the efficiency and effectiveness of the agencies is primarily concerned with accounting, planning, budgeting and results analysis.

In total RRV employs a staff of 280 (1980) and has an annual budget of Skr 51.9 million. It is directly responsible to the Budget Ministry and it is to this Ministry that RRV official Reports are sent - although its effectiveness audit Reports are also sent to the agency concerned and to the agency's sponsoring Ministry.

Statskontoret: the Swedish Agency for Administrative Development

Like RRV, Statskontoret (SAFAD) has ancient origins. It is now the national government's agency for improving the efficiency and effectiveness of the public service. Basically, SAFAD is the government's expert on matters of administration, concerning itself with organisational problems, the development of administrative methods, and more recently with the development of automatic data processing systems within the public sector.

The range of work conducted by SAFAD, under the broad heading of Organisation & Methods and Development, is considerable. For example, it has been involved in important "organisational overhauls" within governmental agencies; has been involved in the development of programme budgeting within the agencies; has developed^guidelines f°r personnel administration; has encouraged long-range planning; and has been extremely active in encouraging the introduction of computers into the public service. It can also be involved in quite detailed matters such as the design of forms and office procedures.

In practice there has been some overlap between Statskontorefs role and that of RRV. In principle, Statskontoret is not involved with programme analysis as such (or indeed with "effectiveness auditing") but the logic of organisational analysis often leads to recommendations which could equally well have resulted from an effectiveness audit or programme review. Thus "organisational overhauls" can and do develop into studies which have a lot to say about the effectiveness of public programmes, even though they are not formally effectiveness audits or programme evaluations. Just as policy often cannot be distinguished from administration, so then organisational matters also have

important policy implications, and questions of effectiveness and organisation are inextricably linked. Similarly, in the more specific area of automatic data processing (ADP) there are also important policy implications and questions relating to effectiveness.

We will discuss the changing role of Statskontoret below, which in fact has led to a lessening of overlap between itself and RRV. However, in 1975, it reorganized itself along sectoral lines (i.e. its own organisation reflected a grouping of policy sectors) which in fact bore some resemblance to the organisational structure adopted by the main effectiveness auditing agency, Riksrevisionsverket. The total staff of Statskontoret was approximately 325 (1980) with a total annual budget of Skr. 57.3 million.

Organisational Style and Programme Evaluation

Having briefly described the development of programme evaluation and effectiveness auditing and the organisational structures involved, we are now in a position to analyse what may be termed the "organisational styles" of the two main organisations. As Downs suggests, bureaus can often develop distinctive ideologies, particularly if they are engaged in highly controversial activities (Downs 1967, 245). To what extent have Statskontoret and Riksrevisionsverket developed a bureau "ideology" or organisational style and how does this relate to their own effectiveness? To what extent is organisational style related to the strategies used by controllers and the controlled in the programme evaluation field?

Stress may occur where an organisational ideology or style has developed which no longer corresponds to other important "ideologies" in the environment surrounding an organisation. On the other hand, an organisation which has an existing style or ideology which becomes "fashionable" in its external environment may benefit considerably in terms of influence or indeed organisational growth. Even without environmental changes, organisations like Statskontoret and Riksrevisionsverket both depend heavily upon developing good external relationships. They have few direct powers over the individual agencies and can affect agency behavior only by persuasion and force of argument or by influencing government itself (in the form of the Budget Ministry or the sponsoring ministries). In formal terms, neither Statskontoret nor RRV is the "competent authority" in respect of individual agencies. RRV does have quite considerable powers of investigation. Thus it decides what agencies to audit and when and in what way. It has the power to issue (and of course publish) Reports resulting from its effectiveness audits — and it has important powers regarding the oversight of traditional financial auditing. But these very important powers do not guaranteee influence over agency behaviour. So although RRV has a statutory right to conduct effectiveness audits, it has no statutory right to enforce its findings. In contrast, Statskontoret has in practice

no statutory right to conduct studies or projects within the individual agencies. It is either "called in" by the agency itself or it is called in at the request of the government or Royal Commission. Like RRV, it depends on other actors to see that its recommendations are carried out. In both cases, therefore, there are rather complicated "control" problems in terms of trying to influence the behaviour of "audited" agencies. In practice, sophisticated strategies have been developed by audited agencies to reduce the influence of both RRV and Statskontoret.

Organisational style is in both cases of central importance in determining organisational effectiveness, because it is a key factor in the relationship between the two organisations and their environment. Moreover, in the particular case of Statskontoret, there have been changes in the political environment which have had an effect on Statskontorefs influence and upon the nature of its activity. It is, therefore, useful to take Statskontoret as our starting point in discussing organisational style, notwithstanding the fact that, for the purposes of our study, RRV is the more central agency.

Statskontoret: an Agency Searching for a New Role?

Statskontoret is a good example of an agency whose external environment has changed and is now more hostile. This has forced Statskontoret to change both the nature of its work and its organisational style. Basically, Statskontoret now places greater emphasis on projects concerned with Automatic Data Processing (ADP) and less emphasis on big organisational studies. It is far less in demand as a consultant to individual agencies and its influence on government has declined very considerably. Its "image" is seriously out of step with current values in the political system. How has this come about and what has been Statskontoret's response?

Statskontorefs traditional role has been more in keeping with the period of Social Democratic rule than with the period of non-Socialist governments, since 1976. Under the old Social Democratic government, when resources for the public sector were expanding, Statskontoret had an extremely important role to play in devising organisational structures for "new" policy areas, such as environmental protection. Thus, as government took on new societal responsibilities, likewise Statskontoret was able to design the necessary structures for the administration and "delivery" of these policies. Statskontoret was also called upon to make the best, in organisational terms, of bargains which had been struck in the Commission system. An important consequence of this was that relatively expensive forms of administration might result, through no fault of Statskontoret.

In a similar way Statskontoret used its expertise in the more technical ADP
field to devise systems for the centralisation of certain areas of public administration
— such as the tax and social security systems. Much of "big government''

' and the expansion of the role of the State can be said to rest on technological developments in the ADP field. Unfortunately, (from Statskontorefs viewpoint) such centralised schemes now run counter to the current decentralist philosophy — particularly as formulated by the Centre Party. Fashions are as important in public administration as in social life generally, and Statskontoret is often seen as a purveyor of yesterday's fashion.

A third factor which has contributed to Statskontoret's current difficulties is that it has been associated with administrative schemes which increased costs. In one sense this was inevitable. As suggested above, Statskontoret was brought in by both government and by Commissions to design systems of administration for new policies and to help set up new agencies. These policies have proved to be expensive and there is a tendency to blame Statskontoret, rather than the policy itself. There is, however, a sense in which Statskontoret may not be quite so blameless in that agencies have, in the past, used it as an organisational consultant in order to gain more resources from central government. An agency's demands for more resources looked all the more justifiable if Statskontoret had been brought in as a consultant (in effect to legitimise demands) beforehand. Statskontorefs popularity with agencies (compared to RRV in its effectiveness auditing role) was possibly because it was prepared to back "rationalisation" schemes which in effect amounted to "... what will the politicians swallow in this reorganisation?" A Statskontoret study could be a useful strategy in the battle for resources. The nature of Statskontoret personnel and even internal promotion policies have helped. For example, it seems likely that Statskontoret has attracted employees who are naturally disposed towards expansionist schemes and whose skills lie in devising schemes which leave no-one in an organisation as a significant loser. Big schemes may also have been good for staff reputations within Statskontoret and this too will have reinforced Statskontorefs image as an organisation which has tended to increase costs as a result of its consultancy work.

All of these factors may have produced a "bureau ideology" within Statskontoret which is not conducive to a climate which values cost reduction more highly than service expansion. As an organisation, Statskontoret seems more suited to a semi-tropical climate of growth in public expenditures rather than the mini-ice age of cutbacks in public expenditure which Sweden is entering. The 1979 Coalition government did not use Statskontorefs expertise in its programme of expenditure reductions — largely because Statskontoret has the image of increasing costs and developing large ADP schemes, rather than as a cost reducer.

The net effect of these environmental changes is that Statskontoret has changed its policy towards the individual agencies. In the past the Statskontoret/Agency Agency relationship was often clientelistic. This has changed quite significantly. The individual agencies themselves have become much more defensive

in outlook. The combination of resource squeeze, with the political uncertainty of an unstable coalition (particularly after forty-four years of Social Democratic rule) caused agencies to be nervous and to seek out political allies as a defence strategy. This resulted in many agencies seeking a new clientelistic relationship, along Anglo-American lines, with their "sponsoring" (or "competent") ministries. This development is of fundamental importance in terms of Swedish public administration and in terms of the Swedish policy style as a whole. It is difficult to fully document changes in the agency/ministry relationship, but in many sectors the agencies are now much closer to their sponsoring ministries and see the cultivation of close and regular contacts at ministry level as crucial in an otherwise uncertain environment. In other sectors the traditionally more distant ministry/agency relationship has been preserved — particularly if an agency performs semi-judicial functions. If an agency is operating in an area of high political salience, it appears that there has been a tendency for it to drift closer to its ministry for what may be termed "steering signals". In some sectors, the ministry, too, feels the need for a very close relationship with its agency, particularly if the sector as a whole is threatened. In this situation both ministry and agency have, ultimately, a common interest in defending their sector and a Swedish version of "departmental pluralism" has developed (Richardson & Jourdan, 1979, 41-74). Moreover, the ministries, being small, can come to depend on the agencies for detailed information necessary to defend the sector. Similarly, if a hitherto "quiet" sector suddenly achieves political salience, this too can affect the ministry/agency relationship — a classic case being nuclear energy, resulting in a much closer ministry/agency/- relationship.

The development of closer ties between agencies and ministries in some sectors is, of course, quite the reverse of the constitutional theory behind Swedish public administration. No longer is it the norm that agencies maintain a rather distant relationship with their ministries — dealing with "administration" while the ministries formulate broad policy guidelines. For very many agencies the norm today is regular day to day contact between agency and ministry with the consequence that the agencies risk losing their traditional autonomy and the ministries are now getting clogged up with administrative detail. The period of coalition government since 1976 has seen a tendency for quite small issues to become matters of principle between the Coalition partners. Issues once resolved quite easily at agency level or by civil servants in the central ministries under the Social Democrats have tended to float up to the political level under coalition government. This great uncertainty also means that ministries sometimes positively avoid formulating broad policy guidelines. As Statskontoret is formally responsible to the Budget Ministry, agencies have naturally become more suspicious of its role as the resource squeeze has tightened. Thus if Statskontoret does approach an agency with a proposal, the reaction is likely to be

"... who is backing you?" Thus, Sweden shows some signs of an "immobilisme" not unlike the French IV Republic; it takes a very long time to reach decisions and there is often stalemate and public conflict between the governing parties. Problem-solving is all the more difficult because many problems are cross-sectoral, demanding co-operation and understanding between the different sectors (ministries). This "co-ordinated rationality" (Gustafsson & Richardson 1979, 427-429) is now difficult to achieve because of the increased pluralism within government referred to above. Moreover, the cultural tradition of the Swedish civil service is rather different to that of France where civil servants have been more confident in terms of making decisions on behalf of the State, rather than the government. In Sweden the tradition is more like that in the UK — of loyalty to the government and an unwillingness to be assertive in the absence of political direction. The Swedish civil service is generally not disposed to fill the political vacuum created at the political level.

A further factor affecting the Statskontoret/agency relationship is the result of specific legislation giving employees in public agencies considerable rights to participate in the internal decision-making processes. The Industrial Democracy Act, 1977 (the so-called MBL law) has had a serious impact on the work of Statskontoret because it is now much more difficult to secure change within organisations. As employees have an entrenched right to participate in the decision-making processes, they can effectively delay or even veto organisational change. This legislation has coincided with the arrival of the resource squeeze, which in turn has meant that extra resources are no longer available to smooth the path of reorganisation.

Statskontorefs position within the administrative system is now, therefore, much more uncertain and its role has been examined by a one-man Royal Commission which reported in 1979. The Commission was appointed to examine the rationalisation and ADP activity within the public sector (SOU 1979, 12). The Commission noted the problem of role conflict in this area and in the case of Statskontoret suggested that it act more as the agent of the government, as a watchdog over efficiency and cost-savings, rather than as a "consultant" to the individual agencies. The Commission, therefore, considered that Statskontoret should, in essence, be a governmental agency charged with increasing effectiveness and reducing costs in the public sector. In the event, although most of the Commission's recommendations were accepted, the consultancy role was not removed from Statskontoret.

The change in emphasis in Statskontorefs role was not opposed by Statskontoret itself (though it did object to the proposal to end its consultancy work altogether) — if only because it had already begun to change in this direction. Not only had the agencies tended to approach Statskontoret for advice less and less, but Statskontoret itself had taken an important internal policy decision to accept tasks only when directly asked by the government. The current position

is that Statskontoret has a firm commitment that the government must give it clear instructions before it accepts a task involving an individual agency. The difficulty is that the governments have been unwilling to face sharp choices in most policy sectors. Even if Statskontoret actively wanted the role of costcutter, it would find this difficult, within the agencies, without the prior political backing of the government. This was probably always the case, but is now of much greater importance where agencies and sponsoring ministries stand much closer together in some sectors. This relationship can only be broken with the backing of at least the Budget Ministry and more probably of the government as a whole.

As we suggested earlier, the logic of the resource squeeze in the public sector is that organisations like Statskontoret should be seen as a central element in the government's strategy of controlling public expenditure. A recent (1979) Report by the Commission on Public Policy Planning: Policy Innovation Through Policy Reappraisal placed great emphasis on the need to evaluate the effects of public policies, and argued that goal-oriented evaluations were a necessary condition for meaningful reappraisal of policies. In particular the Report stressed the role that both RRV and Statskontoret should play in this process. Thus

"... it is essential that existing policies are analysed by observers that are independent of the interests of the agencies ... the National Audit Bureau, the Swedish Agency for Administrative Development, and sectoral research can offer valuable contributions in this regard." (SOU 1979: 61)

Should this proposal be adopted, then Statskontoret (and RRV, see Section
3.3) should be very busy indeed.

For the present, however, is Statskontoret itself in a vacuum? In the absence of clear governmental directives relating to its potential as a cost-cutter, Statskontoret has managed to shift the emphasis of its work more towards ADP related projects than hitherto, i.e. the balance between "organisational" projects and ADP projects has changed in favour of the latter. Though large ADP projects are "out of fashion", the potential for the use of mini-computers in the public sector is considerable. The ADP "revolution" has left smaller agencies untouched or has not impinged upon all the activities of the larger agencies. The projects which Statskontoret now undertakes are much smaller in scope (lasting months rather than a year or more). This is both more in keeping with the current external climate of opinion and exploits the technical potential of mini-computers.

At present, discussions are being held between Statskontoret and the Budget Ministry concerning the type of projects which Statskontoret might take up. One possible model is that, where an agency has been instructed by the government to cut expenditure, Statskontoret might be involved in describing the actual

situation facing the agency. The responsibility for proposing the form of expenditure cuts would rest with the agency itself, and not with Statskontoret. The latter's role would be in organisational analysis, which as an exercise should highlight those areas where cost savings can most easily be made. The development of this role for Statskontoret will, of course, depend on a change in the attitude of the ministries themselves. Moreover, the division of the Finance Ministry into the Budget Ministry and the Economic Affairs Ministry has weakened Statskontoret (and RRV) because the new Budget Ministry (to whom they report) is weaker as a result.

Riksrevisionsverket: from Goal Analysis to Goal Substitution?

Any discussion of RRV needs to take account of the warning that "an agency that appears to be a single organisation with a single will turns out to be several suborganisations with different wills" (Pressman & Wildavsky 1973, 92). It is difficult to make generalisations about RRV. To be really accurate one should specify which Programme, which sector or indeed which project team and which effectiveness audit.

RRV is running approximately eighty effectiveness audit projects in any one year. In selecting agencies/projects RRV has complete discretion, but is influenced by a number of factors such as the size of an agency's budget, its strategic economic or social importance, its political salience, and the extent to which RRV may have reason to believe that the agency is experiencing problems of some kind. Once a project has been selected, it is normal for RRV to engage in discussions with the agency about the nature and organisation of the project. This preliminary stage, which can amount to a bargaining process, can be quite important and can have a significant effect on the actual nature of the effectiveness audit. Thus, although RRV has a statutory right to audit whatever it wants to, in practice the agencies are able to influence the audit process to some degree. This early stage, usually termed the pre-study, is also important in another respect — namely that it can influence the type of relationship v/hich develops between the audit team and the staff within an agency,. The results of the bargaining which can take place between the reviewer and the reviewed may, for example, dissuade the RRV team from reviewing the actual policy which an agency is implementing. It is always difficult for the evaluator to avoid evaluating the actual policy itself and there may be a natural tendency for effectiveness auditors to drift into evaluating policies rather than programme implementation. From an agency's viewpoint this tendency will normally be resisted as they claim no responsibility for policy as such — they find it convenient to adopt the constitutional position, even though in practice they are often the policy-makers themselves (Richardson 1979). There is thus an element of cat and mouse both before and during the pre-study where the agency being audited will try to confine the effectiveness audit to its own preferred areas.

The interactive aspect of the review process is also seen in the usual form of "interface" between the agency and the RRV team during the audit. Thus a "contact group" is formed consisting of the review team and a small group of staff from the agency itself. The dialogue between RRV and the agency does not end with the completion of the pre-study but continues within the contact group. These groups normally work well and a good relationship develops. The contact groups are multi-functional structures in which the participants quite clearly affect each other's perceptions and behaviour. One important function is that in both the pre-study and subsequent working of the contact group, agency staff "educate" RRV auditors in terms of the agency's tasks, environment, problems, procedures, etc. Thus the actual process of explaining what they do, how they do it and why they do it can be a very central part of the evaluation process from the agency's perspective. Having to explain them to a non-expert (but expert evaluator) can itself raise quite fundamental questions. If nothing else, it can be a mechanism for informing senior agency staff about activities within their organisation which they would not normally examine.

The learning process is also important from RRV's viewpoint. The degree to which RRV auditors should be experts in particular policy areas is in fact an item of continuous debate within RRV. On balance the agencies themselves would probably prefer auditors who are already fairly expert in a given policy area (in fact RRV on occasions does hire consultants for a particular study). The central issue in this debate is how best to deal with the problem of ' 'going native". One way is to emphasise evaluation skills rather than specialist knowledge of policy areas. Another is to recruit staff with specialist policy knowledge and maintain effective personnel management policies to deal with cases of "going native" when they arise. In practise, the number of cases of "going native" seems quite small. If there is a problem relating to the evaluator/agency/agen- relationship, it is quite the opposite of "going native". If anything, there is a tendency for auditors to feel obliged to produce strong criticisms of agencies in their final Reports. It is easy to find examples of projects where a good working relationship has developed in the contact group only to be damaged later by the publication of a very critical Report by the RRV team. The costs, in terms of the quality of Reports and damage to RRV/agency relationships, of having inexperienced (both in the policy field and in evaluation) auditors outweigh the costs (real and potential) of having auditors who have direct experience of policy sectors. RRV's attempts to change its recruitment policy reflects a recognition of this. As perceptions and relationships are essential to the ultimate success (as measured in terms of changing agency behaviour) of RRV it seems reasonable to argue that greater attention might be given to the need to encourage more confidence in RRV on the part of the agencies. Agencies often argue that RRV auditors are not sufficiently expert and are on occasions

quite naive. The accuracy of such a view is difficult to determine, apart from
in each specific case. The important thing is that it is thought to be true by the
agencies and can make them disinclined to listen to RRV's recommendations,.

In fact the staff problem is one of the most serious facing RRV. Its staff turnover, particularly in the effectiveness auditing field, is very high. Typically it runs at 25 per cent per annum and in 1979/80 RRV lost forty members of its auditing staff — predictably this has a number of serious effects (though the continuous arrival of "new blood" also has advantages). The most "political" phase of the evaluation process is when the RRV audit Reports are published and it is crucial, from RRV's viewpoint, that individual agencies should not be able to use "the doctrine of superior knowledge" as a means of ignoring RRV Reports. The less knowledgeable the auditor is of the specific policy area in question, the more vulnerable he (and his Report) is to the doctrine of "superior knowledge". This doctrine has proved important to audited agencies as a means of ignoring RRV Reports. Moreover, the fact that certain agencies in some sectors are now very close to the sponsoring ministries makes it much easier for them to use the doctrine to good effect. If an agency has developed a day to day relationship with its sponsoring ministry and has gained the confidence of the civil servants in the ministry, then it will find it relatively easy to say "but RRV just doesn't know about our policy area and does not realise that x is simply impossible or stupid." Yet again, the closeness of the agencies to the ministries emerges as the Achilles heel of the evaluation process. At the end of the day, RRV Reports will amount to nothing if the agency resists and is supported by its sponsoring ministry. Agencies know this and hence "keep their home fences mended" back at the ministry. The agency goal is thus often not to distance itself from the ministry but to develop a symbiotic relationship as the most effective way of dealing with criticism.

That the agency/ministry relationship is becoming more symbiotic in some sectors and is sometimes the Achilles heel of the effectiveness auditing process, is readily recognised within RRV. The problem is not in recognising the problem but in overcoming it. There are two main solutions. Firstly RRV, like Statskontoret, wants stronger government backing prior to the start of projects. It is a particular weakness that RRV has no direct relationship with the individual sponsoring ministries (or indeed with Parliament). At present, RRV has to rely on the Budget Ministry and the Cabinet to press its ideas on the sponsoring ministries and this does not seem to work at all well. Basically, RRV is like a spectator kicking the ball into the field and hoping that one of the players will actually pick it up and run with it. All too often the ball lands in the mud and is ignored! If the "resource squeeze" theory is correct, and RRV is seen as being a valuable instrument for cutting costs, then we might expect government to take the initiative more often by asking RRV to start specific projects. If projects start with a governmental initiative the auditors will be accorded

greater authority in their evaluation project and there will normally be some prior commitment by the government to actually "run with the ball" when it eventually returns to the field of play. This strategy, for RRV, can only work if the politicians themselves likt the idea of running with the ball and do not prefer to ignore it as at present. The key to the strategy, therefore, is the likely reaction of politicians in a continuing resource squeeze situation. One hypothesis is that the early phase of such squeeze is quite irrational, except in a purely political sense. Thus the first phase is one of all-round cuts of x per cent rather than a more selective approach according to clearly stated priorities. Only when a more rational approach to reducing expenditure is developed, can RRV expect the political impetus which it feels is necessary in order to increase its own impact on the agencies. If this does develop then RRV could add a third "Programme" to its activities, specifically to handle requests from the government to conduct projects.

An alternative strategy for RRV is to change its own "style" so as to increase influence on the agencies, irrespective of a more "rational" policy style at Cabinet level. In fact it can be argued that much of the frustration felt by effectiveness auditors is a direct result of RRV's organisational style in Programme I (i.e. effectiveness auditing). It is possible to argue that RRV has been subject to a degree of goal substitution in its activity and that this has made a significant contribution to reducing the effectiveness auditing. There appears to be great emphasis on the production of critical Reports at the end of effectiveness audit projects, and "success" may too often be measured in terms of the length, amount of criticism, and mass media coverage of the Reports. A rather radical (but nevertheless probably accurate) view is that the production and publication of effectiveness audit Reports can be counter-productive in terms of the effectiveness of the evaluations. RRV may have fallen into the very same trap into which it claims other agencies have fallen, i.e. it may have produced internal goals as a substitute for real measures of effectiveness. Thus it is difficult to measure the effectiveness of effectiveness audits and so the "measurable" (or at least recognisable) becomes the goal. In RRV's case the "internal culture" is such as to stress the need for critical Reports. This tendency may be re-enforced by the high staff turnover, as a spell with RRV is seen by civil servants as a very valuable step in career terms. "Success" for the individual has in the past been best demonstrated by a long Report, finding all sorts of things wrong with the audited agency. A longer-term (and real) impact on the agency, achieved by other means, is hardly likely to improve the career prospects of the individual in the short-run. If he wants a job in a central ministry (a common reason for staff turnover) it may be important to be able to point to a detailed critical Report as part of his "portfolio", rather than to less easily recognisable achievements at agency level. It may well be the case that the emphasis on producing critical Reports also coincides with the organisational

goals of RRV, as well as with individual staff goals, irrespective of whether or not the Reports actually influence the agencies. It is normal practice for RRV to issue a press release when a Report is published. This has a perfectly legitimate function of informing the media of the contents of what are often bulky Reports in an attempt to secure accurate reporting. But the fact that the media and the public are very receptive to anything which is critical of the bureaucracy can lead to quite unjustified media criticism of an agency.

The Reports and the media coverage of them re-enforce RRV's image as the agency trying to increase efficiency and highlighting the faults of the bureaucracy with which ordinary citizens have to deal. The Reports are thus quite functional to RRV's image as a "successful" organisation. It is, however, important to ask if they are important to the real goals of RRV — the improvement of efficiency and effectiveness within the agencies. The Reports themselves often create a sense of bitterness within the audited agency because of the adverse publicity they cause, all the more so because they have been preceded by good co-operation in the investigation stage. Another possible explanation for the hostile agency reaction is that criticisms contained in the actual Reports may on occasions be expressed in somewhat blunt terms. Just as the quality of the actual effectiveness audit varies, so does the degree to which criticism of agencies is "gift wrapped" in the final Reports. Many RRV staff would argue that they make very considerable efforts to "soften" the tone of their Reports and that senior management edit Reports with this very point in mind. It does seem, in theory, that more might be done in this direction (although this would be difficult in the Swedish context).

The problem in softening the tone of the Reports is that, apart from being counter to the Swedish emphasis on open government, it could run counter to one of the main objectives of the Reports themselves — which is to gain publicity and to "pressurize" agencies to respond to RRV criticism. In the absence of statutory powers of enforcement, and in the absence of the political backing described earlier, RRV tends to rely on publicity as its main sanction against agencies. Thus RRV can often appear to be adopting an evaluation strategy described as a confrontation strategy, where the evaluator seeks to "force" a change on the decision-maker or system (Palumbo & Wright 1980). Barkdol has also argued that under certain types of evaluation strategies, "... programme analysts (like inspectors) are rewarded when they find something wrong" (Barkdol 1980). The agencies themselves recognise and to some degree fear this weapon. The agencies are now required to make a formal (and public) "reply" to RRV Reports (usually within six months), and so publicity may be seen as an essential means of influencing agencies — all the more so because RRV has the right to return to an agency and conduct a further audit, after which it can publicise the fact that the agency has not responded to criticism. The difficulty, in terms of the effectiveness of RRV, is that publicity as a weapon against

the agencies is at best a two-edged sword. In the long run, it helps to create an image of RRV as constantly criticising everything, because "that is its role". After a while everyone in the system (RRV staff themselves, the ministries, the agencies and the media) fully expects RRV to produce a critical Report at the end of an evaluation study. This then makes it easier for the agencies to dismiss RRV Reports because "... those RRV people just have to find fault, whatever the situation". RRV thus runs the danger, in some cases, of being like the man who always grumbles about everything - eventually no-one listens, whatever justification he may have! In a very real sense, RRV's effectiveness auditing might be more successful if there were no final Reports at all! There is a recognition of this problem within RRV, and contacts with the Budget Ministry and the separate sponsoring ministries are likely to be improved, as a way of "steering" agencies, in addition to formal audit Reports.

In fact RRC has conducted a review of its own effectiveness. The Review Results and Effects of Effectiveness Auditing (RRV Dnr. 1977: 672) was published in 1977. If much of what we have said is critical of RRV, this is not meant to suggest that it is an ineffective organisation - far from it in fact. RRV, however, recognised the difficulty of measuring the results of effectiveness auditing. Thus for example, it is difficult to establish a direct link between changes in the audited agencies and RRV audits, as many factors influence the behavior of agencies. It also recognised that there were a number of impediments to change within agencies — such as RRV's lack of formal powers of direction, the defensive attitude of agencies, lack of motivation within agencies, and the fact that RRV Reports may not be very specific. However, RRV audits often lead to the appointment of Committees to further examine problem areas and to the agencies themselves setting up investigations in response to audits. RRV also sees itself as having had a general impact, through the SEA "system", in encouraging "good practice" within Swedish public administration. It is more difficult to point to direct changes in the activity and output of agencies as a result of audits, if only because neither RRV nor the audited agencies have found it easy to develop measures of the results and effects of agency activity. Moreover, it is rare for an audit to have produced direct savings, except in the sense that funds were used more effectively as a result. A very important effect of effectiveness auditing, claimed by RRV, is of course the "incentive effect" of the mere existence of an auditing agency. Another unmeasurable effect is the diffusion of ideas, as the result of a specific audit. RRV also emphasised the "therapeutic" effects of its audits - namely that ideas were discussed within agencies, during the audit, and that this itself could initiate development processes within agencies, irrespective of the content of the audit Report. Finally, the RRV Review of its own effectiveness, rightly stressed its role in the "remiss" procedure, during which RRV is regularly consulted by Commissions, government and Parliament on proposals for policy

change in virtually all policy areas. The fact that RRV has built up a wide understanding of the system of public administration in Sweden means that its reviews, in the vitally important consultation process in Sweden, carry considerable weight. RRV itself sees its "remiss" work as one of its most significant activities. RRV thus plays a very important function of widening what Laurence Martin has called the "market for ideas" (Martin 1962).

A further area in which we might look for RRV influence is in the internal dynamics of the audited agencies. The process of the effectiveness audit (as opposed to the final Report) can have quite important effects on agency behaviour. The dialogue which develops between the audit team and agency staff can lead to developments within agencies before the Report itself is finalised and published. In particular, the effectiveness audit is an important process for launching new ideas within an agency or at least in giving much greater impetus to ideas which are already "floating around" somewhere in the agency being audited. RRV is thus playing an important role in influencing the policymaking agenda within agencies. Reforms which for internal political reasons have long been resisted may just get adopted if an authoritative outside body lends its weight to them. It may thus be a humble, but crucial, role for RRV to help ideas to "bubble through" an agency until they reach the surface and get acted upon.

Acceptance of this argument does entail a rather unconventional view of the role of the evaluator. The dominant model of the evaluation process (the "decisionistic model") sees evaluation as an activity which functions to alter and enlighten the pursuit of programmatic goals (Floden & Weiner 1980, 9). This model emphasis the impact of evaluation upon discrete decisions made by public managers. An alternative model of evaluation emphasises other, less measurable functions such as conflict resolution and complacency reduction in organisations. Thus evaluation can be just as well regarded "as a means for managing conflict so as to promote a gradual social change" (Floden & Weiner 1980, 13). Under this view participants in organisations at least get to discuss issues with each other during the evaluation process and thus acknowledge the possibility of compromise. Viewing evaluation as "complacency reduction" reflects the argument we put forward earlier about the importance of the process of evaluation rather than the actual Reports. As Floden and Weiner argue "The very act of participating in an evaluation may spur the consideration of new practices by practitioners ... In the complacency reduction model, evaluation serves as a prod to program participants; it leads them to be clear about what they are trying to accomplish and to consider alternatives to the complacently accepted verbal goals" (14).

In fact there may be developments within RRV which could increase its impact
on individual agencies. For example, there is now less emphasis on goal
analysis, as it is now recognised that this can often be of rather limited value

where public policies have become so vague and ill-defined. There is greater emphasis on examining the evaluative capacity of the agencies themselves. Franklin and Trasher have stressed the importance of the "routinization" of evaluation if organisations are to be effective (Franklin & Thrasher 1976) and RRV's greatest impact may yet be in convincing agencies of the value of inhouse evaluations and in generally directing them towards modern management techniques.

Program Evaluations and Effectiveness Auditing and the Importance of the Changing Swedish Policy Style

We have earlier emphasised the essentially political nature of the evaluation process - be it in the form of an effectiveness audit or in the form of an organisational study of the type conducted by Statskontoret. The future success of the Swedish evaluation programme may fairly be said to rest on the interaction between the internal political processes within the two main evaluation agencies and wider developments in the Swedish political system as a whole. That Sweden is advanced in the evaluation field, by international standards, is beyond dispute. We have agreed, however, that broader political trends (e.g. the impact of coalition government, the changing ministry/agency relationship, changes in participation rights, the resource squeeze) have a very direct impact on the evaluation process. Space does not permit an analysis of the reasons for the fact that policy-making in Sweden has become more difficult in the 1970's (Ruin 1982). Despite these trends, however, the evaluation process is considered to be of increasing importance in Sweden (see Tarschys 1981, 24). It is important to ask, therefore, what is the essential role of evaluation in Sweden? One possible answer is that the evaluation process in Sweden may be a good example of Floden and Weiner's tentative hypothesis that one of the functions of evaluation is purely ritual. Thus they speculate that a further function of evaluation may be

"... a ritual whose function is to calm the anxieties of the citizenry and to perpetuate an image of governmental rationality, efficiency and accountability. The very act of requiring and commissioning evaluations may create the impression that government is seriously committed to the pursuit of publicly espoused goals, such as increasing student achievement or reducing malnutrition.

Evaluations lend credence to this image even when programs are created primarily to appease
interest groups." (16)

We would not wish to take this argument too far, but, to a certain degree, it does seem to fit a description of both the developing Swedish policy style, and the role of evaluation within it. Thus RRV audits, whatever they may or may not achieve in terms of specific agency decisions (the "decisional model"), certainly

do hold agencies accountable for their activity. In more general terms the evaluation and review process in Sweden do serve to perpetuate the image of rationality and objectivity which is particularly important in the political culture. For example, there is at least some element of ritual in RRV churning out "rationalistic" Reports which the ministries are often not terribly interested in. Rationality can sometimes be the last thing the ministries want in the highly political atmosphere in which they operate. The "rationalistic" world has been stood on its head. Ministry officials are often concerned with administrative detail and the need to achieve compromise at the political level and between powerful interest groups, whilst RRV, for example, is trying to get them to think rationally in terms of objectives and effectiveness. The culture is so important, however, that it demands at least the trappings of rationality. The rationalistic element must be taken seriously, even if in practice it is often ignored.

However, it is important to note that the fact that a Commission on Policy Innovation Through Reappraisal was set up, and did emphasise the possible contribution of RRV and Statskontoret in the review process. Thus it suggested that RRV should develop an extra programme - the Review Programme — to handle requests from the government relating to the analysis and review of programmes. Statskontoret too, the Commission suggested, should, again in response to governmental requests, play an important role in the review process, both independently and in co-operation with the individual agencies. Interviews at ministry level suggest that this process is already under way, i.e. the resource squeeze has helped the ministries to recognise the potential, to them, in both RRV and Statskontoret, of having two specialist independent agencies to review the rest of the agency system. Moreover, there are considerable advantages to both the Budget Ministry and the sponsoring ministries in having two such "review" agencies, which can tender separate advice. It is thus likely that, increasingly, RRV (and to a lesser extent Statskontoret perhaps) will receive specific requests for analyses relating to reduction of costs in the agencies. Thus more of RRV and Statskontorefs work will be preceded by some form of political commitment from government and can therefore be expected to have a greater impact.

However good the evaluation process is, it ultimately depends on a political impetus for it to have an effect. It is in this area that the future effectiveness of the Swedish evaluation system rests. The decade of experience of the evaluation process has achieved much, particularly in terms of changed attitudes, but also at the "decisional" level. It has also been an important period in terms of experimenting with evaluation techniques. If this experience is to be fully utilised, then decision-making at the political level will have to correspond more closely with the traditional image of Swedish policy-making.

REFERENCES

Downs, A. 1967. Inside Bureaucracy. Boston: Little, Brown & Co.

Barkdoll, G. 1980. "Type 111, Evaluations: Consultation and Consensus," Public Administration
Review, Vol. 40, No. 2, March/April 1980, pp. 174-179.

Berggren, G.R. 1976. "Financial Control and Auditing in Swedish Public Administration",
International Journal of Government Auditing.

Berggren, G.R. 1978. "Effectiveness Auditing in Swedish National Administration", International
Journal of Government Auditing.

Effectiveness Auditing 1971. Stockholm: RRV

Floden, R.E. & Weiner, S.S. 1978. "Rationality to Ritual: The Multiple Roles of Evaluation
in Governmental Processes", Policy Sciences 9.

Franklin, J.L. & Thrasher, J.H. 1976. An Introduction to Program Evaluation, New York:
John Wiley & Sons.

Förnyelse genom omprövning, 1979. SOU 1979: 61.

Gustafsson, G. 1980. Local Government Reform in Sweden, Umeå: CWK Gleerup.

Gustafsson, G. & Richardson, J.J. 1979. "Concepts of Rationality and the Policy Process",
European Journal of Political Research 7: 415-436.

Iversson, S.I. 1974. Program Budgeting in Sweden: Experiments, Experiences, Effects. Stockholm:
RRV.

Iversson, S.I. 1974. Programme Budgeting in Sweden, Beleidsanalyse (Holland) 3

Martin, L.W. 1962. "The Market for Strategic Ideas in Britain: The Sandys Era," American
Political Science Review, LVi: 23-41.

Palumbo, D.J. & Wright, P.J. 1980. "Decision-making and Evaluation Research". Policy Studies
Journal 8: No. 3 (special issue).

Pressman, G.L. & Wildavsky, A. 1973. Implementation. Berkeley: University of California
Press.

Rationalization and ADP in Government Administration. 1979. SOU 1979:12.

Richardson, J.J. & Jourdan, A.G. 1979. Governing Under Pressure: The Policy Process in a
Post-Parliamentary Democracy. Oxford: Martin Robertson & Co.: 41-74.

Richardson, J.J. 1979. "Pollution Control in Sweden: Administration or Policy", Statsvetenskaplig
Tidskrift, Häfte 4: 225-239.

Richardson, J.J. 1982. "Political Problems in Reducing Public Expenditure: The Experience
of the Thatcher Government in Britain", POLITICA 1:5

Ruin, O. 1982. "Sweden in the 1970'5: Policy-Making Becomes More Difficult", in Richardson
(ed), Policy Styles in Western Europe, Allen & Unwin.

Storby, B. 1981. Some Possible Causes of Delays in the Matrix Project Between Different Elements,
RRV, internal paper, undated.

Tarschys, D. 1981. "Public Policy Innovation in a Zero-Growth Economy: A Scandinavian
Perspective". Baehr, P.R. & Wittrock, B. Policy Analysis and Policy Innovation. Beverley
Hills: Sage.

Weiss, C. 1975. "Evaluation Research in the Political Context." Struening, E.L. & Guttentag,
M. (eds), Handbook of Evaluation Research, Vol. I. Beverley Hills: Stage.