THE GENERAL CONTEXT AND POLICY FRAMEWORK

Technology Policy has become an increasingly important part of Economic and Industrial Policy in European countries in recent years. This is obviously because technology is one of the main driving forces behind structural changes in our industries and societies - the other two being globalisation and the change from the industrial society to the information society. From the Agency's point of view Technology Policy provides the starting point for everything. It contains the justification for the Agency's existence and general guidelines for the Agency's activities. Technology Policy represents the will of the Policy makers.

The current trend in Technology Policy is the gradual move from Government planned policy which is technology or industrial sector specific towards policies that try to build functional and innovation friendly environments. This change in Technology Policy means that there is a need to understand and develop the whole innovation system (innovation systems approach). The actual weighting of the Policy can be different in different countries - basic research oriented (e.g. Sweden), making an area attractive to foreign investment (e.g. Ireland), lowering employment costs (e.g. Netherlands) or education, knowledge and skills (e.g. Finland). But regardless of the differences the tools available to implement these policies can basically be quite similar.

What are the tools that are used in implementing Technology Policy? There are basically three classes of tool that can be used to implement Technology Policy. The first class of tools consists of various measures to improve the innovation infrastructure. The measures include the design of higher education institutes, public research facilities, public funding organisations, government owned enterprises, competence centres, consulting and other supporting services for R&D and innovation activities that are provided by the government.

The second type of measure is related to the regulatory framework. The legislation concerning companies - starting up, accounting, investments - is one of the factors deciding the birth and growth of companies. Lately, there has been a trend towards deregulation in a number of European countries to help the formation of a European common market.

The third type of policy measures consist of various incentives for R&D and innovation activities. These can contain direct subsidies, soft loans, tax concessions and indirect subsidies e.g. through public research facilities. The focus of the TAFTIE Agencies and therefore of these Guidelines is on these incentives.

What is the environment where the TAFTIE Agency operates and what are the mechanisms by which it's impact is realised? These questions can be answered by the attached graph where the environment and main interactions are shown. The Agency gets its resources and tasks from the policy makers and is required to set up Actions accordingly. Implementation is based on funding companies which can carry out R&D by themselves or in collaboration with other companies and/or research organisations, depending on the specific activity which justifies the funding. The R&D activities in the companies result in knowledge and products which are then distributed and diffused to the global marketplace in various ways.

There are two basic mechanisms by which the public funding for R&D activities in companies gives a return to society. The main mechanism is the one that could be called market mechanism or economic growth mechanism. The public funding encourages companies to invest more in R&D and thereby strengthens the companies' competitive capabilities in the markets. This helps the companies to succeed and grow which creates jobs and wealth in the society. The society can then redistribute the wealth according to the social agenda.

What does this look like from the companies perspective? The company's main objective is to make a profit today and in the future. The company's activities are based on its business strategy which - at least in technologically mature companies - includes a technology strategy. In the strategy, the company positions itself with respect to resources and markets. The company is continuously faced with competition in the markets and to maintain and improve its market position it needs to take action. To improve its market position the company needs to improve its products in the eyes of the customers. This can be done in a number of ways including better quality, better features, better services, lower price, etc.

One of the measures a company can take is investing in R&D and innovation activities. There are obviously alternatives and public support for R&D is designed to tip the scales in favour of R&D and innovation. The environment surrounding a company can be illustrated with the following figure.

The company sees public R&D incentives as optional resources, which it can use to strengthen itself. The characteristic features of public R&D incentives are that they must be used for R&D and innovation activities and that there are specific criteria which must be met. These criteria affect the behaviour of the company from the beginning. In order to get public funding the company must engage in co-operative research (consortia, use of public research facilities), plan the projects in more detail than they are used to, present initial marketing plans much earlier, etc. From the company's perspective there are definite benefits attached to public funding as opposed to own and private sources. These benefits vary from programme to programme and country to country and include grants, subsidised interests, etc.

What is the public funding for R&D? There are various types of funding schemes that the Agencies use, but they can typically be classified broadly into four classes: grants, loans, equity and tax-incentives.

Grants are direct subsidies to companies for R&D and innovation investment. They represent the highest immediate cost to the government. They are also usually the most wanted form of funding.

Loans used by the Agencies usually include some subsidy element. These subsidy elements can be subsidised interest rate, prolonged pay back, free years (only have to pay interest) and non-reimbursable risk condition (soft loans, e.g. if risks are realised the loan or parts of it do not have to be reimbursed). Loans can be introduced as revolving funds, in which case they represent a lower immediate cost to the government. The costs are, however, dependent on the reimbursement of soft loans. Loans are quite typical for funding among TAFTIE Agencies.

Equity funding is not so common among TAFTIE Agencies, but is found in some TAFTIE countries as a form of public funding. Venture capital markets usually function on a private basis, but there are public elements in some countries especially in the seed phase.

Tax-incentives can be introduced in many different forms. The schemes that are close to the TAFTIE Agencies are the ones that involve the Agency's expert assessment as a major factor in decision making. Tax-incentives in general usually have a less controllable focus and they tend to favour larger companies and other companies already investing heavily in R&D. They can be seen more as a reward for investing in R&D which constitutes the incentive effect. More innovative tax-incentive schemes include expert assessment and various forms of taxes which are relevant even to smaller companies bringing the ability to focus tax-incentives rather close to grants and loans.

How are the incentive schemes implemented? There are two kinds of activities with which an Agency typically works with; top-down and bottom-up activities. The top-down activities are ones which are being planned for specific purposes and where the funding is available by tailored criteria. These activities are specially planned by the Agency to achieve specific impacts in selected areas. Programmes promoting specific technologies, co-operation between companies and/or research units, programmes promoting technology diffusion, SME activation measures, etc. are typical examples of these kind of activities. These activities are usually the most visible part of the Agency's agenda.

In addition to top-down activities most Agencies have funding available also on bottom-up principle, where the criteria are not so strict in terms of purpose or structure. This type of funding is used for covering areas outside top-down activities and is based mostly on the market mechanism of impact.

Performance measurement does not differ greatly in case of top-down or bottom-up activities. The top-down activities are somewhat easier, because performance measurement can be based on specific objectives set for the activity, whereas in bottom-up activities the objectives are typically on a more general level. Similar methods of measurement and indicators as well as portfolio analysis can be used in both approaches.

What is the role of Evaluation and Monitoring in the Policy framework? According to a contract between Policy makers and the Agency at the design phase, the Agency must report and thus verify that it has fulfilled the tasks and reached its objectives. Depending on the outcome the Policy maker will then have the option of realigning Technology Policy implementation as well as Agency resources and objectives. The outcome must be verified according to the requirements of the Policy maker. The problem usually is, that the Policy maker does not specify how this reporting should be done. Thus, the Agency has to provide the reporting usually based on frequently asked questions by the government. Luckily, most of these remain the same with slight changes in importance at various times and in various political situations.

As the Policy maker seldom sets requirements on performance measurement another common practice is presented which is the requirement for independent evaluation of Agency activities. This practice has recently been strengthened in Europe. The problem with independent evaluations is that they are with very few exceptions not comparable or even transparent. Therefore it is very difficult for the Policy maker to draw any general conclusions from them. The conclusion is that even when utilising external and independent evaluation to assess the Agency's performance, the performance measurement must be done in a transparent and comparable manner and that the basic information is best collected on-line by the Agency.

The idea of using sets of key indicators, based on data on individual Projects, that can describe the performance of these Projects in a efficient way and when aggregated in a suitable way, is appealing. Apparent areas for their use, as far as R&D management is concerned, are the management of Actions and the reporting of results. However, in spite of efforts by many hands, good experience of their use is still required.

How does the reporting work and for what purpose? For the purpose of the present work, a framework of contracts consisting of four levels has been defined. In the framework the relationships between the levels are seen as formal contracts where tasks are defined (downwards) and results reported (upwards).

LEVEL		DECISIONS		REPORTING
National priority		decides on priorities between national needs, e.g. R&D (Government)		receives reports on progress and performance on an aggregated level
Policy		decides on the portfolio of policies, defines R&D actions and allocates resources (Ministry and Agency top management)		reports on portfolio progress and performance
Action		manages and allocates action resources, assesses projects at appraisal, monitoring and evaluation (Programme management)		reports action progress and results to the policy level
Projects		manages projects and project resources (Project management in companies and research organisations)		reports progress and results to the action level

In fact, the framework can be more elaborate and include parliament and eventually the electorate - those who in the end pay for the R&D are entitled to understand the results. Also the Ministry and Agency top management have a contractual relationship, even where they belong to the same organisation, which is the case in some countries. However, for the needs of this work, they have been included in one level.

Monitoring and evaluations are needed within all described levels. Their basic purposes are

• to report to the management function of each level (to understand, learn and when possible improve performance, to receive signals on needs for actions, e.g. mid term reviews, etc.)
• to report progress and results to the above levels
• to enhance organisational learning

For the Agencies, the main focus of interest is on defining the information required from the Project level for the purposes of Action management and for reporting progress and results to the Policy and higher levels. The challenge is to define, seek and present the appropriate information.

When should the information be collected and analysed? The data from Project level can be collected at various times for various purposes. Appraisal of a Project application includes information that is registered up to the point of a go ahead decision. From the evaluation point of view it is useful to register the ex-ante situation in order to establish a reference for later evaluation efforts.

The idea of monitoring is to check Project performance in a routine way, e.g. annually or at pre-set milestones. It involves analyses based on a limited amount of preferably quantitative and easily available data. The role of monitoring is to control performance, including receiving signals in case measures on Projects from the action management are needed, and to report progress. Mid term evaluations have a similar role.

By ex-post evaluation is meant the analysis of the outcome of an R&D effort at a time after completion of a Project or an Action when effects may be described or at least estimated with reasonable accuracy. How long after completion is a matter of judgement and depends on the nature of the R&D effort. Typical time scales vary from Project or Action completion to 3-10 years after completion. The role of ex-post evaluation is to report on effects at an aggregated level.

The role of performance indicators. Performance indicators are tools for seeking and communicating information, both for monitoring purposes and for evaluations. They should be seen in context with other ways of seeking information with which they form the basis for analyses and reports.

In the appraisal and monitoring situations, it is desirable to collect data in a routine manner, which focuses on the most vital information for the purposes in question. A number of key indicators can be obtained from the basic data for management purposes.

The quality of the information received is dependent on those who supply the information understanding it to be meaningful. Competent managers of R&D Projects should be seen as a scarce resource in the national perspective, and for that reason should not be asked to supply information that is not essential from the Action management point of view.

The design of evaluations should be done systematically. The Agency should prepare an evaluation strategy, which gives guidelines and answers to the basic questions concerning evaluation processes. The design of an evaluation could be seen as a three phase process according to the graph.

In the first phase (understanding) one should consider the drivers and objectives of the evaluation. Who is the evaluation commissioned by and why? Who will use the results of the evaluation and how? Who are the actual stakeholders and what is their role in the evaluation? What is the context of the Programme being evaluated?

The second phase is strategy, which consists of defining the scope and the general approach for the evaluation. All evaluations tackle some typical issues, like economy, effectiveness and efficiency. Quality, appropriateness, efficacy, process efficiency and improvement, impact, additionality, displacement and strategy are some of the other possible issues that can come within the scope of the evaluation. Other things influencing the scope and the general approach of the evaluation are the nature, timing and aim of the evaluation.

The tactics phase consists of the more detailed planning of the evaluation. Selection of methodologies, techniques and actual tools to analyse resource input, processes, output, outcome and impact. The choice of methods for data collection, data sources, variables, indicators, systems, etc. is made at this phase.