MicrosoftEvaluation of Promotion and Information Actions
Anuncio
Evaluation of Promotion and Information Actions for Agricultural Products Final Report Volume I – Main report November 2011 Study realised by ADE in collaboration with Metis and Agrotec Rue de Clairvaux 40, bte 101 – B 1348 Louvain-la-Neuve – Tel +32 10 45 45 10 – Fax +32 10 45 40 99 E-mail [email protected] – Website www.ade.eu This report has been prepared by ADE at the request of the European Commission. The views expressed are those of the Consultant and do not represent the official views of the European Commission. Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC Table of contents LIST OF ACRONYMS LIST OF PROGRAMMES MEMBER STATE ABBREVIATIONS GLOSSARY EXECUTIVE SUMMARY INTRODUCTION ............................................................................................................1 1 DESCRIPTIVE CHAPTER: OVERVIEW OF THE EU POLICY FRAMEWORK ............... 3 1.1 DEVELOPMENT OF EU PROMOTION POLICY AND ITS IMPLEMENTATION FROM 2002 ONWARDS ........................................................................................................ 3 1.2 DESCRIPTION AND DEVELOPMENT OF OTHER PROMOTIONAL MEASURES APPLIED UNDER THE CAP ..............................................................................................20 1.3 COMPARISON OF EU PROMOTION WITH OTHER COUNTRIES ....................................34 2 EVALUATION METHODOLOGY ........................................................................... 37 2.1 OVERALL APPROACH AND DATA COLLECTION............................................................37 2.2 SELECTION OF MEMBER STATES AND PROGRAMMES FOR DETAILED EXAMINATION ..................................................................................................................39 2.3 SCOPE AND LIMITS OF THE EVALUATION METHODOLOGY .......................................43 3 COMPREHENSIVE SYNTHESIS OF THE EVALUATION REPORTS ON PROMOTION PROGRAMMES ............................................................................... 45 3.1 CONTEXT AND METHODOLOGY ....................................................................................45 3.2 THEME I – COHERENCE BETWEEN THE IMPLEMENTATION OF THE PROGRAMMES AND THE OBJECTIVES OF THE REGULATION ......................................49 3.3 THEME II – ACTIONS AND INFORMATION CHANNELS USED AND THEIR COSTEFFECTIVENESS ................................................................................................................59 3.4 THEME III – COVERAGE AND CONTENT OF THE PROGRAMMES ..............................69 3.5 THEME IV – IMPACT AND EFFECTIVENESS OF THE MEASURES .................................74 3.6 THEME V – COMPLEMENTARITIES BETWEEN MEMBER STATE PROGRAMMES AND THOSE SUBMITTED BY THE PROFESSIONAL ORGANISATIONS ...........................82 3.7 HORIZONTAL CONCLUSIONS AND RECOMMENDATIONS ...........................................86 3.8 GOOD PRACTICES.............................................................................................................94 4 REPLIES TO THE EVALUATION QUESTIONS ....................................................... 97 4.1 INTERVENTION LOGIC ....................................................................................................97 4.2 REPLIES TO EVALUATION QUESTIONS OF THEME 1 – POLICY RELEVANCE AND EFFECTIVENESS .................................................................................................... 101 4.3 REPLIES TO EVALUATION QUESTIONS OF THEME 2 – MANAGEMENT OF INFORMATION AND PROMOTION PROGRAMMES................................................. 152 4.4 REPLIES TO EVALUATION QUESTIONS OF THEME 3 – COHERENCE AND COMPLEMENTARITIES WITH OTHER CAP AND NATIONAL AND PRIVATE PROMOTION INITIATIVES ............................................................................................. 184 Final Report November 2011 Table of contents Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC 5. CLOSING CHAPTER ........................................................................................... 209 5.1 CONCLUSIONS ............................................................................................................... 209 5.2 RECOMMENDATIONS .................................................................................................... 221 LIST OF TABLES Table 1: Table 2: Table 3: Table 4: Table 5: Table 6: Table 7: Table 8: Table 9: Table 10: Table 11: Table 12: Table 13: Table 14: Table 15: Table 16: Table 17: Table 18: Table 19: Table 20: Table 21: Table 22: Table 23: Table 24: Table 25: Table 26: Table 27: Table 28: Table 29: Final Report Main Goals and Targets by Products and Themes ................................................. 5 Overview of the situation and goals (objectives) for the main products and themes with indicative annual budget ...................................................................... 7 EC Expenditure by Product/Theme in €m ........................................................... 10 EC Expenditure by Major Implementing Member States on Key Products/Themes (€m 2002-2010) ......................................................................... 11 Main differences between support to the promotion of fruit and vegetables under Regulation (EC) No 1234/2007 and Regulation (EC) No 3/2008 ........ 22 EU contribution to promotion measures under the CMO for fruit and vegetables in 2008 and 2009 .................................................................................... 23 Budgets of the reformed wine CMO measures ..................................................... 26 Promotion under national support programmes of the reformed wine CMO (in million euros)........................................................................................................ 27 Comparison of the support to promotion of wine under Regulation (EC) No 1234/2007 and Regulation (EC) No 3/2008 ................................................. 28 Comparison of the support to EU or national food quality schemes under Regulation (EC) No 1698/2005 and Regulation (EC) No 3/2008.................... 31 Use of measure 133 among Member States or regions ........................................ 32 Member States which have benefitted from EU co-funding to implement at least one promotion programme approved between 2008 and 2010 ................ 41 List of programmes chosen for analysis ................................................................. 42 Sample Programmes Analysed in the Framework Contract Evaluation Reports on Promotion Programmes ...................................................................... 46 Comparison of the Scope and Structure of the Framework Contract Evaluations and the Current Evaluation ................................................................ 47 Objectives of the promotionnal measures ............................................................. 51 EU value added of the EU co-funding ................................................................... 55 Multi-country and multi-target programmes on the internal market ................. 57 Multi-product, multi-country and multi-target programmes on the third country market ........................................................................................................... 58 Use and appreciation of the cost-output ratio ....................................................... 60 Efficiency of Different Channels ............................................................................. 62 Impacts of the programmes ..................................................................................... 75 Logos promoted by co-funded programmes and results ..................................... 78 Contribution of the programmes on the improvement of the image of EU agricultural products on the internal market ......................................................... 80 Contribution of the programmes on the improvement of the image of EU agricultural products worldwide and on the development of new markets ...... 81 Assessment of the reasons for which synergies and complementaries were (or were not) developed with the private sector ................................................... 83 EQ 1 Judgment criteria and indicators ................................................................. 103 Themes and products which cannot be analysed with the usual statistical sources and proxy indicators used ........................................................................ 103 Evolution of the list of themes and products covered by promotional measures on the internal market since 1994 ........................................................ 104 November 2011 Table of contents Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC Table 30: Evolution of the list of themes and products covered by promotional measures on third country markets since 1994 ................................................... 106 Table 31: Product classification at the EU level in the worldwide context ...................... 109 Table 32: Product classification at the internal market level ............................................... 110 Table 33: Products classification at the extra-EU level........................................................ 112 Table 34: Products and groups of products classification according to the EU export value in 2008............................................................................................................. 113 Table 35: EQ 2 Judgment criteria and indicators ................................................................. 115 Table 36: Third-country markets in which promotional measures may be carried out .. 116 Table 37: The scoring system applied to the China case ..................................................... 118 Table 38: Scoring of the listed third-country market in which promotional measures may be carried out regarding their relevance ....................................................... 119 Table 39: EQ 3 Judgment criteria and indicators ................................................................. 123 Table 40: Number of co-funded PDO, PGI and TSG programmes implemented on the internal market and on third country markets59F............................................ 127 Table 41: Number of co-funded programmes on organic food and farming implemented on the internal market and on third country markets ................ 128 Table 42: High level trade missions and promotional events implemented at the Commission’s initiative since 2000 ....................................................................... 130 Table 43: EQ 4 Judgment criteria and indicators ................................................................. 132 Table 44: Satisfaction with guidance provided by the Commission for multi-country programmes .............................................................................................................. 136 Table 45: Advantages and disadvantages of the multi-product, multi-country and multi-target programmes ........................................................................................ 138 Table 46: EQ 5 Judgment criteria and indicators ................................................................. 144 Table 47: EQ 6 Judgment criteria and indicators ................................................................. 153 Table 48: Issues in terms of selection procedures underlined by the framework contract evaluations covering the period 2002-2007.......................................... 155 Table 49: EQ 7 Judgment criteria and indicators ................................................................. 158 Table 50: EQ 8 Judgment criteria and indicators ................................................................. 166 Table 51: EQ 9 Judgement criteria and indicators ............................................................... 173 Table 52: Standard Selection Grid by Member States.......................................................... 175 Table 53: Most Common Comments in EC Assessments on 2009 Programmes ........... 177 Table 54: Programmes approved and rejected from July 2006 to end of 2010................ 178 Table 55: EQ 10 Judgment criteria and indicators ............................................................... 184 Table 56: List of Member States that declare having a national promotion strategy at general or at product levels. ................................................................................... 187 Table 57: EQ 11 Judgment criteria and indicators ............................................................... 191 Table 58: EQ 12 Judgment criteria and indicators ............................................................... 199 Table 59: Differences between Funds concerned with promotion of fruit and vegetables .................................................................................................................. 201 Table 60: Differences between Funds concerned with promotion of wine ..................... 202 Table 61: Differences between Rural Development Policy and Regulation (EC) No 3/2008 ....................................................................................................................... 203 Table 62: Objectives, messages conveyed, groups targeted and channels used of EU co-financed programmes and other CAP promotion measures ....................... 205 Table 63: Indicative figures of annual EU budget for each of these measures shown in the descriptive chapter 1.2 ................................................................................. 207 Final Report November 2011 Table of contents Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC LIST OF FIGURES Figure 1: Summary of Key Regulations ..................................................................................... 4 Figure 2: Total EC Expenditure for Promotion Measures on the Internal Market and in Third Countries (€m).............................................................................................. 9 Figure 3: EC Expenditure by Product/Theme as Share of Total Expenditure ................ 10 Figure 4: EC Expenditure by Proposer Country ................................................................... 11 Figure 5: Expenditure by Target Country ............................................................................... 12 Figure 6: Average annual expenditure per capita per farm holding and per Agricultural Labour Unit (2002 – beginning 2011) .............................................. 13 Figure 7: Number of Programme agreements signed in each year ..................................... 14 Figure 8: Number of Active Programmes by country........................................................... 14 Figure 9: Institutional Arrangements ....................................................................................... 16 Figure 10: Financial Plan (EAFRD) for measure 133 for the period 2007-201314F in €m .. 32 Figure 11: Tools used during the desk and field phases of the evaluation process ............ 38 Figure 12: European labels (former and current PDO; PGI, TSG, former and current organic logos) ............................................................................................................. 77 Figure 13: Reconstructed intervention logic of promotion and information measures for agricultural products ......................................................................................... 100 Figure 14: Extent to which competent bodies consider the listed themes and products relevant on the national or EU market................................................................. 107 Figure 15: Extent to which competent bodies consider the listed themes and products relevant on third country markets ........................................................ 108 Figure 16: Extent to which competent bodies consider the listed third-country relevant to achieve objectives of the Regulation ................................................. 120 Figure 17: Results of the online questionnaire to Ministries/ Competent bodies in the EU-27 Member States about the usefulness and the easiness to use of the guidelines .................................................................................................................. 162 Figure 18: Survey response on clarity of evaluation requirements...................................... 171 Figure 19: Response from the online survey about EU added value ................................. 192 Final Report November 2011 Table of contents Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC List of Acronyms Acronyms AIAB Associazione Italiana per l’Agricoltura Biologica AMAB Associazione Mediterranea Agricoltura Biologica ALU Agricultural Labour Unit BIVB Bureau Interprofessionel des Vins de Bourgogne BNIC Bureau National Interprofessionel du Cognac CAP Common Agricultural Policy CIRCA Communication & Information Resource Centre Administrator CFPR Consorzio del Formaggio Parmigiano-Reggiano CN China CH/IN/SEA China, India and South-East Asia CMO Common market organisation CNIEL Centre National Interprofessionel de l’Economie Laitière CPM Cost per thousand/mille CPP Consorzio del Prosciutto di Parma CS Serbia & Montenegro CTPV Consorzio Tutela Provolone Valpadana DE-TAG Information on the 5 am Tag promotion programme DG Agri Directorate-General for Agriculture and Rural Development EAGF European Agricultural Guarantee Fund EAFRD European Agricultural Fund for Rural Development EC European Commission EQ Evaluation Question EU European Union EUOAP EU Organic Action Plan FR & VEG Fruit and vegetables HLTM High-Level Trade Mission I&P Information & Promotion IVDP Instituto dos Vinhos do Douro e Porto IVSI Instituto Valorizzazione Salumi Italiani Final Report November 2011 List of Acronyms Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC IN India IVIQGM Instituto del Vino Italiano di Qualità Grandi Marchi IVOLI Olive Oil Quality Testing Institute MEA Middle-East & Africa MMF Milk Marketing Forum MS Member State NAF North Africa NAM North America NSP National Support Programme NZO Nederlandse Zuivel Organisatie OIAOE Organización Interprofesional del Aceite de Oliva Español PDRH The French Rural Development Programme (Programme de Développement Rural Hexagonal) PE Promotional Event PDO Protected Designation of Origin PDO+ PDO, PGI & TSG PGI Protected Geographical Indication PO Proposing Organisation RDP Rural Development Programme REG Regulation RU Russia SEA South-East Asia SPS Single Payment Scheme SRWRP Association of Polish Butchers and Producers of Processed Meat Sz-Nor Switzerland & Norway TSG Traditional Specialty Guaranteed UA Ukraine UIS Unione Italiana Seminativi UIV Unione Italiana de Vini USA/CAN The United States of America and Canada Final Report November 2011 List of Acronyms Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC List of Programmes Acronyms Programme Programme Title DE-TAG 5 am Tag promotion programme 5 am Tag EAS Rethymnis promotion programme Information and promotion actions for the Protected Designation of Origin (PDO) Cheese of Crete, (PDO Graviera Kritis, PDO Xinomizithra Kritis) in the markets of United Arab Emirates and Saudi Arabia Seko Dramas promotion programme Information and promotion activities of fresh vegetables (European tomato in third countries (Russian Federation, Ukraine and Serbia) ES-OIA OIAOE promotion programme The flavour of life (UK) Une vie de riche c’est bien. Une vie de luxe c’est mieux (France and Belgium) Cocina con amor (Spain) FR-BIV BIVB, CFPR, CPP & IVDP promotion programme - French component EL-EAS EL-SEK BNIC promotion programme Programme de promotion de l’IG Cognac cofinancé par l’UE et la France sur les territoires des Etats-Unis d’Amérique, de la Russie et de la Chine CNIEL & NZO promotion programme - French component Campagne d’information et de promotion France-Pays-Bas mettant en avant les bénéfices santé du lait et des produits laitiers (3 est un chiffre magique) FR-BNI FR-CNI FR-INT Interfel & Assomela promotion programme - French component FR-UGP UGPBAN, Caraïbes melonniers & Anafruit promotion programme IT-AIA Final Report Discover the Origin AIAB, Coldiretti & AMAB promotion programme The apple is a fruit of many pleasures Promotion sur le marché intérieur du symbole graphique des régions ultrapériphériques (« RUP ») dans le secteur de la production des ananas, bananes, melons et fruits exotiques, de Guadeloupe, Martinique et Réunion (Son goût unique, c’est tout le caractère de nos volcans ; Sa qualité unique n’est pas le fruit du hasard ; Choisir nos bananes, c’est apprécier le travail de planteurs indépendants) Bio Sotto Casa November 2011 List of Programmes Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC BIVB, CFPR, CPP & IVDP promotion programme - Italian component Discover the Origin IT-BIV Interfel & Assomela promotion programme - Italian component The apple is a fruit of many pleasures IT-INT UIS, IVOLI, CTPV & IVIQGM promotion programme European Art of Taste IT-UIS IT-UIV UIV-IVSI promotion programme Made for Taste CNIEL & NZO promotion programme - Dutch component Campagne d’information et de promotion France-Pays-Bas mettant en avant les bénéfices santé du lait et des produits laitiers (De magische 3) SRWRP promotion programme European Table – Tradition, Modernity, Quality PT-FEN Fenalac promotion programme Milk is not all the same BIVB, CFPR, CPP & IVDP promotion programme - Portuguese component Discover the Origin PT-BIV UK-MMF MMF promotion programme Make Mine Milk NL-CNI PL-SRW Note: further elements regarding this list (budget and themes) are provided in Chapter 2.2 Final Report November 2011 List of Programmes Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC Member State Abbreviations Abbreviation Final Report Member State’s name AT Austria BE Belgium BG Bulgaria CY Cyprus CZ Czech Republic DE Germany DK Denmark EE Estonia EL Greece ES Spain FI Finland FR France HU Hungary IE Ireland IT Italy LT Lithuania LU Luxembourg LV Latvia MT Malta NL Netherlands PL Poland PT Portugal RO Romania SE Sweden SK Slovakia SI Slovenia UK United Kingdom November 2011 Abbreviations Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC Glossary Advisory Group: Group consulted on the invitation of the Commission, with representatives of various social and economic interests according to art.17a of Council Regulation (EC) N°3/2008. Annual evaluation report: as defined by Commission Regulation (EC) No 501/2008 (article 19), report to be submitted by the proposing organisation to the Member State within four months of completion of the annual programme measures. This report, which shall include a summary of the work carried out, evaluation of the results obtained and a summary financial statement showing all scheduled and incurred expenditure, is mandatory in respect of payment of the balance of the co-funding of the programme. Coherence: The extent to which the intervention logic is not contradictory/the intervention does not contradict other interventions with similar objectives Commission initiative: the Commission’s decision to carry out information and promotion measures as defined by Council Regulation (EC) No 3/2008, Article 10 (a) (ii) and (b). The measures referred to in Article 2(1) (d) (impact assessments) are not under the scope of this evaluation. Except for high level trade visits, the Commission may take the initiative in measures that are of Community interest or where no appropriate proposal has been submitted in accordance with articles 6 and 9 of Regulation (EC) No 3/2008. Complementary/synergetic effect: effect obtained as a result of complementarities/synergies at EU and/or Member State level. These complementarities/synergies can appear at different stages of the intervention logic; EU co-funded programmes might complement each other in their choice of objectives pursued, information and promotion measures, channels used, target groups, or messages conveyed. Cost per mille/Cost per thousand: Measure of the cost-efficiency of publications. Calculated by dividing the rate or specific advertisement cost by the circulation or number of readers Design: preparation of the promotion programming documents, including selection and hierarchy of information and promotion measures (with quantified goals at programme and measure levels, selected channels, messages to convey and quantified targets), how the programme and its measures will be monitored and evaluated, the total budget, and the budget for each measure. Effective evaluation (in the case of co-funded information and promotion programmes): evaluation which provides at measure/programme level clear and accurate information on the coverage of audience/ target groups and the extent to which they were reached, results and impacts in terms of improvement of the knowledge/perceptions of target groups on agricultural products and related themes, (new) market penetration, increased consumption or increased sales. An effective evaluation should draw lessons learned of the design and implementation of the measures/programmes. Evaluation recommendations should be taken into account during programme implementation (if Final Report November 2011 Glossary Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC on-going) or for the design of s programmes (if the programme is to be continued for a further period of three years). Effective monitoring (in the case of co-funded information and promotion programmes): monitoring that provides at measure or programme level clear and accurate information on implementation of the programme; i.e. funds spent, progress of information and promotion measures, channels used, duration, delivered outputs, and use of promotion material. Effectiveness: the extent to which objectives set have been achieved. EU added value: as specified in the guidelines AGRI/60787/2007-rev.5 (point 6.6), any benefits which will be derived from the programme at EU level and therefore justify part-financing by the EU EU promotion policy framework: the set of Regulations defining rules for promoting agricultural products on the internal market as well as on third country markets; i.e. Council Regulation (EC) No 3/2008 and Commission Regulation (EC) No 501/2008. The scheme for promoting agricultural products is part of the Common Agricultural Policy (CAP). EU co-funded programmes designed and implemented by Member States as well as promotion carried out at the Commission’s initiative are both financed by the European Agricultural Guarantee Fund (EAGF). Evaluation: as defined by the European Commission service (DG Budget-Evaluation Unit, Evaluating EU activities- A practical guide for the Commission service, July 2004, page 9), judgment of interventions according to their results and impacts and the needs they aim to satisfy. Guidelines: as defined by Council Regulation (EC) No 3/2008 (article 5 (1)), general indications, in particular concerning (a) objectives and targets to be reached; (b) one or more themes to be the subject of the measures selected; (c) the types of measures to be implemented; (d) the duration of programmes; (e) indicative distribution, by market and type of measure envisaged, of the amounts available for the EU’s financial contribution to programmes. As far as the internal market is concerned, these general indications are provided by Annex I (B) of Commission Regulation (EC) No 501/2008. In addition, the European Commission also published guidelines (Guidelines to be considered when assessing and managing part-financing programmes for promoting Community agricultural products, 1st July 2008) describing how to assess and manage co-funded programmes. According to Council Regulation (EC) No 3/2008 (article 5(2)), the EC may also adopt specific guidelines defining the strategy to be followed in proposals for information and promotion programmes in third country markets. Implementation requirements: as defined by the Regulation and the guidelines, requirements for MSs, including: provision of performance security; contract formats; payment approval, including approval of the final payment against reporting of work outputs, evaluation of results and financial statements; quarterly and annual reports; annual checks on at least 20% of the programmes completed in the previous year. Implementation: execution of the designed and approved programme, including carrying out information and promotion measures, and also monitoring and evaluating them. Final Report November 2011 Glossary Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC Implementing organisation (or implementing body): the organisation selected by the proposing organisation (or the Commission in the case of promotion carried out at the Commission’s initiative) to implement information and promotion measures. As defined by Council Regulation (EC) No 3/2008 (article 11(3)), this organisation shall have specialist knowledge of the products and markets concerned as well as the resources necessary to ensure that the measures are implemented as effectively as possible taking into account the EU dimension of the programme concerned. Internal market: European market, including the EU-27 Member States. List of themes and products: themes and products as defined in Annex I and Annex II of Commission Regulation (EC) No 501/2008; List of third countries: third countries as defined in Annex II of Commission Regulation (EC) 501/2008 Management Committee: the committee composed of Member States ministry representatives and EC promotion unit officials. During meetings, the Management Committee issues opinions on the draft Commission decision approving the programmes. Member States competent authorities: as defined in Commission Regulation (EC) No 501/2008 (article 2), competent authorities (or “competent national authorities”/ “competent bodies”) designated by and in each Member State for implementing the Regulation. Among other tasks, competent authorities are responsible for preselecting programmes before sending them to the EU Commission for selection for co-funding. Monitoring: as defined by the European Commission (DG Budget-Evaluation Unit, Evaluating EU activities- A practical guide for the Commission service, July 2004, page 10), the continuous and systematic process carried out during the duration of an intervention, which generates quantitative data on the implementation of the intervention, but not usually on its effects. Monitoring results should facilitate subsequent evaluation. Multi-country programme: a programme put forward by more than one Member State. As underlined in Commission Regulation (EC) 3/2008 (cf. article 8.1), priority shall be given to the programmes proposed by several Member States or providing for measures in several Member States or third countries. In this report, only programmes put forward by more than one Member State are called multi-country programmes. Programmes put forward by a single Member State but involving measures in more than one Member State are called multi-target programmes. Multi-product programme: a programme including at least one information and promotion measure for two or more agricultural products Multi-target programme: a programme put forward by a single Member State but involving measures in more than one Member State or Third Country National promotion initiatives: promotion campaigns on agricultural products and related themes that are financed by Member States without CAP support and financing from the European Agricultural Guarantee Fund (EAGF). Objectives: the objectives laid down in the Regulation (EC) No 3/2008; i.e. increased knowledge, enhanced image of EU agricultural products and to open new markets Final Report November 2011 Glossary Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC (specific objectives) with a view to increasing consumption of these products (global objective). Overall framework: the framework from the EU Commission, through Member States competent authorities, proposing organisations and implementing organisations used to manage EU co-funded information and promotion (I&P) programmes on internal and third country markets. The figure nearby illustrates the relationship between all the involved entities as far as the selection of the I&P programmes to be co-funded is concerned. Private promotion initiatives: promotion campaigns on agricultural products and related themes that are financed by the private sector. Programme: as defined by Commission Regulation (EC) 501/2008 (cf. article 1.2), a coherent set of operations of a scope sufficient to contribute towards improving information on, and sales of, the products concerned. Proposing organisation (or proposing body): organisation which submits a programme promoting agricultural product on internal or third country markets for pre-selection by the competent national authorities. If the programme is selected by the EU Commission, the proposing organisation is responsible for managing the implementation of the co-funded programme. Relevance: the extent to which an intervention’s objectives are relevant to needs, problems and issues; Relevant criteria: as defined by Council Regulation (EC) No 3/2008 (article 7 (1)), these criteria are defined by each MS for their pre-selection of the proposed programmes. For internal market programmes, this is governed by article 9(3) of Commission Regulation (EC) No 501/2008. For third country programmes, this is governed by article 9(2) of Commission Regulation (EC) No 501/2008, which requires the following criteria to be checked by MSs: (a) consistency between the strategies proposed and the objectives set; (b) the quality of the proposed measure; (c) the likely impact of measures in terms of increasing demand for the products concerned; (d) assurances that the proposing organisations are effective and representative; (e) the technical capacities of the proposed implementing body and assurances that it is sufficient. Rural development policy : Article 20 and 33 of Council Regulation (EC) 1698/2005 on support for rural development by the European Agricultural Fund for Rural Development (EAFRD) provide for support to producer groups for information and promotion activities for products under food quality schemes (Measure 133 of the Rural Development Programmes) Selection procedure: as defined by the Regulation and the guidelines, including requirements governing: calls for proposals launched by Member States and selection of programme proposals by Member States. Proposals should include: provisional detailed budgets and the types of expenditure; a strategic and marketing analysis; a coherent strategy with objectives, proposed measures with selected channels, messages, target groups and countries; duration and timetable for implementing measures; monitoring and evaluation considerations. The financial, economic and technical Final Report November 2011 Glossary Evaluation of promotion and information actions for agricultural products ADE-METIS - AGROTEC capacities of proposing organisation(s) and implementing organisation(s) should be demonstrated. Specific support : Article 68 of Council Regulation (EC) No 73/2009 of 19 January 2009 establishing common rules for direct support schemes for farmers under the Common Agricultural Policy and establishing certain support schemes for farmers Strategy: as defined in Annex I (B) of Commission Regulation (EC) No 501/2008, messages, target groups, channels and duration of programmes presented by agricultural product category. An overview of the situation providing general justification for the promotion and goals is also mentioned, by agricultural product category. In addition, four general principles shall be observed whatever the agricultural product category. The Regulation: Council Regulation (EC) No 3/2008 and Commission Regulation (EC) No 501/2008 Wine sector : Article 103p of Council Regulation (EC) No 1234/2007 of 22 October 2007 establishing a common organisation of agricultural markets and on specific provisions for certain agricultural products (Single Common Market Organisation Regulation) Without compromising necessary monitoring and evaluation requirements: without reducing or weakening the quality, value, or degree of precision of the required monitoring and evaluation requirements (cf. Evaluation Question 8). Final Report November 2011 Glossary Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Executive Summary Objective and scope of the evaluation This evaluation commissioned by the DG Agriculture and Rural Development of the European Commission aims at providing an overall independent assessment of its promotion policy. The objective of this evaluation is twofold: First, to analyse and synthesise the evaluation reports on promotion programmes carried out within the framework contracts (commissioned in 2006, which assessed the effectiveness of information and promotion programmes on the EU market and in third countries) and to draw horizontal conclusions and recommendations for promotion on the internal market and in third countries. Second, to examine the relevance and effectiveness of the EU information and promotion policy for agricultural products with respect to achieving the objectives laid down in Council Regulation (EC) No 3/2008 as well as its coherence with other promotion measures applied under the CAP. The scope of the evaluation concerns main instruments covered in Article 2 of Council Regulation (EC) No 3/20081, namely co-funded information and promotion programmes and Commission initiatives including high-level trade visits. The evaluation covers all EU Member States benefitting from EU co-financed promotion programmes. The Member States that are the main beneficiaries of promotion programmes and the Member States that take part in multi-country promotion programmes are examined in more depth. The examination period covers the time span from 2002 to 2010. The 2002-2008 period is covered by the synthesis of the previous framework contract evaluations; whereas promotion throughout 2008-2010 is subject to a further in-depth analysis based on current programmes. The evaluation examines the relevance and effectiveness of the above-mentioned measures with respect to the achievement of their objectives, as well as the management of programmes. Coherence of the above-mentioned measures with other promotional measures applied under the CAP, notably in the fruit and vegetable and the wine sectors, under the rural development policy, and under Article 68 of Council Regulation (EC) No 73/2009 is examined too. Complementarities with promotional policies and initiatives implemented by private actors and Member States are studied as well. 1 Council Regulation (EC) No 3/2008 of 17 December 2007 on information and promotion measures for agricultural products on the internal market and in third countries Final Report November 2011 i Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Evolution of EU promotion policy In the framework of the CAP, the Commission support to promotion and information on agricultural products has been evolving significantly since the early 1980s. From 2000 to 2007, these activities were ruled by two distinct regulations, one concerning the internal market and the other targeting third countries2. Since 2008, the two regulations have been merged in one single harmonised regulatory scheme (Council Regulation (EC) No 3/2008 and Commission Regulation (EC) No 501/2008) with no significant modification in terms of content, except an enhanced support to fruit and vegetables in schools and the introduction of poultry meat as eligible product following the avian influenza crises. Currently, the promotion policy includes a large number of eligible agricultural products and food products, at the first processing stage. The main objective of this scheme is to improve the image and raise the awareness on the quality and specific production methods of EU agricultural products in order to reverse static or declining consumption (i.e. fruit and vegetables, milk), expand the demand (i.e. olive oil, organic products) or open new markets, depending on the case. The European dimension of the initiative is justified by the fact that it supplements, reinforces, and has a multiplier effect on Member States’ and private actions. The scheme is based on cofinancing, usually up to a maximum of 50% of EC contribution3, with at least 20% of financial participation from the private sector and the remainder by the Member States concerned. Promotion actions must be generic and focusing on the intrinsic quality of products. They may cover public relations work, promotion and advertising as well as information campaigns. They have to be presented in the form of a programme submitted by a professional trade or branch organization in response to national calls for proposals. The programmes are pre-selected by the Member State and final selection is done by the European Commission. Over the recent years, a high number of submitted programmes (up to 57%) were rejected by the European Commission (DG AGRI), mainly for eligibility and quality reasons. To clarify requirements and expectations, DG AGRI has issued several guidelines since 2006, incorporating lessons learned from the experience and the framework contract evaluations. The yearly overall EC expenditure for promotion measures increased from minor amounts in 2002 to relatively stable amounts since 2007, fluctuating between €45m and €50m, the three quarters being roughly dedicated to internal markets and the rest to third countries. Regarding products and themes, six of them account for 78% of all EC expenditure from 2002 to 2010: fruit and vegetables (26%); dairy (15%); meat (11%); European quality schemes (PDO, PGI, TSG) (9%); wines (8%); and organic products (8%). As for the 2 Council Regulation (EC) No 2826/2000 of 19 December 2000 information and promotion actions for agricultural products on the internal market and Council Regulation (EC) No 2702/1999 of 14 December 1999 on measures to provide information on, and to promote, agricultural products in third countries 3 The EC contribution can be increased from 50 to 60% for actions to promote the consumption of fruit and vegetables targeted at children in educational establishments, and for information on responsible drinking patterns and harm linked to hazardous alcohol consumption Final Report November 2011 ii Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC repartition per country, 56% of expenditures by the Commission have been for programmes proposed by Italy (19%), France (14%), Spain (8%), Germany (8%) and Greece (7%). A further 16% has been allocated to multi-country programmes (common programmes put forward jointly by several Member States). Over the total of 458 programmes signed between 2001 and 2010, single proposer programmes targeting internal market in their own country are dominant. Multi-country programmes, which are prioritised in the selection process, represent less than 10% of the total number, with 41 programmes through 2001-2010, but 16% in terms of expenditure which shows that they are often of larger financial amount. Different promotion measures exist under the CAP other than the scheme governed by Council Regulation (EC) No 3/2008. Important ones are in the wine sector as well as in the fruit and vegetables sector, both under Council Regulation (EC) No 1234/2007 (Single Common Market Organisation) and measure 133 under the Rural Development Policy. They all have their own specificities, differing in terms of beneficiaries, targeted markets or possibility to mention brands. Best efforts are made to avoid overlaps and establish demarcation lines between these different schemes. Methodology The evaluation was developed in four phases: structuring, observing, analysing and judging. It combined both desk and field work. The descriptive chapter is mainly based on EU Regulations and documentation provided by DG AGRI on promotion programmes numbers and budgets. The comprehensive synthesis is based on 9 evaluation reports produced by the framework contract evaluations commissioned by DG AGRI (4 sectorspecific evaluations the Internal Market, 5 country evaluations of promotion in Third Countries)4. The replies to the Evaluation Questions (EQs) are the core of the study. They are based on elements gathered for the descriptive chapter, from the comprehensive synthesis and on a survey among Competent Bodies in all 27 Member States. An in-depth analysis of the promotion programmes adopted between 2008 and 2010 was done through the analysis of a sample of 15 programmes originating from 9 EU Member States5. Both programme document analysis and stakeholder interviews were performed for these programmes in all 9 EU Member States. The evaluation is organised around 12 evaluation questions which cover the following three themes: (i) policy relevance and effectiveness, (ii) management of information and promotion programmes and (iii) their coherence and complementarity with other initiatives. 4 5 The Four sector-specific evaluations on the internal market were structured according to the following product sectors and themes: 1) organic products (2006), wine (2007), fruit and vegetables (2007) and dairy (2008). Five evaluations in third countries were structured according to the following third-country markets: 1) USA and Canada (2006), Russia (2007); Japan (2007); Norway and Switzerland (2007) and China, India and South-East Asia (2008). France, Germany, Greece, Italy, the Netherlands, Poland, Portugal, Spain and the UK. Final Report November 2011 iii Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Conclusions Theme 1: Policy relevance and effectiveness The list of themes and products The current list of eligible products and themes includes a large number of agricultural products and agri-food products at the first processing stage, grouped under 15 products/groups of products and themes. The list is to a large extent inherited from the sectoral approach of the CAP in the eighties and nineties. Since the early 2000s, three criteria were guiding the integration of additional themes and products eligible for EU cofunding, namely (i) the fact that highly differentiated and valued products are typical or produced in a quality scheme, (ii) the need to handle markets or consumers confidence in individual sector crises and (iii) the potential export opportunities in third country markets. The current list of eligible themes and products is large, as well as heterogeneous reflecting the diversity of agricultural products (e.g. including fresh and processed products at the first stage of processing; single products (such as milk) or group of products (such as fruit and vegetables or organic products); “standard” products (such as olive oil) versus products under EU quality schemes). The list of themes and products is broadly relevant. Thanks to a large scope of eligible products, it contributes to enhancing the image and improving knowledge of EU agricultural products. The large scope does also allow responding to a high heterogeneity of situations and various trends (growing/stable or declining markets and the relative position of EU products, etc). However, the list of products and themes alone is not enough to achieve the objectives of the regulation. Although the principles of the policy are defined, an overall strategy is missing. The list of eligible third country markets As for the list of eligible third country markets, it covers almost all regions of the world, and as such is rather unfocused. This is not per se a problem for stakeholders (competent bodies and proposing organisations) since the list now provides flexibility for their actions. Most important trading partners for Europe are in the list, which is essential. In order to assess the relevance of this list of countries compared to the objectives of the Regulation, an indicative scoring system has been developed based on EU exported product trends to third countries, third countries consumption of covered products and their GDP trends. This indicative scoring system shows a large range of relevance among the listed countries. To arrive at the list of eligible third country markets the statistical approach needs to be completed by more qualitative criteria. On the other hand, the concept of geographical areas eligible for promotion actions, such as for instance Latin America, is not considered useful as such, except the flexibility it provides to Member States, because they encompass highly diverse national or sub-national realities. Final Report November 2011 iv Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC In other words, both the list of (i) themes and products and (ii) third countries eligible to the scheme are not in essence a hurdle to the achievement of the objectives laid down in the regulation because they are large and allow flexibility and adaptation to markets. Yet, they offer an unexploited possibility to define more structured strategies or to prioritise actions in programmes. Commission initiatives In line with the subsidiarity principle, the Commission may take the initiative in measures that are of Community interest or where no appropriate proposal has been submitted otherwise by Member States. On the internal market the Commission undertook initiatives in the organic sector throughout the 2005-09 period, with large visibility, which had a leverage effect on other promotion campaigns launched in Member States. There is little available information on the Commission’s initiative in third countries, namely for the European Authentic Tastes campaign (EAT) promoting European quality schemes (PDO, PGI, TSG and organic), implemented over the 2005-07 period. Although there was evidence of minor coordination problems, it also led to synergies with some cofunded programmes. Furthermore, the Commission organises on average once a year so-called high level trade visits to third countries that include participation in international events, mainly trade fairs. They aim to value a positive European image of agricultural products, facilitate further relations for national stakeholders and enhance business opportunities for exporters. The immediate feedback from these visits is positive, although the longer-term impact on effective business opportunities is complex and not yet assessed. Synergies between Commission initiatives and EU co-funded programmes may gain from enhanced communication between the different decision levels, in order to harmonize messages, coordinate agendas and increase efficiency possibly through the use of common tools. Multi-product and multi-country programmes There are three types of “multi-programmes”, namely multi-country programmes (put forward by more than one Member State), multi-product programmes and multi-target programmes. Multi-product and multi-country programmes are of particular importance for the EU promotion policy, because they bring EU added value such as economies of scale, leverage effects, wider target group reach, cooperation and socio-economic cohesion between countries. They therefore have the potential to bring an EU dimension to initiatives taken at lower levels. Although the EU promotion policy framework has encouraged multi-country programmes and multi-product programmes, they had limited occurrence so far. The number of proposals is hampered by the specific difficulties they face for design and implementation. Final Report November 2011 v Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The risk exists that multi-product and multi-country programmes be proposed for opportunist reasons such as increasing the chances to be selected by Commission services but without meaningful rationale behind. The main difficulty with multi-product approaches is to find adequate and relevant associations among the list of eligible products that give sense for promotion, but meaningful combinations exist not only for cheese and wine. Overall, difficulties to identify reliable partners, different rules among MS and different interpretation of guidelines and regulation, as well as language and cultural hurdles and administrative burden complicate the setting up and implementation of multi-country programmes. In order to overcome these management difficulties, strengthening the EU support to multi-country programmes might contribute to their development. Programmes involving measures in more than one Member State or in more than one third country (multi-target programmes) present less difficulties and are more widely and easily adopted. They have the advantage to penetrating new markets with sufficient critical mass, allowing for economies of scale and leverage effects. Evolution of effectiveness The analysis of improvement of programmes implemented since 2008 is based on the 15 sample programmes chosen for in depth analysis. However, as many of the programmes have only been operating for a year, there is limited evidence yet of the achievement of objectives. Good practices for programme design such as market analysis, definition of clear objectives, setting targets, etc. are widely supported and adopted practices among the proposing organisations of the analysed programmes, with some exceptions such as the justification of the choice of communication channels which should be based on the best possible efficiency criteria. This is reflected in an improving standard of selected proposals. Most of the proposing organisations and national competent bodies interviewed believe that the overall quality and effectiveness of the programmes is improving, also thanks to the experience gained over the years. But this trend is not translated into higher success rates at the level of the EC selection, characterized by important fluctuations without clear trend until the end of 2010. Indeed, there are still weaknesses in a number of submitted programme proposals, such as unclear or inconsistent activities, and lack of details in actions proposed and insufficient strategy especially for third country programmes. According to the Commission, applicants should better adapt their communication channels to the target group, with best possible cost efficiency and potential impact. Final Report November 2011 vi Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Theme 2: Management of information and promotion programmes Overall management set-up The overall management set-up, from the European Commission, to competent bodies in Member States (often Ministry) and proposing organisations (branch organisations) down to the implementing bodies (often advertising agencies), is clearly defined in the Regulations and further guidelines. The roles of each stakeholder in the overall management set-up are described in details. However, issues are met at two levels: 1) the two step selection process of programme proposals and 2) the implementation procedures. While the overall management structure is not questioned, both these issues impede the achievement of objectives of the Regulation. In the first case, the two step selection process, at Member State level (pre-selection) and at EC level (selection) is causing an important time spent on the procedure (over 6 months) and some degree of duplication in selection roles. This time span is considered long in the context of promotion. In the second case, implementation procedures from the European Commission down to the implementing organisations – including programme modifications, monitoring and evaluation, are heavy and time consuming. As far as multi-country programmes – programmes put forward by more than one Member State – are concerned, more time is needed to fulfil administrative and technical requirements (e.g. agreement on messages conveyed in the Member States implementing the programme) and lack of coordination between involved proposing organisations and competent bodies may slow down the implementation of the co-funded programme. Guidelines The available guidelines and accompanying documents are generally appreciated by the Member States, particularly for the design of programmes, even if some streamlining is possible in terms of quantity, as well as simplifications. On the other hand, guidance documents are reported as less adapted for the implementation of the programmes, particularly for multi-country programmes where clarifications are needed. There are no specific guidelines for designing and implementing programmes in third countries. Consequently proposing organisations tend to base these programmes on guidelines for the internal market, which may not be fully adapted to third countries. Some aspects of the guidelines, in particular in annexes of Commission Regulation (EC) No 501/2008 which give a short overview of the situation of the sector and mention the main target groups, messages and communication channels have not always been updated. On the other hand, the variety of guidelines and good practices in different binding or guiding documents may become confusing in the absence of a streamlining exercise. Final Report November 2011 vii Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Monitoring and evaluation reports In terms of reporting, a source of confusion is the distinction between monitoring and evaluation. First of all, reporting requirements are judged excessive (mainly but not only) by proposing organisations. Particularly the quarterly monitoring reports are judged too frequent, even if one should keep in mind that these reports are triggering the payments. On the other hand, the template for annual reports (essentially aggregates of quarterly monitoring reports) is largely oriented towards monitoring and hence not used as annual evaluation of the results obtained. Overall, the annual reports have facilitated the monitoring of promotion programmes but only partially contributed to effective evaluation. The format provided in the contracts does not include the explicit request of evaluation of the results obtained although the description of expenditure in the contracts mentions this possibility. The final evaluations, for which 5% of the budget can be earmarked, should provide relevant information on programme impact and achievements6. Administrative requirements for selecting and implementing programmes As already underlined above, the two-step programme selection process, based on shortlisting at the Member State level and final selection by the Commission services, is considered long by proposing organisations (3 months at Member State level and 3 to 4 months at EU level) and generates misunderstandings, particularly when the rejection rates are high at the Commission level (57% from July 2006 to end 2010) in a context of budget availability, which is often the case in this EC-supported scheme. While the Commission endeavours to undertake at its level an objective selection (eligibility check followed by quality assessment by external reviewers based on four main criteria: quality, European dimension, impact and cost effectiveness), the selection approaches in Member States vary. Member States check eligibility and quality of proposals, some of them using evaluation grids usually also based on four criteria (general interest, quality and effectiveness, EU dimension, cost/effectiveness). Some Member States do a careful preselection including support to applicants to respond to the Commission requirements while others tend to handover most (all) proposals received leaving the responsibility of selection to a more distant body. Furthermore, some selection requirements are too thorough, namely the detail of unit costs of activities throughout the programme. Overall, overlaps between roles at the two selection stages exist (both checking eligibility and quality) and sometimes insufficient transfer of information to the proposing organisations can generate misunderstandings. As far as implementation requirements are concerned, the main issues relate to the high reporting frequency, the details of unit costs, and the lack of flexibility in terms of budget 6 The final evaluations of the studied promotion programmes were not yet available, since these programmes were still in operational phase. Final Report November 2011 viii Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC transfers leading to administrative burden especially for multi-country programmes, as already mentioned under the overall framework. Theme 3: Coherence and complementarity with other CAP and national and private promotion initiatives Complementarities and synergies with national and private initiatives Complementarities and synergies between EU-co-funded information and promotion programmes on one hand and other national or private promotion initiatives on the other hand is a central preoccupation in this scheme. Indeed, as explicitly stated in the recitals of Council Regulation (EC) No 3/2008, the promotion policy seeks to supplement and reinforce schemes run by Member States. If such complementarities definitely exist according to stakeholders (competent bodies and proposing organisations), they are almost never the result of a well structured promotion strategy. They are so to say coincidental or contextual, as a result of a broad basis of shared grounds between programmes and initiatives such as the main messages and goals (healthy food, food quality, etc.), the target groups (e.g. schools) and the communication channels (common websites, food fairs, etc.). Two major constraints were identified to synergies and cooperation with the private sector though. This is the exclusion of brands from the scheme, as well as the restrictions to mention origins except in cases of recognised quality schemes. Overlaps between EU co-funded programmes and national/private initiatives are not reported as an issue. EU added value EU funding has an important leverage effect on generic, multi-country and multi-product programmes. Without EU co-funding, promotion programmes would have a much smaller scale and would be funded mainly by the private sector (PO). Due to this private funding, they would be brand-oriented and not generic. Some programmes would probably not take place at all, especially for third country promotion, where a minimum critical mass is needed in terms of budget, not often affordable without EU support for smaller operators. Co-funded multi-country programmes also generate an important leverage effect as several parties participate in a programme, allowing a higher impact. These elements emerge from the previous evaluation reports and interviews with proposing organisations and competent bodies in Member States. The transnational meetings that are organised by proposing organisations to elaborate multi-country programmes are mainly to coordinate the programmes and agree on their design (channels, messages, etc.). These meetings do however, as a side effect, contribute to the exchange of experience among organisations. Final Report November 2011 ix Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The I&P programmes are generally thought to improve the image of EU products and contribute to the development of positive connotations (e.g. high quality, safety, taste, etc.) according to framework contract evaluations and interviewed competent bodies. The specific contribution of multi-country programmes on the image is recognised by competent bodies and concerned proposing organisations. Promoting the intrinsic value of the products also has a positive effect overall on consumer behaviour. However, there is a lack of clear evidence as to the direct economic impact (e.g. exports and sales) of generic campaigns, working on the image and awareness. Coherence of I&P programmes with other CAP promotion measures Promotion of EU agricultural products can be financed in a horizontal way through Council Regulation (EC) No 3/2008, on a smaller scale for national or EU quality schemes through rural development programmes (RDPs) and in a sector-specific way for wine and fruits and vegetables7. The promotion and information scheme is coherent with these other CAP measures in terms of objectives, messages conveyed, groups targeted and channels used except the possibility of mentioning brands under certain conditions under the CMO (wine as well as fruit and vegetables), which makes an essential difference with Council Regulation (EC) No 3/2008. Regulations and implementation of CAP promotion measures focus on demarcation lines between each-other rather than actively building complementarities and synergies. More precisely, there are no overlaps between these initiatives either because most of them have explicit or implicit specificities in terms of types of beneficiaries or coverage, demarcating them appropriately or thanks to additional demarcation lines. Nevertheless, the coexistence of an increased number of promotion measures resulting from recent CMO reforms, with important financial allocations such as for the wine sector, could limit the overall efficiency of the promotion policy. Recommendations With a view to improve the EU promotion policy in light of the above-mentioned conclusions, the evaluation has come to the following recommendations: 7 The global objective of the scheme is to support demand and consumption of EU agricultural products. This should be mentioned explicitly in the regulation together with the potential benefits for the producers and the consumers. This is considered a necessary step in order to remove ambiguities and better shape promotion Several CAP measures provide support for promotion activities, namely measure 133 of Rural Development Programmes (Council Regulation (EC) No 1698/2005) for quality schemes promotion (€29m co-funding per year), the single CMO for wine and fruit and vegetables (Council Regulation (EC) No 1234/2007) with respectively promotion on Third Countries for wine (approximately €150-250m per year); and for fruit & vegetables in the framework of operational programmes (approximately €30-40m co-funding per year) and the school fruit scheme (€90m co-funding per year). This has to be compared to around €50m co-funding per year from Council Regulation (EC) No 3/2008 Final Report November 2011 x Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC strategies. The specific objectives and targets might differ between the Internal Market on one hand and Third Countries on the other hand. While the lists of eligible countries, regions, products and themes for promotion actions may remain large and even be enlarged to enable flexibility, it should be associated to a European promotion strategy that gives focus and priorities to ensure contribution to the global objective. A European strategy for promotion of agricultural products should be defined and should encompass all promotional measures of the CAP in order to ensure internal coherence. It could be differentiated between the Internal Market and Third Countries. Market trends should be included as a dimension of the strategy. The evaluation proposed an approach to position products/themes compared to market trends. This positioning could be a first basis to set priorities. The strategy should be revised periodically depending on the changes of priorities. A set of further criteria could be used to identify priorities such as the European dimension, European production standard, the need to address crises or sectoral difficulties etc. In addition to the development of a European promotion strategy, Member States should be asked to define their own national strategies, clarifying their priorities in terms of products/themes and their potential public support to promotion. National strategies should include national and regional support but also potential EU cofunding including that from rural development, from some sectors under the single CMO etc. This step would maximize potential for developing synergies and complementarities of EU promotion actions with the actions of private sector. Given the EU value of multi-country programmes (common programmes set-up by more than one Member State) and their associated constraints (difficulties in design and implementation of such programmes); the EU should strengthen support to such programmes. This incentive could take the form of an additional share of overheads and fees of implementing bodies to support additional management and coordination costs. Through an attractive budget and by simplifying the administrative requirements of their implementation (e.g. clarifying the role of the coordinating proposing organisations), and by clarifying different interpretation of guidelines in different Member States with clear responses (e.g. through an EU exchange platform below), such programmes could be made more attractive in the future. In terms of the selection of programmes, the overall procedure could remain unchanged with essentially a clearer distinction of roles and a better communication between the EC and Member States. Competent Bodies could take care of eligibility checks as well as other verifiable criteria (e.g. market analysis, relevance to national strategy, duration). At the end of this first step, they should provide sufficient documentation and justification to the EC before the second step. The Commission on the other hand would be better suited to assess the potential impact of programmes, their EU dimension and their compliance with the European strategy, in the second step. A scoring system would be used at both levels in order to underpin final selection. Based on these elements, the selection procedures could be improved in terms of legibility and coherence. Final Report November 2011 xi Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The administrative burden of the different types of programmes could be alleviated by reducing the level of details required in the proposals in terms of unit costs of actions for last years of programme implementation, adding flexibility to implementation procedures (e.g. facilitating budget transfers over 10% within a programme if this is well justified and improves efficiency) and reducing monitoring and reporting frequency and requirements, limited to annual (or bi-annual) progress reports that need to include an internal evaluation of results obtained so far (for annual reports) and retrospective evaluations at the end of the programme. Streamline the guidelines into one document which would not be part of the Regulation and which would encompass EU priorities and guidance for the design (as currently Annex I of Council Regulation (EC) No 501/2008) and operational requirements for implementation and evaluation. The leverage effect of co-funded programmes could be improved if the rules governing the reuse of co-funded communication ‘material’ (e.g. slogans, messages...) were made more explicit in order to make this 'material', such as slogans, accessible also to the private sector under specified conditions. To ensure a European return of generic campaigns and enhance visibility of EU promotion, introduction of a European identification to the promotion programmes, including visual and/or content elements, should be considered. Finally, to ensure improved procedures and enhanced mutual understanding among all parties involved, it is suggested to set up a permanent EU Exchange Platform on the Promotion and Information Actions for Agricultural Products. This Platform would not interfere with the existing decision mechanisms involving the Commission and the Management Committee which would remain unchanged. Its main role and objective would be to maintain an active community of stakeholders interacting between them, as well as with the competent authorities in Member States and the European Commission: top-down and bottom-up exchanges of views, networking of stakeholders, transfer of know-how and good practices, formulation of proposals and suggestions. Some of main deliverables could be: a website, workshops, technical documents, catalogues of good practices, lists of FAQs, a helpdesk, etc. Final Report November 2011 xii Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Introduction This preliminary final deliverable presents the results of the work carried out during the desk phase and the field phases of the Evaluation of the Information and Promotion Actions for Agricultural Products. As requested by the Terms of reference, this report includes replies to the evaluation questions and a closing chapter of overall conclusions and recommendations of the evaluation. The report is structured as follows: Section 1: Descriptive chapter: this section presents the EU promotion policy framework from 2000 onwards, including its rationale, budgetary elements, and administrative requirements. Other promotional measures applied under the CAP and examples of promotion policies of other countries are also presented. Section 2: Methodology: the approach used for data collection is presented under this section. It details the criteria for selection of the sample of 9 Member States and of 15 information and promotion programmes to be part of the scope of this evaluation. The way data were collected during the desk and the field phases is also explained. Section 3: Comprehensive synthesis: this section summarises all the lessons learnt through the nine evaluation studies carried out under the two framework contracts for studies to assess the effectiveness of information and promotion programmes for agricultural products on the internal market and in third countries as commissioned by DG AGRI in 2006. At the end, the synthesis includes horizontal conclusions and recommendations for both promotion programmes on the internal market and in third countries. Section 4: Replies to the evaluation questions: the reconstructed intervention logic of information and promotion actions for agricultural products financed by the European Commission is presented and discussed as a first step in this section. The answers to the evaluation questions are then structured following the evaluation’s three themes: Theme 1 – Policy relevance and effectiveness, Theme 2 – Management of information and promotion programmes, and Theme 3 – Coherence and complementarities with other CAP and national and private promotion initiatives. The answer to each question includes explanations about the rationale of the question, the approach and judgment criteria used to structure the reply, the limitations of the approach, as well as the detailed answer. Section 5: Closing chapter: this chapter presents the overall conclusions and recommendations of the evaluation. Final Report November 2011 Page 1 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 1 Descriptive chapter: Overview of the EU policy framework 1.1 Development of EU promotion policy and its implementation from 2002 onwards 1.1.1 Promotion measures used, change in role and importance The EU has been implementing and co-financing information and promotion measures for agricultural products in a horizontal way since 1999, bringing together a wide range of product specific initiatives that had been supported as part of the CAP support to different products since the early 1980s. These earlier initiatives were motivated by the desire to increase demand (consumption and outlet), which would benefit producers by increasing sales. Rationale for EU promotion policy All three Council regulations namely (EC) No 2826/2000 on the internal market, (EC) No 2702/1999 in third countries and the latest merged Council Regulation (EC) No 3/2008 consider that the promotion policy allows to: supplement and reinforce the schemes run by Member States by boosting product image in the eyes of consumers in the EU and in third countries, in particular as regards the quality, nutritional value and safety of foodstuffs and the methods of production; open up new markets in third countries; and have a multiplier effect on national and private initiatives. This is supported, on the internal market, by specific objectives for each product and theme, defined initially in Commission Regulation (EC) No 94/2002, then in Commission Regulation (EC) No 1071/2005 and updated in Commission Regulation (EC) No 501/2008. The specific objectives are related to the market situation for the product. Promotion of agricultural products from 2000 until 2007 The support for promotional measures of agricultural products was originally governed by two Regulations: Council Regulation (EC) No 2826/2000 for internal market activity; and Council Regulation (EC) No 2702/1999 for third countries. Final Report November 2011 Page 3 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Figure 1: Summary of Key Regulations Internal Market Council Regulation (EC) No 2826/2000 - New Regulation for internal market giving more power to Member States. Amended by 2060/2004 and 1182/2007. Implemented by Commission Regulations (EC) Nos 94/2002 and 1071/2005 Third Country Markets Combined Council Regulation (EC) No 3/2008 Revised Regulation consolidating internal and third country markets regulations Implemented by Commission Regulation (EC) No 501/2008 Council Regulation (EC) No 2702/1999 - New Regulation for third country markets with more products and trade missions. Amended by 2060/2004. Implemented by Commission Regulations (EC) No 2879/2000, 67/2005 and 1346/2005 Internal Market Council Regulation (EC) No 2826/2000 governed the information and promotion programmes for agricultural products on the internal market. These programmes aim at providing to the EU’s consumers the most appropriate and complete information on agricultural products and their methods of production by stimulating and reinforcing the measures taken by the Member States through co-financing of programmes on the internal market. Compared to previous Regulations (1994-2000) a number of new features were introduced: the initiative and management responsibilities of Member States and professional organisations have been increased compared to the period 1994-2000, whereas management was previously centralised in the European Commission; key new features of the period from 2000 onwards include the growing contribution from Member States which have to co-finance campaigns alongside with the professional organisations; Member States have a responsibility for programme selection and their evaluation; the management of these campaigns is decentralised and entirely the responsibility of professional organisations and Member States 8; whereas programmes previously were financed mainly by the EU, a co-financing rule has been introduced: maximum 50% by the EU’s financial participation; at least 20% by proposing organisations (POs) and the remainder by the Member States concerned; and programmes are not targeted on specific products but emphasise general characteristics and common topics: quality, safety, labelling, specific production methods, respect for animal welfare and the environment. 0F 8 The 1994-2000 promotion policy presents considerable differences with the period from 2000 onwards. Several products or themes and their campaigns were directly or indirectly managed by the Commission (olive oil, flax, beef and veal) and financed by the Commission only. Member States were only involved to a limited extend and for a small number of products. Source : Evaluation of the Community policy for the promotion of agricultural products, UBM Consulting, November 2002 Final Report November 2011 Page 4 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The detailed rules for the internal market were laid down in Commission Regulation (EC) No 94/2002 of 18 January 2002 and its amendment 1071/2005. These were replaced by Commission Regulation (EC) No 501/2008 when the internal and external market operations were consolidated. These Regulations provide the list of products and themes covered, along with an overview of the situation, goals, main targets, messages, channels, duration and indicative budgets. Table 1 summarises this for the largest products and themes. Table 1: Main Goals and Targets by Products and Themes Increased consumption and outlets Import Substitution Improved consumer awareness of production conditions Increased quality awareness Benefits on a balanced diet Fresh fruit ++ ++ +++ and vegetables Processed ++ +++ + fruit and vegetables Olives +++ + Milk +++ + + Wines +++ + +++ +++ Organic +++ ++ ++ + Products PDO+ +++ +++ +++ Quality +++ +++ +++ scheme Meat +++: primary focus/goal; ++: significant mention; +: minor or implicit goal. Source: ADE based on Commission Regulation (EC) No 501/2008 Improved Environmental impact +++ Commission Regulation 1071/2005 provided some complements on goals for the following products/themes: for wines, the 2005 regulation added the goal of increasing consumption of EU wines; for Protected Designation of Origin (PDO), Protected Geographical Indication (PGI) & Traditional Specialty Guaranteed (TSG) (PDO, PGI & TSG are also sometimes quoted as “PDO+”), the 2005 regulation added goals of providing information and enhancing knowledge of logos; for organic products, the 2005 regulation added goals of encouraging producers to participate and improving knowledge of environmental and animal welfare benefits. In addition, the same regulation added the following products as being eligible for promotional measures in 2005: fibre flax, olive oil and table olives, seed oils, quality meat, honey and beekeeping products and graphic symbol for the outermost regions. Final Report November 2011 Page 5 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Third Country Markets Council Regulation (EC) No 2702/1999 governed the information and promotion programmes for agricultural products on third country markets. These programmes aim at promoting the image of EU agricultural products on international markets as regards food quality and safety in particular, with a view to helping open up new markets for EU agricultural products and having a multiplier effect on Member States and private initiatives. Compared to previous Regulations a number of new features were introduced since 2000: the list of products to be covered by promotional measures has been enlarged from two products (olive oil and flax) to twelve agricultural products; and high-level trade visits and studies of new markets with a view to expanding market outlets have been added to the list of measures providing information on, or promoting, agricultural products. The detailed rules for third country promotion were laid down in Commission Regulation (EC) No 2879/2000 and its amendments. This Regulation provides the list of third-country markets in which promotional measures may be carried out (Australia; China; India; Japan; New Zealand; Norway; Russia; South Africa; South Korea; Switzerland; Turkey; Ukraine and the Balkan countries of South-East Europe) along with eligible regions (North Africa, North America, Latin America, South-East Asia and the Middle East). It also specified the list of products that were eligible, which included the same products as for the internal market, plus products processed from cereals and rice and spirits with a geographical indication and excluding honey, seed oils, fibre flax and products from outermost regions. Regulation 2879/2000 was replaced by Commission Regulation 67/2005 and 1347/2005 with a view to introducing amendments in the light of experience gained during first years of Regulation implementation. Evaluation of EU co-funded I&P programmes on internal and third country markets implemented between 2002 and 2008 In 2006 the Promotion Unit of the Directorate-General for Agriculture and Rural Development (DG AGRI) commissioned two framework contracts for evaluation studies to assess the effectiveness of information and promotion programmes implemented between 2002 and 2008 on the internal market and in third countries. In total nine evaluation studies were carried out, respectively four related to the internal market (organic products, wine, fruit and vegetables, and dairy products) and five related to third countries (USA and Canada; Switzerland and Norway; Russia; Japan and India; China and South-East Asia). These evaluations are synthesised in Section 3. The lessons learnt were taken into account in amending the Regulation framework for the I&P programmes. Final Report November 2011 Page 6 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Promotion of agricultural products since 2008 At the beginning of 2008 the regulations for internal market and third countries were consolidated into a single Council Regulation 3/2008 without significantly modifying the content of these Regulations (see above). The detailed rules for the application of this Regulation are laid down in Commission Regulation 501/2008. By merging the regulations, the requirement to include plans for evaluating the attainment of objectives that had been introduced for internal market programmes in 2005, was extended to third country programmes. Each programme proposal should include an evaluation proposal of its information and promotion activities in terms of attainment of its objectives. Table 2 summarises the goals, along with the indicative annual budget for the main products and themes. There has been little change in the goals, except for the following details: for fruit and vegetables, the 2008 regulation adds targeting schools; Poultry meat was added as eligible product in 2008 (501/2008) as response to the avian influenza crisis. Table 2: Product 9 & Budget 10 Fresh fruit and vegetables (€10m) Processed fruit and vegetables (€2m) Olive Oil & Olives (€7m) Milk & Milk Products (€4m) Overview of the situation and goals (objectives) for the main products and themes with indicative annual budget Overview of the situation 1F Goal 2F Wines under EU quality schemes (€3m) Organic farming (€3m) production is increasing consumption is healthy, but static and declining for young people demand is gradually increasing, but there is strong import competition supplies are increasing and traditional markets have little additional potential consumption of liquid milk is declining, partly compensated for by increased demand for milk products consumption is static EU & international production is strong understanding is limited EU Action Plan for Organic Products improve image of product increase regular consumption, especially amongst young people modernise the image encourage consumption expand demand in newer markets consolidate demand in old markets increase consumption, or to prevent decline (including liquid milk and milk products) especially for young people as future adult consumers increase consumption of EU wines inform consumers of the quality and variety of EU wine increase consumption awareness of labelling and benefits encourage producers, processors, retailer groups to convert to this 9 The Regulations also provide for egg labelling (€2m), seed oils (€2m), fibre flax (€1m), honey (€1m), outermost regions (€1m), ornamental horticulture (€3m) and poultry. The market conditions for these products vary and the goals include a similar range to those in the table above. 10 Budgets are presented in the regulations as indicative annual budgets. Final Report November 2011 Page 7 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Product 9 & Budget 10 Overview of the situation 1F Goal 2F PDOs, PGIs and TSGs (€3m) Quality Meat (€4m) campaigns to promote groups of schemes, which are central to the CAP quality theme declining in consumer confidence production method and to sell these products increase demand improve understanding of schemes encourage farmers to set up schemes informing consumers of the benefits of meat from quality schemes Source: ADE based on Commission Regulation (EC) No 501/2008 Several changes have already taken place within the 2000-2008 period as described above. In addition, Council Regulation (EC) 2060/2004 amending Council Regulation (EC) No 2702/1999 and (EC) No 2826/2000 already sets important steps about the harmonisation of the regulations concerning the internal market and that regarding third country markets. It includes a harmonised approach regarding programme submission and selection, the possibility given to proposing organisations to implement parts of the programmes themselves and to select implementing bodies after selection, as well as a change of the share of EU contribution and responsibility of conformity check. In addition to the changes in objectives for products, the merger of regulations in 2008 meant that third country programmes were subject to the requirements for evaluation that were introduced for internal market programmes in 2005. To summarize, there have been very limited changes between the period 2000-2008 and the current period since 2008 onwards. Major changes have taken place between 1994-2000 and within the 2000-2008 period, harmonisation of both regulations was introduced since 2004 (2060/2004). The following changes were explicitly identified: According to Council Regulation (EC) N° 3/2008 article 5: specific attention should be paid to promotion measures of fresh fruit and vegetables intended for children in schools; in parallel, the maximum share of co-funding from the EU was increased from 50% to 60% for those particular promotion measures (art.13.2); in 2009, amendments to the Regulation enlarged the 60% co-funding rate from the EU to information on responsible drinking patterns and harm linked to hazardous alcohol consumption ; The amended designation of EU wines, (replacing quality wines produced in a specified region (PSR) and table wines with a geographical indication by PDO, PGI and wines with a designation of wine grape variety; Introduction of poultry meat as eligible product in annex II of Commission Regulation (EC) No 501/2008 in order to gain consumer confidence and consumption following the avian influenza crises. The requirement for Proposing Organisations to submit an annual evaluation report of the programme. Final Report November 2011 Page 8 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC In addition, Council Regulation (EC) 153/2009 allows Member States to accept an international organisation as implementing body in particular when the programme regards the promotion of olive oil and table olives in third countries. Finally, in response to the economic difficulties of the milk sector in 2009, Commission Regulation (EC) 698/2009 was adopted allowing derogating from the usual timetable for 2009. It allowed proposing organisations to introduce promotion programmes by an especially fast procedure 11. 3F 1.1.2 Budgetary elements The budgetary elements of information and promotion actions (i.e. total expenditure, expenditure per country, per product, etc.) are detailed below. This chapter presents data on expenditures by the Commission. Total Expenditure. Total EC expenditure from 2002 to 2010 was €298.2m, (with a provisional budget of around €58m for 2011). Annual expenditure grew until 2008. During the last three years since 2007, it has been between €45m and €50m, as shown in Figure 2. Figure 2: Total EC Expenditure for Promotion Measures on the Internal Market and in Third Countries (€m) Source: DG Agri Expenditure Data. 11 A time span of 6 weeks (31 October – 15 December) is foreseen between submission of programmes from Member States to the Commission and final decision of the Commission Final Report November 2011 Page 9 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Expenditure by Product Six products accounted for 78% of all commitments from 2002 to 2010: fruit and vegetables (26%); dairy (15%); meat (11%); PDO+ (9%); wines (8%); and organic products (8%). The other major products are: multi-product programmes that promote more than one type of product (7%); olives (5%); and ornamental horticulture (5%). The remainder (other) was accounted for by egg labelling, honey, oils and outermost regions. Promotion of milk and milk products has varied between 10% and 20% of total spending as shown in figure 3 reaching about €10m in 2010. Fruit and vegetables grew rapidly in the first year and then varied between €8m and €12m for the remaining years. Meat shows a maximum in the years 2007-2009. Other products have remained fairly stable, with some variation from year to year, showing however and increasing trend over the period. The evolution of spending share is presented in Table 3. Table 3: Product Dairy Fruit and vegetables Horticulture Meat Olive Organic PDO+ Wine Other Multi Total 2002 0.2 0.2 0.0 0.1 0.0 0.0 0.2 0.1 0.1 0.3 1.0 EC Expenditure by Product/Theme in €m 2003 2.3 8.3 0.6 2.0 0.0 0.7 0.4 0.9 0.3 1.4 16.8 2004 3.6 11.5 1.4 3.2 0.2 1.9 1.5 2.1 0.4 1.3 27.1 2005 2.5 8.8 1.8 2.8 0.8 3.5 2.7 2.4 1.2 3.0 29.5 2006 5.8 8.2 2.6 2.1 1.2 3.8 2.8 3.0 2.1 2.4 33.9 2007 5.8 9.4 3.7 6.6 3.0 4.7 5.3 3.5 3.0 3.0 47.9 2008 7.3 11.4 3.2 7.5 1.4 4.5 5.2 4.0 2.2 2.8 49.4 2009 6.2 11.5 2.2 5.5 2.1 3.3 5.8 3.6 3.4 2.8 46.2 2010 10.4 10.8 0.9 3.6 3.1 3.5 4.2 4.0 3.2 2.6 46.4 Total 44.0 80.0 16.2 33.3 11.8 25.7 28.2 23.6 15.8 19.5 298.2 Source: DG Agri expenditure data Figure 3: EC Expenditure by Product/Theme as Share of Total Expenditure Source: DG Agri expenditure data Final Report November 2011 Page 10 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Expenditure by Country As shown in Figure below, 56% of expenditure by the Commission has been for programmes proposed by Italy (19%), France (14%), Spain (8%, Germany (8%) and Greece (7%). A further 15% has gone to multi-country proposals. Figure 4: EC Expenditure by Proposer Country Source: DG Agri expenditure data Note: the graph includes only countries where EC spending to date on I&P has been more than €5m Table 4: Country EC Expenditure by Major Implementing Member States on Key Products/Themes (€m 2002-2010) Dairy Fr&Veg Meat Olive Organic PDO+ Wine Other Multi Total 1.5 2.5 1.3 Austria 4.0 2.1 0.3 Belgium 9.1 2.2 6.1 Germany 0.7 11.5 2.4 Spain 2.3 16.6 2.8 France 2.0 4.5 1.3 Greece 2.8 9.1 1.9 Italy 0.3 4.1 4.9 Netherlands 4.2 1.7 1.7 Poland 2.6 0.3 0.0 Portugal 5.1 0.0 2.7 UK 6.2 2.4 2.3 Other 3.3 23.1 5.6 Multi 44.0 80.0 33.3 Total Source: DG Agri expenditure data Final Report 0.0 0.0 0.0 1.1 0.3 5.5 4.9 0.0 0.0 0.0 0.0 0.0 0.0 11.8 3.4 0.9 1.3 1.0 5.5 0.3 8.0 0.5 1.5 0.0 0.0 2.1 1.3 25.7 November 2011 0.0 0.1 0.6 1.5 7.9 4.4 10.3 0.0 0.7 0.0 0.0 0.0 2.6 28.2 0.2 0.0 1.4 1.8 0.6 1.4 6.6 0.0 0.0 4.6 0.0 2.9 4.1 23.6 2.1 1.6 4.4 1.7 6.7 1.2 4.4 5.5 0.3 0.1 1.2 2.1 0.6 32.1 0.0 0.0 1.2 1.4 0.0 0.0 6.6 1.4 0.0 2.0 0.0 1.0 6.0 19.5 10.9 8.9 26.2 23.1 42.7 20.5 54.7 16.6 10.2 9.7 9.0 19.1 46.6 298.2 Page 11 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC There is a wide variation in the extent to which proposing countries are involved in programmes that are organised by many countries and that target many countries. Figure 5 presents the type of programme that each country is involved in, for all countries with more than €10m total expenditure, using the following terminology: The black solid bars are single proposer programmes in their own country. These are the dominant programmes for Austria and Germany and are also important in Belgium, France, Poland and the UK. The dark grey solid bars are single proposer multi-target internal market programmes and are important for Spain, Greece, Italy and Portugal. The light grey solid bars are participation by countries in multi-proposer multi-target internal market programmes. These are relatively small, but are significant for Belgium and the Netherlands and, to a lesser extent, for France and the UK. The dark wavy bars are single proposer programmes in third countries and are important for Greece, Italy, Poland and Portugal. The light wavy bars are multi proposer programmes in third countries. These are less important, but are significant for France and Portugal. Figure 5: Expenditure by Target Country Source: DG Agri expenditure data Final Report November 2011 Page 12 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Distribution of Expenditure per Member State This section assesses the allocation of funding per Member State. Two perspectives are considered, expenditure per farmer and per inhabitant. According to Eurostat and to the DG Agri data, and as shown in Figure 6, the programmes have disbursed a total of €0.61 per capita and €21.5 per farmer across the EU since the year 2000. Because there are differences in farm size amongst Member States, it is also useful to compare expenditure per agricultural labour unit (ALU). Figure 6 presents the average annual expenditure per capita, per farm holding and per ALU and shows that most countries were fairly close to the average, with some large exceptions. Expenditure per farm holding and per ALU is high for Belgium and the Netherlands and, to a lesser extent, for France, for Germany and Austria. Expenditure per capita is high for Greece and low for Germany and the UK. Poland’s spending only began in 2005 and its amounts are therefore divided by a smaller number of years than the other countries. The conclusions from the graph show that the main absolute beneficiary countries of the I&P programmes (i.e. Italy, Spain and France) do not generally receive much greater than average funding per capita or per farmer. Figure 6: Average annual expenditure per capita per farm holding and per Agricultural Labour Unit (2002 – beginning 2011) Source: DG Agri expenditure data; population and number of farmers from Eurostat 12. Note: ALU = Agricultural Labour Unit, which provides an alternative measure of farmer beneficiaries. 4F 12 Eurostat data provides bulk data on the numbers of farmers and thus no information on the numbers effectively concerned by promotion programmes. Indeed, data does not distinguish between professional farmers and other which introduces a severe bias especially for the southern MS. Final Report November 2011 Page 13 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Number of programmes A total of 458 programme contracts were signed between 2001 and 2010 (Figure 7), of which 23 were abandoned before any expenditure had taken place. There are 131 programmes at various stages of operation and 304 programmes have closed. Figure 8 shows the number of programmes that were active in each year. The number increased over the first five years, to over 70 in 2005, and has since declined to about 40 until 2010, when less than 30 were signed. Figure 7: Number of Programme agreements signed in each year Source: DG Agri data, assuming that the date in the contract name reflects the date of signature of the contract Figure 8: Number of Active Programmes by country Source: DG Agri expenditure data Final Report November 2011 Page 14 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 1.1.3 Development of administrative requirements (selection and implementation) The scope and balance of the I&P programmes is defined by Regulations (EC) No 3/2008 and (EC) No 501/2008. These Regulations specify the products and themes that can be covered and the strategy and guidelines for each product and theme. The list of products and themes is revised by the Commission every two years, in line with the criteria defined under article 3 of Council Regulation No 3/2008. The I&P programme cycle starts with calls for proposals issued by each interested Member State’s Competent Body (typically a Ministry of Agriculture). Two calls are issued each year, one for internal and one for third country markets. Proposals are prepared by Proposing Organisations (typically trade organisations) and normally specify the Implementing Bodies (typically professional bodies with experience). The proposals are submitted to the Member States Competent Bodies who select a shortlist and score the programmes to be presented to DG Agri. The Commission, DG Agri, first checks eligibility and then makes the final selection, and informs the management committee. The latter is composed of the representatives of the Member States and EC promotion unit officials. The committee issues opinions on the draft Commission decision. Programmes are implemented by Member States, which issue standard contracts to Proposing Organisations, who then issue sub-contracts to Implementing Bodies. Financial disbursement starts with an advance of 30% of the contract and then continues with quarterly disbursements, based on quarterly monitoring reports, assessed by a Monitoring Group, chaired by the Member States Competent Body. DG Agri also monitors expenditure and checks that claims are not above budget. Proposing Organisations are required to submit an annual evaluation report and payment of the final balance is subject to approval of the final annual evaluation report. The Commission is also assisted by an Advisory Group which gathers, on invitation of the Commission, representatives of various social and economic interests in order to discuss policy issues. Figure 9 provides the overall framework for the institutional arrangements of I&P programme cycle. Final Report November 2011 Page 15 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Figure 9: Institutional Arrangements Source: ADE, 2011 Budgeting. Regulation (EC) No 501/2008 gives indicative annual budgets for each of the products and themes. The funding rules require the Proposing Organisations to provide at least 20% of funds and the Commission not more than 50% 13, with Member States making up the balance of funding. The Commission Regulations have included indicative annual budgets per product or theme for the EU contribution. These apply across the whole EU and there is no indicative budget per proposing country. 5F Proposals. According to Regulation (EC) No 3/2008, the annual proposal process starts with calls for proposals, prepared by the competent national bodies of Member States and announced publicly. The competent bodies decide which themes and markets to include in their call for proposals. Proposing organisations in Member States prepare proposals and submit these to Member State governments by 30 November for internal market and 31 March for third country programmes. Member States must submit their shortlist of programmes to the Commission by 15 February for internal market programmes and by 30 June for third country operations. Guidelines. The Commission provides three main types of guidelines to design and implement I&P co-funded programmes: through Regulation (EC) No 3/2008, through Regulation (EC) No 501/2008, as well as through document AGRI/60787/2007, which was released on 1st July 2008 (fifth version).This document, available on DG Agri Website on promotion of European farm products, provides applicants to the co-funding of I&P 13 As already mentioned (under section 1.1.1), the share of the Commission can be up to 60% for fruit and vegetables in schools and for moderate consumption of alcohol. Final Report November 2011 Page 16 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC programmes with guidelines to be taken into account while preparing, evaluating and managing co-funded programmes. The guidelines to applicants (AGRI/60787/2007) cover the following subjects 14 grouped by topics. 6F In terms of contents and justification of the programme: the requirements for a strategic market analysis; details of the target markets and groups; objectives and relevance to the Regulation objectives, referring to at least one of the objectives listed in the Regulation; content, including type of measure (which may be subject to amendment in the light of evaluation conclusions); About the messages: clarification that any claims of health benefits are based on objective scientific evidence; clarification that all campaigns are based on the ‘intrinsic quality’ of products, that any mention of country or region of origin is secondary to the main message of promotion and that no specific commercial interests (such as individual brands) are promoted; About the detailed content: a detailed timetable for up to three years; a provisional budget with sufficient detail for meaningful monitoring and details of funding contributions, including definitions of the costs for each activities over the period of the project (3 years in general) and the main outputs from these activities, such as improvements in consumer awareness, new market outlets, involvement of professionals; confirmation that arrangements for monitoring and evaluation are sufficient, including the production of a final evaluation report; the extent to which the programme will generate benefits at a EU level; About the applicants: details of the constitution, experience and legal and financial status of the proposing organisation; details of any implementing bodies (which must normally handle at least 50% of the work), including their financial and technical capacity and confirmation that they have been selected by competitive, transparent and non-discriminatory tendering; clarification of whether the programme is a continuation of an earlier programme; for multi-country applications, evidence of cooperation amongst the Member States. 14 Many of these elements are in line with the good practices identified under the comprehensive synthesis and used in the fifth evaluation question. Final Report November 2011 Page 17 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Commission Regulation (EC) No 501/2008 contains four further subjects in the guidelines (Annex I B): Multi-country programmes should have coordinated strategies, actions and messages; Programmes should have sufficient scope to have a significant impact on the target markets and should preferable be multi-annual. They may also be implemented in more than one Member State; The messages should provide objective information about the intrinsic quality, nutritional value, production methods and/or environmental impact of the products or themes; and Key messages should be of interest to consumers, professionals and/or the trade. While designing their programme proposals, applicants have to follow a specific structure proposed by the European Commission. This structure, which relates to key information highlighted above, is provided in document Ref. ARES (2009)347563-27/11/2009 “Application form for promotion programmes part financed by the EU”. This document provides Member States with the template to be used to prepare the submission of the I&P programme to be co-financed. An explanatory note on various points of the application form is also available in that document. In addition to all the official guidelines, the European Commission has released in 2009 a document with clarifications following to further questions asked by Member States. These clarifications are provided in document AGRI-64545-2007 (fourth version) called “Positions and interpretations in connection with implementation of promotion and information programmes”, of 23rd November 2009. Selection. Selection is a two stage process, with Member States presenting a shortlist, based on eligibility ; relevance ; and value for money The Commission is making the final selection based on their eligibility and quality criteria. The guidelines imply that the Commission first checks that applications are eligible (i.e. have met all the requirements) and then gives priority to those that are multi-country in nature or providing measures in several Member States or third countries and those that demonstrate complementarity with other programmes funded by Member States or the private sector. Member States are required to fill in a scoring grid with the following weights: relevance to market (20); relevance to target group (10); coherence (10); scope and coverage of actions (10); quality of messages (5); impact measurement (5); quality of presentation (5); EU dimension (10); cost-effectiveness (20). During the period under review, some Member States have tended to present a large number of applications, leaving it to the Commission to make the selection, whilst others have made a careful pre-selection. A significant proportion of the rejections by the Final Report November 2011 Page 18 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Commission have occurred because applications have been found to be ineligible. The Commission selects the highest quality applications from amongst those that are eligible. At the same time, the Commission has been encouraging Member States to undertake more substantial short-listing by preparing criteria that cover the level of details required, the checks to be undertaken and various success criteria. The Commission’s system for selection involves appointing a first and second reader for each programme and examining all programmes by an evaluation committee. In addition, most of the programme proposals are also being evaluated by independent external experts. Special attention is paid to the selection of implementing bodies, which must be done by competitive tendering. These procedures are designed to ensure that selection is objective across all programmes. In 2001 and 2002, the Commission received over 60 applications, all of which were approved. In the next three years, the rejection rate rose to nearly 20% and from 2006 it has been at about 50%. At the Commission level, this is explained by quality issues of programme proposals, which do not always fit quality requirements as provided in the guidelines. In 2007 and 2008, the Commission introduced further criteria and explanations about the selection procedures. The bulk of this information was released in the document AGRI/60787/2007 already mentioned before presenting Guidelines to be considered when assessing and managing part-financing programmes for promoting Community agricultural products. These guidelines are structured in 15 sub-sections (see description above under Guidelines). In addition, the Commission released the document Ref Ares (2009)256506 15, which is the guidance used in the framework contract evaluations. This document consists of a set of 14 standard questions which can be used by proposing organisations and competent bodies to question themselves about the quality of the design of the programme proposals before submission for approval. These questions are structured in five themes usually covered by ex post evaluations: Coherence between information and promotion measures and objectives of the Regulations ; Information and promotion activities and communication channels used and their profitability; Coverage and content of the programmes; Effectiveness and impact of the programmes; and Complementarities between programmes implemented by Member States and those implemented by professional organisations. 7F Implementation. Programmes are implemented by contracts between Member States and the proposing organisations, using standard contract formats. The proposing organisations then organise sub-contracts for the implementing bodies. Payments start with an advance of up to 30% of the contract total, followed by regular quarterly reimbursement claims. A 15 This document from September 2009 replaces document AGRI/63454/2007 Final Report November 2011 Page 19 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC final payment is made within four months of completion, subject to provision of accounts and the evaluation report. Monitoring of programmes, including ensuring the validity of payments, is the responsibility of Member States, which establish a monitoring group presided over by the national authority including a representative of the Commission. The Member State is required to carry out financial checks on a sample of at least 20% of programmes each year, selected to include those most likely to have problems, on the basis of a risk assessment. The Commission also has a role in monitoring expenditure and has been introducing new systems for monitoring expenditure that have helped to streamline procedures, to reduce occasions when claims above budget have been paid and to ensure that MS match funding is provided. Work plans and planned budget are defined for the whole period of the programme. While implementing the programme, proposing organisations are not allowed to modify planned budget items without EC approval, except in one specific case. According to document AGRI-64545-2007 mentioned above from 23 November 2009 and focusing on positions and interpretations provided in the framework of the implementation of information and promotion programme, the budget transfer from one budget item to another budget item of the programme without amendments to the programme and without EC approval is limited to 10% of each budget item. This transfer is allowed if: Proposing organisations implement information and promotion activities as foreseen in their contract ; The budget to be transferred has to be provided by savings from another budget item of the approved programme; The budget is transferred to an information and promotion activity that was already foreseen and approved by the EC. 1.2 Description and development of other promotional measures applied under the CAP Information and promotion of agricultural products is not only financed by Council Regulation (EC) 3/2008 through EAGF but also by other CAP measures, both under the first or the second pillar. Following recent sectoral CAP reforms, some sectors include promotional measures into their support programmes, especially the wine sector (reformed in 2008) and fruit and vegetables sector (reformed in 2007). Council Regulation (EC) 73/2009 regarding direct support to farmers includes under its article 68 the possibility of specific support measures that could include promotion. Council Regulation (EC) 814/2000 on information measures on the CAP, that is not directly engaged in promotion of products, is nevertheless mentioned as some very specific activities linked to schoolchildren and healthy eating have been implemented. Final Report November 2011 Page 20 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Rural development policy, the second pillar of the CAP, includes a possibility to support promotion of products covered by food quality schemes since 2003, when the Council Regulation (EC) 1257/1999 was modified accordingly. Information, promotion and advertising activities undertaken by producer groups exclusively for food quality schemes were introduced as eligible for the first time since then. These other promotional measures will be shortly presented hereafter, including their actual importance in terms of promotion focusing in particular to the 2007-13 period 1.2.1 Fruit and Vegetables With a view to governing the sector’s production and trade in the EU and to achieve the CAP’s objectives, the Common Market Organisation (CMO) for fruit and vegetables was reformed in 1992. Through the Council Regulation (EC) No 2200/96, this CMO was reformed for the first time. Producers’ organisations (PO’s) became the key instrument for supporting EU fruit and vegetables growers under the CAP. In 2007, 33% of the EU production was traded through PO’s. Financial support was brought to PO’s to conduct operational programmes. Measures financed under these programmes included improving quality, marketing, promotional campaigns, developing organic or integrated production, and other environmentally friendly measures. A second reform of the CMO for fruit and vegetables took place in 2007 and the reformed CMO started on 1 January 2008. The aim of the reformed CMO is to improve the competitiveness and market orientation of the fruit and vegetable sector, reduce income fluctuations resulting from crises, promote consumption – so contributing to improved public health – and enhance environmental safeguards. Article 103c of Council Regulation (EC) No 1234/2007 of the single CMO regulation relates to the promotion of products in the fruit and vegetable sector (fresh or processed) in the framework of the single CMO including fruit and vegetables. Under the reformed CMO, operational programmes are designed and implemented by POs potentially including promotion measures with a view to increasing fruit and vegetable consumption and to improving POs marketing. Duplication is avoided by requiring applicants to the information and promotion programmes to confirm that they are not applying for any other source of funding from the EC. It should also be emphasized that the various CAP measures cover different categories of beneficiaries, the PO’s in the framework of the single CMO and trade organisations for Council Regulation (EC) No 3 / 2008 (see table 4 hereafter). The possibility of including promotion measures in operational programmes did already exist before this last reform. In addition to the EU co-funding of operational programmes, additional EU budget is available for promotion of fruit and vegetables targeted at children in educational establishments, as well as for distribution of fruit and vegetables to schools, hospitals and charitable bodies. The support provided by Council Regulations (EC) No 1234/2007 and (EC) No 3/2008 is similar even if it shows differences in the application, especially in terms of beneficiaries, use of collective trademarks and selection and implementation of activities. Table 5 below shows these differences. Final Report November 2011 Page 21 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 5: Item Promotion of trademarks Type of beneficiaries Main differences between support to the promotion of fruit and vegetables under Regulation (EC) No 1234/2007 and Regulation (EC) No 3/2008 Under Regulation (EC) No 1234/2007 (Common Market Organisation) Promotion of the collective trademark of the producer organisation is allowed (e.g. Fraises de Wépion, in Belgium). Producer organisations (e.g. Coopérative de l'Yerne scrl in Belgium). Place of promotion measures Under Regulation (EC) No 3/2008 Only generic promotion is allowed. Professional Trade/ Branch organisations representing the sector (e.g. Interfel in France). Information and promotion measures are the only measures of the programme. Promotion measures are part of an operational programme, which includes also support to other activities (e.g. production, quality, etc.). Selection and The operational programme (OP), which The information and promotion approval of the may include promotion measures, is programme is pre-selected by the Member programme approved as a whole by the Member State. State and selected by the European There is no EU ceiling for the whole sector Commission. and no selection. Level of the EC European Union financial assistance to PO An annual indicative budget of €10m is support and their operational programme (not foreseen for fresh fruit and vegetables and specifically promotion measures) is limited €2m for processed fruit and vegetables for to 4.1% of the value of the marketed 27 MS. production of the producer organisation The average annual expenditure over 2002and to 50 % of the actual expenditure 2010 to fruit and vegetables (fresh and incurred (or 60% in some cases). However there is no EU budget ceiling for the whole processed) promotion was €8.5m. sector. The more OPs, the higher the budget. In 2008, expenditures were about €40.0m while these provisional expenditures for 2009 were around €28.3m (see details below). Member State Under the general rule, Member States does According to Council Regulation (EC) co-financing not contribute. OPs are co-financed by the 3/2008, Member States can contribute EU and the PO itself. from 0% to 30% of the budget. Source: ADE 2011, based on the analysis of Regulation (EC) No 1234/2007, Regulation (EC) No 3/2008 and ADE calculations made from Execution promotional programmes (Excel file) provided by Dg AGRI, 2011. On average, promotion financed through the CMO has represented 5.2% (€34.4m annually) of the budget of producers’ organisations over 2008-2009 (provisional data). This amount is almost 4 times larger than that of promotion through Council Regulation (EC) 3/2008 It must be added that Council Regulation (EC) No 1234/2007 allows for the EC cofinancing share to be increased from 50% to 60% under certain conditions, including for: multi-country applicants; organic products; new Member States; first applications by producer organisations; organisations from outermost regions; and actions to promote the consumption of fruit and vegetables targeted at children in educational establishments. The co-financing share can be increased to 100% for withdrawals for distribution to charities, prisons and schools, within a limit of 5% of total volume marketed by each producer organisation. Final Report November 2011 Page 22 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC European Union resources allocated to the EU fruit and vegetables sector amounts to around 3% of the budget of the CAP. Producers’ organisations under the CMO for fruit and vegetables benefited from the largest allocations to the sector: €645m in 2008 and €676m in 2009 (provisional data). More specifically, table 6 hereafter provides budgets allocated to promotion measures under the CMO for fruit and vegetables by Member State and for the year 2008 and the year 2009. The total support of the EC to promotion measures amounted to €40.0m in 2008, while in 2009 this amount was around €28.3m (provisional data). EU production of fruit and vegetables rests to around one third on producer organisations (around 1,400 PO’s in the EU). The level of structuring is the highest in Belgium and in the Netherlands (above 80% of the production provided by PO’s) and the lowest in Portugal and Greece (respectively 5% and 10%), as well as in the EU-10. Table 6: EU contribution to promotion measures under the CMO for fruit and vegetables in 2008 and 2009 Member State IT BE FR NL ES HU AT DE EL UK SE CZ PT SK MT PL FI CY DK IE BG RO EE LT LU LV SI Total EU Total (€) for 2008 Total (€) for 2009 16,294,93 9,083,841 8,960,551 679,276 989,883 400,451 493,263 1,147,116 204,390 974,370 585,795 202,866 274,114 8,727 29,490 581 101,866 63,323 6,638 835 - 13,600,706 5,173,293 4,017,658 1,359,526 944,261 714,768 658,684 566,098 369,399 344,362 247,649 206,863 124,427 32,915 20,736 3,501 1,408 - 40,502,308 28,386,255 Source: Ranking promotion (Excel file), DG Agri Olive oil and horticultural products Unit (C.2), 2011 and Communication of DG AGRI, Unit C2, September 2011 Final Report November 2011 Page 23 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The School Fruit Scheme In November 2008 the Agriculture Council of Ministers agreed on a Commission proposal for a European Union-wide scheme to provide fruit and vegetables to school children. The overall annual EU budget for the Scheme is € 90 million. The EU provides cofinancing for the Scheme (50% or 75% for convergence regions) and the EU funds must be matched by national and/or private funding. The Scheme started in the school year 2009/2010 and 24 Member States are participating, either at national or regional level. The School Fruit Scheme is introduced in article 103ga of Council Regulation 1234/2007 and its detailed rules for implementation are laid down in Commission Regulation 288/2009. The three pillars of the scheme are: (I) purchase and distribution of fruit and vegetables in schools; (II) monitoring, evaluation and information as integrated part of Member States School Fruit Schemes and (III) accompanying measures (no EU co-financing). Essentially, this EU aid for the supply to children in educational establishments of products of the fruit and vegetables sector must be accompanied by measures to improve the target group's knowledge on the fruit and vegetable sector and to promote healthy eating habits. These accompanying measures can take the form of websites, farm visits or gardening sessions, all organized and financed by the beneficiary Member States and/or schools and fruit and vegetables producers. Apart from meeting the wider objectives of the CAP, including an improved competitiveness in the fruit and vegetables sector, this initiatives intends to ensure a high level of health protection to children, particularly with regard to obesity (an estimated 22 million children in the EU25 are overweight, and 5.1 million of them are obese). This health objective should be reached by promoting healthy eating habits. A durable increase of the share of fruit and vegetables in the diets of children at the stage when their eating habits are being formed is wanted and motivated by the fact that in recent years the consumption of fruit and vegetables in the EU has been falling. The majority of Europeans and in particular children fail to meet the minimum intake of 400 g per day recommended by the World Health Organisation. The School Fruit Scheme is to be seen as a driving initiative with a multiplying effect in the longer term with regard to consumption and health. According to the 2008 Impact Assessment (SEC(2008)2225), prior evidence and experience (e.g. Denmark, Ireland and UK) shows that this action, apart from resulting in a direct short-term annual increase of 97,500 tons of fruit and vegetables, based on a portion of 120 grams per child per week for 30 weeks, might have a leverage effect with a magnitude of 10 in the long term. Final Report November 2011 Page 24 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 1.2.2 Wine The wine Common Market Organisation (CMO) was reformed in 2008. The objective of the reform was to improve the competitiveness of the EU wine sector by reducing market intervention and reallocating European funds towards pro-competitive measures. The wine sector is now included in the single CMO regulation (Council Regulation (EC) 1234/2007). Within the wine sector, EU funds are allocated to the MS through national programmes, under MS responsibility. MS may choose which support measures to adopt among eleven possibilities described hereafter from the regulation. Some measures, similar to previous support measures of the wine sector are being phased out. Their support may be granted until July 2012 the latest. MS may adapt their support programme twice a year depending on the Regulation applied (currently Commission Regulation (EC) No 555/2008). The eleven measures are the following 16: 8F a) b) c) d) e) f) g) h) i) j) k) “Single payment scheme support (Art. 103o); Promotion (Art. 103p); Restructuring and conversion of vineyards (Art. 103q); Green harvesting (Art. 103r); Mutual funds (Art. 103s); Harvest insurance (Art. 103t); Investments (Art. 103u); By-product distillation (Art. 103v); Potable alcohol distillation (Art. 103w); Crisis distillation (Art. 103x); Use of concentrate grape must (Art. 103y);” Unlike for Fruit and Vegetables, promotion is a new measure under the reformed CMO. Provisional budgets from aggregated national support programmes (NSP) are shown in table 7 hereafter. Restructuring and conversion is the most important measure covering over one third of the overall budget allocated to the wine CMO. Promotion represents 15%, Investments and SPS respectively 10%. 16 Consolidated Council Regulation (EC) No 1234/2007, Article 103m Final Report November 2011 Page 25 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 7: Budgets of the reformed wine CMO measures Budget (€m) Measure 2009 2010 2011 2012 2013 2009-2013 % of national support Single payment / 33 157 160 160 510 10% scheme (SPS) Promotion 35 87 144 236 265 768 15% Restructuring and 326 401 402 468 472 2,067 39% conversion Investments 19 74 95 179 179 546 10% By-product 85 96 96 97 97 471 9% elimination Others 282 320 126 109 58 894 17% Total 747 1,011 1,020 1,248 1,231 5,257 100% Source: ADE based on European Commission DG AGRI C3 Wine, alcohol, tobacco, seeds and hops – Financial table of the national support programmes 2009-2013, situation in September 2011 NB: The “others” line includes the other measures of Article 103 listed above. The most important ones are potable alcohol distillation and concentrated grape must. Promotion measures of EU wines mentioned under Article 103p of Council Regulation (EC) No 1234/2007 relate to third countries only. Promoted products are wines with a protected designation of origin (PDO) or a protected geographical indication (PGI) or wines with an indication of the wine grape variety. The objective of promotion under the reformed CMO is to improve the competitiveness of EU wines in third countries. On average, Member States have decided to allocate 15% of the total wine sector support envelope from 2009 to 2013 (over €150m per year) 17 to promotion, as illustrated by the table 6 before. Since 2009, its first year of use, it has been one of the most important wine sector measures. So far, 9 out of the 17 wine-growing Member States have activated the measure and certain countries, such as France, Italy and Spain, have decided to allocate an important share of their wine sector envelope to promotion under Article 103p (e.g. Italy has decided to allocate €102m in 2013) see table 8 hereafter. F 17 Source: DG AGRI, Wine CMO, Financial table of the national support programme, Commission Regulation EC 555/2008, 2009-2013 data. Final Report November 2011 Page 26 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 8: Promotion under national support programmes of the reformed wine CMO (in million euros) Member State Budget (in €m) 2009 2010 2011 2012 2013 2009-2013 % of national support Total 35 87 144 236 265 768 15% Italy 5 25 48 82 102 263 17% France 18 27 30 85 95 256 22% Spain 7 24 40 40 40 151 10% Portugal 3 3 14 15 14 49 18% Austria 1 1 2 2 2 8 14% Greece / 5 4 4 4 17 14% Bulgaria / / 2 3 3 8 7% Germany 0 0 2 2 2 6 4% Others 1 1 2 3 3 10 1% Source: ADE based on European Commission DG AGRI C3, Wine, alcohol, tobacco, seeds and hops – Financial table of the national support programmes 2009-2013, promotion. Promotion measures consist in public relations, advertising, and participation to events, fairs or exhibitions of international importance, information campaigns, studies of new markets and studies to evaluate the results of these measures. Beneficiaries of promotion under Article 103p can be private companies, professional organisations, producer organisations, inter-branch organisations or public entities. The EU support is limited to 50% of the eligible expenditures. The remaining 50% may be entirely financed by beneficiaries or shared between beneficiaries and Member States. The origin is mentioned in the case of the wine, in the form of a geographical indication. In addition, promotion of brands and collective trademarks is possible as long as no State Aid is provided for the financing of the promotion actions and the latter aim in particular at underlining the advantages of the EU products in particular quality, food safety or environmental protection. Member States are responsible for the selection of programmes, based on compliance with the requirements 18 and the following selection criteria: Coherence between the strategy and the objectives ; Quality of the actions ; Impact on the growth of demand ; Effectiveness and technical capacities of the operators ; Cost-effectiveness. 19 10F 1F 18 Requirements include the following: wines must present potential for export to third countries and a high added value, the origin of products is mentioned in the case of an information or promotion action in the form of a geographical designation, the action must be clearly defined, support to a given beneficiary in a given third country is limited to three years (with possibility of adding up to two years), the beneficiary must be able to face constraints and implement effectively. 19 Source : DG AGRI, « Nouvelle organisation commune du marché vitivinicole – mesures d’aide à la promotion en pays tiers », 29 Septembre 2010, Paris. Final Report November 2011 Page 27 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC With the abolition of certain market interventions by 2013 (e.g. potable alcohol distillation and crisis distillation), promotion is expected to represent an important place in the wine sector support in the years to come, although the post-2013 orientations are yet to be decided. Main differences between support to the promotion of wine under the reformed CMO or under Council Regulation (EC) 3/2008 are shown in the table below. They mainly concern the type of products on third country markets, the beneficiaries, the markets targeted, the possibility to mention brands, the duration and the average budgetary allocation. Table 9: Comparison of the support to promotion of wine under Regulation (EC) No 1234/2007 and Regulation (EC) No 3/2008 Item Type of product Beneficiaries Under Regulation (EC) No 1234/2007 Wines with a PDO, PGI or with an indication of the wine grape variety Purpose of the measure Producer organisations, Private companies, Interbranch organisation Professional organisation Public entities Improve competitiveness of EU wines in third countries (art.103p) Markets targeted Third country markets Coverage of information and promotion measures Information campaigns on the EU system covering wines with a PDO or PGI or wine grape variety,... but also public relations, promotion or advertisement measures Mention of origin or brands Origin of the product may be indicated as part of an information or promotion operation in the case of wine with a geographical indication (Art.4b Commission Regulation (EC) 555/2008) Indication of individual or collective brands is possible as long as no State aid is involved Final Report November 2011 Under Regulation (EC) No 3/2008 Wines with a PDO, PGI or with an indication of the wine grape variety Trade or intertrade organisations Inform consumers about the variety, quality and production conditions of EU wines and about responsible drinking pattern (annex I Commission Regulation 1313/2008) Internal market or Third country markets Information campaigns on the EU system covering wines with a PDO or PGI or with an indication of the wine grape variety.” (art.2) Instruments also include public relations, training for distributors or caterers, contacts with specialized press, fairs and exhibitions. Promotion measures shall not be brand-oriented or encourage the consumption of a product on grounds of its specific origin. However, the origin may be indicated in the case of designations conferred under EU rules (art1.2 of Council Regulation (EC) No 3/2008). Page 28 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Item Place of promotion measures Duration of programmes Under Regulation (EC) No 1234/2007 Call for proposals: beneficiaries apply to the promotion measure inside the National Support Programme 3 years with a possible extension of 2 years Under Regulation (EC) No 3/2008 Furthermore, the reference to the origin must be secondary to the central message of the campaign (art 4.2 and 5.2 of Commission Reg (EC) No 501/2008) Call for proposals: Trade or intertrade organisations submit a programme to MS 1-3 years Selection and approval of the programme Member States define selection The information and promotion procedures in accordance with art.5 programme is pre-selected by the of Commission Regulation Member State and selected by the 555/2008 European Commission Amount of the EC Proportional to the National The annual indicative budget is support Support Programme, which are €3m for the Internal Market for around €157m/year in average from 27 MS. The actual EC indicative figures from National contribution was around €3Support programmes with an 5m/year over the 2004-2010 increasing trend reaching €265m in period 2013. Source: ADE 2011, based on the analysis of Regulation (EC) No 1234/2007 and Regulation (EC) No 3/2008 and financial tables of the National support programmes The number of programmes under EC support to information and promotion measures for agricultural products (Council Regulation (EC) 3/2008 and former Council Regulations (EC) 2826/2000 and 2702/1999) developed as follows (including the recent period where the CMO covers promotion): 20 from 2004 to 2009, 28 programmes were implemented covering wine, spirit and wines or PDO-PGI-TSG and wines for around €64m of which €32m from EC contribution. the annual indicative contribution may be estimated around €3-5m/year 20. eight of these 28 programmes result from the current regulation (Council Regulation (EC) 3/2008), of which 6 concern the internal market and two third country markets. EC contribution amounts to €12m, thus around €1.5m per programme. 12F Figures include promotional programmes accepted through 2004-2009 (there was no programme on wine in 20022003). They are all of a 3 years period, thus some of them covering 2010 and 2011. Programmes accepted in 2010 are not included. Some programmes do not only cover wine but also PDO-PGI-TSG (3 programmes) and one covers spirits and wines. Final Report November 2011 Page 29 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 1.2.3 Measure 133 under Rural Development Policy Article 20 and 33 of Council Regulation (EC) 1698/2005 on support for rural development by the European Agricultural Fund for Rural Development (EAFRD) provide for support to producer groups for information and promotion activities for products under food quality schemes (Measure 133 of the Rural Development Programmes). According to Commission Regulation (EC) 1974/2006, rural development measures are foreseen for each of the four thematic axes, the first one concerns the improvement of the competitiveness of the agricultural and forestry products. Among the first axis, a series of measures are available aiming to improve the quality of agricultural products. This measure 133 (Article 33 of Council Regulation (EC) 1698/2005) providing support to producer groups for information and promotion activities is however linked to measure 132 (Art. 32 of Council Regulation (EC) 1698/2005) providing support to farmers participating in food quality schemes. Such food quality schemes can either be EU schemes or schemes recognised by the Member States. Only products covered by food quality schemes, which benefited under measure 132, are eligible for support under measure 133. Measure 133 aims, along with other measures of the first thematic axis, to improve the overall competitiveness of the agricultural sector. In particular, its objective is the promotion of agricultural products or foodstuff produced under food quality schemes on the internal market. Supported activities include, in particular, the organisation of, and/or participation in, fairs and exhibitions, public relations and advertising via different channels of communication or at the points of sale. Promotion has to draw attention to the specific features or advantages of the agricultural products, such as their quality, specific production methods, high animal welfare standards and respect for the environment linked to the quality scheme concerned. The range of beneficiaries is narrower than that funded under I&P programmes as only producer groups are eligible under rural development policy, and the schemes are generally of smaller and shorter scale. Duplication of funding is avoided by requiring applicants to I&P programmes to confirm that they are not seeking funding from any other EU source. Relevant demarcation criteria are also included in the rural development programming documents. A comparison of the support to EU or national food quality schemes under Rural Development Policy or under Regulation (EC) 3/2008 is given in table 10 hereafter. Final Report November 2011 Page 30 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 10: Comparison of the support to EU or national food quality schemes under Regulation (EC) No 1698/2005 and Regulation (EC) No 3/2008 Item Type of products Type of beneficiaries Purpose of the measure Markets targeted Under Regulation (EC) No 1698/2005 Agricultural products produced under EU or National food quality schemes Producer groups. Professional and/or interprofessional organisations representing one or more sectors cannot qualify as ‘producer groups’. Improve consumers’ awareness of the existence and specifications of products produced under food quality schemes. Inform consumers and promote products provided under quality schemes supported by MS within their RDPs (Recital 28). I&P activities should be designed to induce consumers to buy the agricultural products or foodstuffs (art.23.2 Commission Regulation (EC) 1974/2006). Internal market Under Regulation (EC) No 3/2008 PDO, PGI, TSG or organic products Trade or inter-trade organisations representing the sector (art.6 Reg 3/2008) Provide information on the Community PDO, PDI, TSG and organic production schemes including a comprehensive information about the schemes, enhance knowledge of Community logos, encourage producers/processors to participate and stimulate demand for the products (annex I of Commission Regulation 501/2008) Internal Market or Third country markets Mention of origin or The origin of a product may be Art.4.2 and 5.2 of Reg 501/2008: brands indicated provided the mention of any reference to the origin of the origin is subordinate to the products shall be secondary to the main message. central message of a campaign. Activities related to the promotion However, the origin of a product may be indicated as part of an of commercial brands are not information or promotion eligible for support. (art.23.3) operation in the case of a designation under Community rules. Selection and RDPs are defined by MS and/or The I&P programme is preregions and approved by the selected by a Member State and approval of the Commission. Individual projects selected by the European programme are selected by MS and/or regions. Commission Level of the EC Up to 70% of eligible costs but An annual indicative budget of support may be much less (25%); a €3m is mentioned in annex I. The summary description of the type of average amount of the 28 coeligible costs and rates of support is funded programmes since 2004 is provided in the RDPs; according to almost €2.5m preliminary data from MTE projects range from €25,000 up to €1m or more. Source: ADE 2011, based on the analysis of Regulation (EC) No 1234/2007 and Regulation (EC) No 3/2008 Final Report November 2011 Page 31 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Under rural development policy, rural development programmes are drawn up at national level or in some Member States at regional level (DE, ES, IT, UK...). 21. Member States and regions are free to include in their programme the measures which are most appropriate for their national or regional priorities. They are not obliged to use all the measures available. However, in the case of promotion, measure 133 is strictly linked to measure 132. As shown in table 11, half of the MS (some regions) included measure 133 in their 2007-13 Rural Development Programmes and 12 already implemented it at the stage of the midterm evaluation. 13F Table 11: Use of measure 133 among Member States or regions MS (regions) who included measure 133 in their rural development programme AT BE (Flanders) CY DE (only one region) DK EL ES FR IT (most regions) MT NL PL PT SI MS (regions) who implement measure 133 AT CY DE (only one region) DK EL ES FR IT (most regions) MT PL PT SI Source: ADE based on DG Agri G1Consistency of rural development data (as of Dec. 2010) The Financial Plan for EAFRD contribution as programmed in February 2011 (see figure below) shows a financial allocation of around €200m over the whole programming period 2007-2013 or around €29m/year. This amount is very concentrated among a few MS, with almost 40% foreseen by Italy with around €80m, followed by Spain, Greece, Poland and France with around €20m each. Figure 10: Financial Plan (EAFRD) for measure 133 for the period 2007-2013 22 in €m 14F AT BE CY DE DK ES FR EL IT MT NL PL PT SI 7.2 0.1 1.5 2.3 10.0 25.4 19.8 21.3 82.5 0.5 0.8 22.5 9.0 5.0 0 10 20 30 40 50 60 70 80 90 Source: ADE based on DG Agri, G1 Consistency of rural development, February 2011 21 22 Overall, there are 94 RDP’s within the 27 EU MS. Situation in July 2010 Final Report November 2011 Page 32 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 1.2.4 Article 68 of Council Regulation 73/2009 Council Regulation 73/2009 has been introduced following CAP Health check and concerns direct support schemes to the farmers under the CAP. Article 68 concerns specific support to farmers. It allows all Member States to use some of the Pillar 1 budget to fund schemes that address particular needs, such as protecting the environment, improving the quality and marketing of products, supporting particular types of production in economically vulnerable or environmentally sensitive areas or type of farming. Article 68 – which is a follow-up to Article 69 of Council Regulation 1782/2003 – includes five measures, among which promotion could be financed under specific support to improve the marketing of agricultural products. Final beneficiaries of such support are farmers. An explicit link is established with Council Regulation (EC) 3/2008, which should ensure coherence 23. In this scheme, farmers may receive annual additional payments to promote agricultural products. 15F Measures under Article 68 are limited to 10% of the global envelope allocated to a Member State for Direct Payments. Within these 10%, the “Improving the marketing of agricultural products” measure is considered to be potentially coupled to production and is thus limited to 3.5% of these 10%. Member States chose in 2009 which measures to activate within Article 68. Most Member States largely activated measures related to specific types of farming or activities with enhanced environmental benefits as well as improvement of the quality of agricultural products. Only one Member State – Sweden – activated the “Improving the marketing of agricultural products” measure, for a total budget of €300,000 per year, for three years. The choice of Sweden was to pursue the same actions, and with the same budget, as those it undertook under Article 69 of Council Regulation 1782/2003. It can be concluded that, as currently applied by Member States, Article 68 hardly concerns information and promotion of agricultural products. 1.2.5 Council Regulation 814/2000 According to Council Regulation (EC) N° 814/2000, the Community may finance information measures relating to the CAP. These are of a much wider scope than information on agricultural products under Council Regulation (EC) N° 3/2008, however there are some very specific activities targeted towards schoolchildren and healthy eating that may be considered as complementary. Indeed, according to article 1 of the regulation, it includes in particular promoting the European model of agriculture and public awareness raising about the CAP. Under this regulation, general communication on the CAP is financed including DG Agri Website, participation in national agricultural events, publications on the reformed CAP by sector etc. 23 According to article 68 of 73/2009, “improving the marketing of agricultural products may only be supported if it satisfies the criteria laid down in Articles 2 to 5 of Council Regulation (EC) No 3/2008 of 17 December 2007 on information provision and promotion measures for agricultural products on the internal market and in third countries. Final Report November 2011 Page 33 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The specific issues that are more directly linked to this evaluation concern activities that were designed for schoolchildren on the promotion of better nutrition: In September 2009, DG Agri launched a campaign on healthy eating for schoolchildren from 8 to 15 years old: “The Tasty Bunch”. It was designed to encourage the consumption of milk, fruit and vegetables, and playing sports. Under the “Eat it, Drink it, Move it” slogan, the programme launched a two-month road show visiting schools to raise awareness on healthy eating habits in 7 MS 24 A website which provided interactive material and games (“food stories” pages to familiarise with the different products and the importance of good consumption and living habits, a treasure hunt game, a toolbox for parents and teachers, and pictures of the road show). 1F This healthy eating campaign acts as complement to Commission Regulations (EC) N° 657/2008 and (EC) N° 288/2009 which lay down the rules for supplying milk & milk products, and of fruit and vegetables in school establishments. 25 Under these two Regulations, the products are supplied in schools through co-funding by the EU and MS. 17F 1.3 Comparison of EU promotion with other countries 1.3.1 Other sources of public funding in Europe Within the EU, the EC funding must be at least doubled to take account of co-financing by Member States. In addition, some Member States have their own programmes. The level of spending by the EC policy is of roughly similar order of magnitude to the budget of Switzerland for similar activities (€36m in 2008) and is also a similar order of magnitude to the budgets for promoting individual products from the larger Member States, either by government (e.g. €100m in 2013 for Italian wine) or private trade association (e.g. €21m in 2008 for the Interprofessional Council of Bordeaux Wine). 1.3.2 Other sources of public funding on other continents Although representatives from several other key players on the international food markets were contacted, very little information was found on promotion policies outside of Europe. In the USA, several programmes articulate to promote agricultural products nationally and internationally. Internationally, the following two programmes support promotion: The “Market Access Program” (MAP) uses funds from the US Department of Agriculture’s Commodity Credit Corporation to co-fund actions with non-profit trade associations, agricultural cooperatives, state-regional trade groups and small businesses. They share costs of activities such as trade shows, market research, consumer 24 BE, FR, EE, LT, PL, UK, IE 25 These Commission Regulations are the applications of the amended Council Regulation 1234/2007. Final Report November 2011 Page 34 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC promotions, technical assistance, trade servicing and seminars to educate overseas customers. From 2008 to 2013, the MAP’s budget is set at $200m annually. The Foreign Market Development Program: this programme provides funds to small companies, to help reduce market impediments, improve processing capabilities, etc. It also includes support to generic promotion. The indicative annual budget for the global programme is $34.5m. On the US national market, several programmes support promotion for agricultural products: The Farmers’ Market Promotion Program: this programme provides grants to help improve and expand domestic farmers’ markets, roadside stands, communitysupported agriculture programs, agri-tourism activities, and other direct producer-toconsumer market opportunities. The programme’s budget for 2010 was $5m; and it has been increased to $10m for 2011 and 2012. The programme also brings a strong focus on consumers, as one of its objectives is to help eliminate food deserts and increase food access in low-income communities in the country. State promotional activities: states have a dedicated marketing arm in their agriculture department dedicated to providing branding logos and marketing material for state agriculture and marketing the products to local, national and international markets. In Australia, the most recent scheme is the “Promoting Australian Produce Program”. This programme assists Australian food industry organisations and groups to better develop their capacity to market their products. Over 2009-2012, the programme will co-fund promotion programmes for a total of $A5m (approximately €3.7m). The measure is not product-specific. In Argentina, several programmes support agricultural products promotion : The PROARGEX is a support programme dedicated to the promotion of agri-food exports. Its total cost is US$4.6m and it receives support from the Inter-American Development Bank. The programme takes places within the framework of the ValorAR programme (national programme for added value) which has an indicative budget of US$15m dedicated to “Studies, follow-up and promotion of agro-industrial and agrifood markets” The national export promotion agency, ExportAR Foundation, is a public-private body run mainly by the Ministry of Foreign Affairs, International Trade and Worship. ExportAR also actively agricultural products promotion (no figures available). Furthermore, in all three countries, sector-specific promotion is financed as well through levies on the value of production (through producers’ organisations funds rather than public funding). Final Report November 2011 Page 35 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 2 Evaluation methodology This section presents the evaluation methodology. It consists in three sub-sections. The first section explains the overall approach to the evaluation, including primary data collection during the desk and the field phases. The second section details the criteria for selection of the sample of 9 Member States and of 15 information and promotion programmes to be part of the scope of this evaluation. The third section details the scope and limits of the evaluation methodology. 2.1 Overall approach and data collection The evaluation covers two parts. A first part concerns a synthesis of previous 9 evaluation reports realised under two framework contracts on the internal market by sector and on third country markets. This synthesis and its horizontal conclusions and recommendations feed in the evaluation process. Findings from these reports have also been integrated to the current evaluation questions where relevant elements were identified. The second part of the evaluation, built on the synthesis, covers 3 themes and 12 EQs. The evaluation and data collection is organised around these two interlinked parts. As shown the figure 11 hereunder, four main tools have been implemented through the whole evaluation process to collect data and information. These tools are the following: data analysis (EU Export Helpdesk, FAO and World Bank databases), documentary analysis (various regulations, programme documents, reports...), online questionnaire sent to the competent bodies in all EU Member States, and face-to-face interviews in the 9 Member States selected for case studies (see 2.2 hereafter). All these different but complementary tools have been used at key moments and with different purposes. These tools, their targets and the nature of the data and information collected are discussed below. Information collected through these tools has been systematically analysed and filled in the data collection grid. The data collection grid was developed from evaluation questions provided in the Terms of Reference and includes proposed judgment criteria and indicators to structure each evaluation question Final Report November 2011 Page 37 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Figure 11: Tools used during the desk and field phases of the evaluation process The evaluation team has also prepared programme fiches with key information on the 15 selected I&P programmes taking into consideration in the evaluation process. These fiches include information at programme level about: The proposing organisation, the Competent body and the proposing country/countries; The products and themes covered by information and promotion actions ; Targeted countries ; Contract date, the European Commission reference, the programme start and end dates; Budget ; Overall objectives, specific objectives and expected results, messages, and target groups; Types of actions and examples of outputs. The data analysis started with the data collection at EC level. These data were completed by data available in the nine evaluation reports about the EU I&P actions undertaken between 2002 and 2008, in available monitoring and evaluation reports of the I&P programmes under review, as well as completed by an in-depth data research in relevant databases (e.g. FAOSTAT) dealing with agricultural products trade, consumption and GDP. The collected data were analysed by the evaluation team and filled into the data collection grid at the level of the appropriate indicators. The document analysis was carried out from available documents collected at EC level and at Member state level (mainly at the level of the proposing organisations), as well as the nine evaluation reports about the EU I&P actions undertaken between 2002 and 2008. Final Report November 2011 Page 38 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Each evaluation team member covered one or several countries as well as one or several programmes, depending on languages used. Results of the document analysis were included at the level of the appropriate indicators. In March 2011, an online questionnaire was carried out to gather relevant information at the level of Ministries and competent bodies from the EU-27 Member States. At the end of March 2011 and after several recalls from the evaluation team, 24 out of the 27 contacted Member States provided answers to questions developed on the basis of evaluation questions in the Terms of Reference and structured in judgment criteria. Information collected through the online questionnaire has been analysed and incorporated in the data collection grid. Finally further information and data were collected during face to face interviews organised in March and April 2011 at EC Headquarters in Brussels and for the 15 selected I&P programmes involving the nine selected Member States. When necessary, phone interviews have completed face to face interviews to get further details. Information and data collected both, through face-to-face and phone interviews, have been analysed and included in the data collection grid. After completion of the data collection grid at programme level, including the data analysis, the results of the data analysis, the document analysis and interviews a quality control of the grids was carried out by members of the core evaluation team (mainly the team leader and the quality controller). This necessary step allowed harmonizing information gathered and filled by all the different evaluation team members and ensuring the highest quality level of the data collection grid to be used to draft replies to evaluation questions and identifying limitations of replies to each evaluation question. 2.2 Selection of Member States and programmes for detailed examination According to the Terms of Reference, the second part of the evaluation has to focus on specific promotion issues of particular relevance for the period from 2008 onwards. In that perspective, a selection of nine Member States and fifteen programmes was made for detailed examination. The selection of the nine Member States was based on the following criteria: Selected Member States are the main beneficiaries of promotion programmes, i.e. the total budget of co-funded programmes approved in 2008 and 2009 (cumulated value including both years) in these Member States has exceeded €12m; selected Member States are the main beneficiaries of promotion programmes, i.e. the number of co-funded programmes approved in 2008 and 2009 (cumulated value including both years) in these Member States is at least five (if a Member State is part of one-multicountry programme, that programme is counted as one programme); the selection includes at least four Member States which have taken part in multi-country promotion programmes; Final Report November 2011 Page 39 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC the selection includes at least four Member States which have taken part in programmes promoting multi-products; at least one EU10 Member State shall be covered by face-to-face interviews (even if the approved total budget for the Member State is below €12m); specific attention has to be paid to Member States which include co-funded programmes promoting the following products or themes: - EU systems of protected designation of origin (PDO), protected geographical indication (PGI), traditional speciality guaranteed (TSG); - wines with a protected designation of origin or geographical indication; - wines with an indication of the wine grape variety; - spirit drinks with a protected geographical indication; - organic farming; - other EU quality and labelling schemes as well as the graphic symbols for the outermost regions. Table 12 hereafter presents a summary of all the programmes per Member State. Based on above-mentioned criteria, information provided in this table was used to choose the nine Member States, which are: France ; Italy ; Greece ; The Netherlands ; Spain ; United Kingdom ; Germany ; Portugal ; and Poland. Final Report November 2011 Page 40 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 12: Member States which have benefitted from EU co-funding to implement at least one promotion programme approved between 2008 and 2010 Approved budget (2008-2010) (€) Member State Single Multicountry Total Number of approved programmes (2008-2010) Multi-country Single Lead Other countries FR 43,853,901 21,807,934 65,661,834 9 6 2 IT 58,300,976 4,430,684 62,731,658 18 0 2 EL NL ES UK DE PT 35,809,383 19,592,881 16,557,936 14,598,567 13,705,000 12,290,900 12,880,233 12,290,900 10,148,125 3,170,235 3,108,515 2,217,618 3,686,100 0 2,370,505 2,351,661 2,344,617 1,648,634 1,427,778 1,372,761 1,338,885 0 633,104 115,300 2,143,584 8,173,401 2,479,378 3,309,237 0 1,500,000 0 0 2,024,850 1,995,200 1,456,941 2,084,903 0 2,992,800 0 0 0 0 62,990 0 0 813,876 0 0 55,275,790 37,952,968 27,766,282 19,037,314 17,907,804 13,705,000 13,360,453 12,880,233 12,290,900 12,172,975 5,165,435 4,565,456 4,302,521 3,686,100 2,992,800 2,370,505 2,351,661 2,344,617 1,648,634 1,490,768 1,372,761 1,338,885 813,876 633,110 115,300 11 7 1 4 6 5 5 4 4 1 2 1 1 0 3 2 1 2 1 2 2 0 1 1 AT PL BE BG IE DK SK CY SI SE LT FI LU CZ LV RO MT EE Multi-country Total 275,384,067 Final Report Market Total Promoted products & themes IM TC Fibre flax, fruit and vegetables, PDO, PGI & TSG, Meat, Dairy, Olives, 17 Outermost Regions, Spirits, Wine Organic, Cereal-based, Wines, Dairy, Olives, fruit and vegetables, Meat, 20 PDO, PGI, TSG, Horticulture Fruit and vegetables, Dairy, Olives, Organic products, PDO, PGI & 12 TSG, Spirits 10 Fruit and vegetables, horticulture, meat, dairy 4 Fruit and vegetables, pdo, pgi & tsg, olives, wine 7 Fruit and vegetables, horticulture , dairy, organic, meat 6 Fruit and vegetables, meat, dairy, pdo, pgi & tsg, organic 6 Dairy, wine 5 Fruit and vegetables, dairy, organic, horticulture 4 Meat, fruit and vegetables 7 Fruit and vegetables, honey, dairy, meat 2 Dairy 3 Fruit and vegetables, meat, dairy 2 Fruit and vegetables, dairy 1 Dairy 1 Dairy 3 Dairy, organic 2 Fruit and vegetables, Seed oil 1 Spirits 2 Dairy 2 Honey, dairy 2 Honey, dairy 2 Fruit and vegetables, dairy 1 Fruit and vegetables 1 Dairy 1 Meat 1 0 0 3 0 3 1 2 0 0 0 1 0 0 0 0 3 0 0 1 0 1 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 13 17 330,659,857 94 13 107 S : special dairy promotion programmes of 2009 (on the Internal Market) Source: ADE based on DG Agri Website November 2011 S V V V V V V V V V V V V V V V O V V V O V V O V V V V O O V V O V O V V O V O V V O O V O O V O O O O V O O V V O V V V V O O V V O O O O O O V O O O O V O Page 41 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC When the nine Member States were selected, different criteria were used to select information and promotion programmes under the scope of the evaluation. They are: Preference for larger budget programmes (to include the most significant uses of funds in the analysis); Inclusion of multi-country promotion programmes; Inclusion of multi-products promotion programmes; Respect to overall proportion of promoted products. In addition to these criteria, specific attention has been paid to the following promotion products or themes as already mentioned for the selection of the Member States: EU systems of Protected Designation of Origin (PDO), Protected Geographical Indication (PGI) and Traditional Speciality Guaranteed (TSG); Wines with a protected designation of origin or geographical indication; Wines with an indication of the wine grape variety; Spirit drinks with a protected geographical indication; Organic farming; Other EU quality and labelling schemes as well as the graphic symbols for the outermost regions. It is also important to note that the duration of the programmes appeared as a constraint limiting the possible choice. Programmes approved in 2010 were then considered too recent to be part of the scope of the evaluation. Table 13 hereafter provides the sample of 15 information and promotion programmes that were selected under this evaluation (among those approved since 2008). Table 13: Proposing Organisation BIVB, IVDP, Consorzio Prosciutto Parma & Consorzio Parmigiano-Reggiano List of programmes chosen for analysis Year of approval Proposing country Targeted area Promoted products Total budget (€m) 2008 IT, FR, PT UK Wine, PDO, PGI & TSG €6m UGPBAN 2008 FR FR SRWRP 2009 PL CNIEL & NZO 2009 NL, FR BNIC 2009 FR OIAO 2009 ES EAS Rethymnon 2009 EL Final Report CN, RU, SEA, UA FR & NL CN, NAM, RU BE, ES, FR, UK, NL MEA November 2011 Outermost region products Beef, veal and pig meat Dairy products Spirits Olive oil & table olives PDO, PGI & TSG €13.02m €3.10m €8.84m €1.82m €16.56m €1.22m Page 42 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Proposing Organisation Year of approval Proposing country SEKO Dramas 2009 EL 5 am Tag e.V. 2008 DE 2009 UK 2009 PT PT MULTI UIS E ALTRI 2008 IT NAM, IN, RU UIV – IVSI 2008 IT IT, FI, SE Interfel & Assomela 2008 IT, FR MEA, RU, NAF, SEA, CN 2009 IT IT, FR, DE Milk Marketing Forum FENALAC AIAB +COLDIRETTI Source: ADE, 2011 2.3 Targeted area Promoted products Fruit and CS, RU, UA vegetables Fruit and DE vegetables Dairy products UK Dairy products Wine, dairy products & olives Wine, PDO, PGI & TSG Fruit and vegetables Organic Total budget (€m) €2.86m €1.05m €9.04m €4.20m €3.76m €3.27m €3.17m €4.26m Scope and limits of the evaluation methodology As mentioned above, the evaluation is carried out on the global promotion policy, based on the regulation documents, previous sectoral/geographical evaluations, statistics on market situations and an online questionnaire and case studies in MSs. Given the limited use that can be made of the statistics to answer evaluation questions, a significant part of the information used is of qualitative nature (i.e. through interviews in MSs). The use of standardized judgement criteria and indicators has aimed to reduce the risks of subjectivity that could result from opinions of interviewees. The fact that common themes are picked up across several MSs suggests that there has been success in standardisation. Difficulties have nevertheless been encountered in accessing information from various sources: No precise information was made available at Member State level on national or private promotion initiatives, or on operational programmes of fruit & vegetable producer organisations. Secondly, the final version of data at DG Agri level on programme expenditure, high level trade visits, and selection criteria were only available several months after the beginning of the evaluation, when preliminary answers to evaluation questions were already drafted. Furthermore, the programmes implemented under the new Council Regulation (EC) No 3/2008 have only be operating for one (more rarely for two years), and of course there was currently limited evidence of achievement of the objectives. Final Report November 2011 Page 43 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 3 Comprehensive synthesis of the evaluation reports on promotion programmes 3.1 Context and methodology 3.1.1 Context In 2006, the Promotion Unit of the Directorate-General for Agriculture and Rural Development (DG AGRI) commissioned two framework contracts for evaluation studies to assess the effectiveness of information and promotion programmes implemented between 2002 and 2008 on the internal market and since 2001 on third country markets. These promotion programmes were undertaken under the former Council Regulations (EC) 2826/2000 (for the internal market) and 2702/1999 (for third countries) and their subsequent amendments. In total nine evaluation studies were carried out: four related to the internal market (organic products, wine, fruit and vegetables, and dairy products); and five related to third countries (USA and Canada; Switzerland and Norway; Russia; Japan and India, China and South-East Asia). Each of the evaluations was based on a first review of all programmes, including a desk review of the available literature, sectoral data and information on programme expenditure. In addition, each evaluation conducted more detailed analysis of a sample of programmes, drawn from a range of Member States. The sample generally covered between 30 and 40 percent of the number of programmes and over 50 percent of the total expenditure (see Table 14). For each sample programme, the full range of available programme literature was assessed and interviews and workshops were conducted with key informants. Three of the internal market evaluations included target group surveys to supplement the programme evidence with primary data collection: the milk evaluation ran 6 surveys 26; the fruit and vegetables evaluation ran 4 surveys 27; and the organic products evaluation ran 3 surveys 28. Each of the third country evaluations ran a survey covering between 100 and 200 professionals and 300 to 500 consumers, except in China where the consumer survey was replaced with focus group discussions. These surveys helped to validate the programme evidence and to provide additional insight. 19F 20F 21F 26 The surveys concerned respectively consumers in Germany and Greece, schools in Italy and Sweden, and health professionals in Portugal. 27 The different surveys covered households in Austria, Doctors and internet users in France, and schools in the Netherlands. 28 A questionnaire in Denmark and surveys in Italy and France. Final Report November 2011 Page 45 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 14: Sample Programmes Analysed in the Framework Contract Evaluation Reports on Promotion Programmes Programmes Sample Total Internal Market Organic Products Fruit and Vegetables Wine Milk Third Country Japan USA and Canada India, China and S-E Asia Russia Switzerland and Norway % Total Budget in €m Sample Total % 7 10 5 12 24 28 13 39 29% 36% 38% 31% 18.1 64.9 10.4 43.1 36.4 99.6 19.0 82.2 50% 65% 55% 52% 6 15 12 11 13 34 25 22 10 88 Source: ADE from framework contract evaluations 14 212 46% 44% 48% 50% 71% 10.8 26.5 23.2 7.1 3.5 207.6 22.4 51.9 48.5 12.2 5.0 377.2 48% 51% 48% 58% 69% Total Lessons learned, conclusions and recommendations from these studies were structured around five themes: i) coherence with the objectives of the Regulation; ii) measures taken and information channels used, and their cost-effectiveness; iii) coverage and content; iv) impact and effectiveness; and v) complementarities with programmes funded by MSs and the private sector. These conclusions and recommendations were taken into account during the amendment of the Regulation framework for the I&P Programmes and were used to help improve guidelines to improve the effectiveness of co-funded programmes on both internal and third country markets. The evaluation reports on promotion programmes on the internal market had 15 standard questions and the third country evaluations had 14. Twelve of the questions were common to both internal market and third country evaluations. Six of the standard questions of the framework contracts are also covered by one or more of the standard evaluation questions (EQs) of the current evaluation. The correspondence between the scope and structure of the framework contract evaluations and the current evaluation questions is summarised in Table 15. Final Report November 2011 Page 46 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 15: Comparison of the Scope and Structure of the Framework Contract Evaluations and the Current Evaluation Framework Contract Evaluations (2006-08) Coherence between implementation of the programmes and the objectives of the regulation 1.1 Do implemented measures correspond to the global objectives of the Regulation? 1.2 (TC) To what extent are overall European themes and general European characteristics of the products concerned covered in the messages of the programmes? 1.2 (IM) Do the guidelines give useful guidance and are they applied? 1.3 To what extent does EU co-financing contribute to promotional initiatives which would not have taken place otherwise? Actions and Information Channels 2.1 Are the actions and information channels chosen efficient and are their costs reasonable? 2.2 Which actions prove to be the most efficient in terms of cost versus impact (taking into account the objectives and target groups)? 2.3 Are the management methods applied appropriate/efficient? Current Evaluation (2011) EQ1 To what extent has the list of themes and products been relevant to achieving the objectives? EQ2: To what extent has the list of third-country markets been adequate to achieve the objectives? EQ3: To what extent have the information and promotion measures implemented at the Commission’s initiative been relevant to achieve the objectives? EQ 11 To what extent have the information and promotion programmes created EU added value? EQ7: To what extent have the guidelines provided useful guidance for the design and implementation of promotion programmes? EQ11: To what extent have the information and promotion programmes created EU added value? EQ4: To what extent has the EU promotion policy framework been conducive to encouraging multi-product and multi-country programmes? EQ5: To what extent has the effectiveness of the programmes implemented since 2008 been improved in terms of achieving the objectives? EQ6: To what extent has the overall framework for managing promotion programmes been appropriate to achieve the global objectives? EQ8: To what extent have the annual evaluation reports enabled effective monitoring and evaluation of the promotion programmes? EQ9: To what extent are the administrative requirements adequate to ensure that information and promotion programmes are selected and implemented in a transparent manner and based on relevant criteria? Is there any room for simplification of administrative requirements? Coverage and Content 3.1 Do the programmes have sufficient critical mass for effective coverage? Final Report November 2011 Page 47 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 3.2 Do the programmes reach a sufficiently wide audience or target group to be effective? 3.3 Are the programmes of adequate duration to have significant impact? Impact and Effectiveness 4.1 Do the measures contribute to improving image of EU products? 4.2 Is there a link between the measure and the evolution of demand for EU products? 4.3 (TC) To what extent do the measures taken contribute to the opening up of new markets and enhancing the export possibilities for European products? 4.3 (IM) Does the portfolio of approved programme make a coherent whole (synergies)? EQ5: To what extent has the effectiveness of the programmes implemented since 2008 been improved in terms of achieving the objectives? Complementarity with National/Private 5.1 How well the programmes complement those initiated by Synergy with MS/private sector? EQ10: To what extent have the information and promotion programmes had complementary or synergetic effects on national or private promotion initiatives? EQ11: To what extent have the information and promotion programmes created EU added value? 5.2 (IM) Whole programmes have a clear identity? 5.3 Multiplier effects with MS/private actions? EQ12: To what extent have information and promotion measures applied under Council Regulation No 3/2008 been coherent with other CAP promotion measures? Note: The second and fifth themes were switched for the internal market and Third Country evaluations 3.1.2 Methodology for evaluation screening in this chapter In order to organise the synthesis, the information collected from these nine framework contract evaluations was structured in a grid, based on 77 standard questions generated by reading of the reports and selecting the issues that came up most frequently and forcefully. The information from each report on each question consisted of a short answer and a score ranging from 1 to 5 29. This scoring system was used to set the appropriate tone for balancing evidence among answers. Examples were then described with reference to the target country (on third country markets) or to the product or theme (on the internal market) to illustrate the statements. 2F 29 The scoring scale is the following: 5: clear evidence encouraging the current practice of EU co-funded information & promotion programmes; 4: good but not conclusive evidence; 3: mixed messages; 2: non-conclusive evidence against the current practice of EU co-funded information & promotion programmes; 1: conclusive evidence against the current practice) Final Report November 2011 Page 48 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The final section on horizontal conclusions and recommendations, including best practices, aims only to summarise the conclusions and recommendations presented in the nine framework contract evaluations. No additional interpretation is introduced in this section, in order to avoid any ambiguity over the evidence and duplication with the final conclusions and recommendations presented at the end of this report. For every issue mentioned in the synthesis, the presentation aims to mention each of the nine evaluations, for which there were clear conclusions. If some of the nine evaluations are not mentioned for any issue, this means that they did not address the issue in a manner that can be used constructively in the synthesis. Extent and Limits of the Approach. The assessment of the evaluations was undertaken by a small number of independent experts using a set of common evaluation questions. This ensured that the scope and focus of the evaluations was consistent and comparable. Despite this, the nature, strength and coverage of the conclusions were very mixed, because the experience of the different products, themes and third countries was highly varied. In addition, the approach to reporting within each programme is quite varied. This is particularly obvious in relation to the reporting on impact. When synthesising the evaluation reports, it is natural that interesting findings are given a prominent place and this can give the impression that the findings are of wider relevance than can be justified by the full range of evaluations. This is sought to be avoided by providing evidence from the individual evaluations to illustrate the range of content and depth of evidence provided by the evaluations. The synthesis is structured according to the five themes of the framework contract evaluations mentioned before, namely: i) coherence with the objectives of the Regulation; ii) measures taken and information channels used, and their cost-effectiveness; iii) coverage and content; iv) impact and effectiveness; and v) complementarities with programmes funded by Member States and the private sector; and the standard questions that were prescribed for the nine evaluations of the framework contracts. 3.2 Theme I – Coherence between the implementation of the programmes and the objectives of the regulation Theme one dealt with the extent to which the programmes implemented were consistent with the objectives of the regulation 30. The first EQ concerns the extent to which implemented programmes correspond to the global objectives of the Regulation. For internal market programmes, the evaluations assessed under the second EQ whether the guidelines were useful and were applied 31. For third market programmes, the second question asked whether the programmes promoted EU-wide messages. The third standard question was common to both internal market and third country programmes and asked whether the promotion activities would have taken place if EC funding had not been 23F 24F 30 There is some link with current EQ1, EQ2 and EQ3 about the relevance of products, themes and countries covered to reach the objectives of the Regulation but EQ are formulated in quite different terms. 31 This EQ is equivalent to EQ7 in the current evaluation. Final Report November 2011 Page 49 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC available. It has similarities with theme V of the same framework contract evaluations tackling with complementarities. 3.2.1 Objectives of the promotional programmes coherent with the Regulation Regarding the internal market programmes, the objectives are defined in the Commission Regulations (EC) 94/2002 and 1071/2005. These objectives are specific to each product and theme and are presented in the annex of the Regulations. They mainly concern the image of EU products, knowledge of production methods (including the respect of animal welfare and the environment), awareness about quality, safety, labelling etc. in order to contribute to the global objective: to increase the consumption of EU products. Regarding the programmes in third country markets, there are four generic objectives in the Commission Regulations (EC) No 2879/2000, 67/2005 and 1346/2005 according to framework contract evaluations in third country markets: Increasing awareness of the targets groups about European products’ quality and food safety (and other attributes such as traceability, control, labelling, respect for the environment, animal welfare etc) Contributing to opening new markets Increasing demand for European products in third countries Improving the image of European products on international markets Overall, the objectives of promotional programmes were in line with those of the Regulation (Table 16). However, some of the evaluations stated that the objectives stuck too closely to the Regulation or that they were filled in as a formality. Thus, most programmes broadly followed the same objectives, without much differentiation. There was rarely any distinction between global objectives (increase consumption) and specific objectives (awareness, image, knowledge, etc.). For each of the evaluations, there was little ordering of the objectives (e.g. into global and specific), little prioritisation amongst objectives and little quantification of objectives. Final Report November 2011 Page 50 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 16: Sector / Third country market Fresh fruit and vegetables Processed fruit and vegetables Objectives of the programmes (global mainly) To improve the image as being fresh and natural especially among young consumers To increase consumption among young consumers To modernise the image and to make it more youthful To increase consumption in general Wine Milk Organic products To inform consumers about the variety, quality and production conditions of EU wines and the results of scientific studies To increase consumption of EU wines To increase liquid milk consumption in markets where potential of growth exists and maintained consumption levels in saturated markets To encourage consumption of milk and milk products by young people as future adult consumers To increase consumption of milk products in general Three specific objectives Increased consumption of organic food Increased quantity of organic products offered by distributors Individual producers/processors/retai ler groups convert to this production method - Final Report Objectives of the promotionnal measures Comments The objectives pursued were those mentioned in the Regulation or slight variations thereof. The choice of objectives was mostly based on the results of a context analysis, but there was no clear ranking of objectives (global, specific). In addition the objectives were often quantified at global level (consumption) but rarely at specific level (image, awareness). The objectives were in line with the goals of the Regulations and were chosen on the basis of the Regulations, on the basis of the stakeholders’ experience and market studies. However, the evaluation mentions that “the programmes lack a clear strategy, due to the fact that they do not focus on specific objectives, target groups and messages but rather attempt to combine most of those mentioned in the Regulations without defining the clear causal linkages between them”. In addition, the expected final impact on the demand was not quantified, either in figures or in time. The overall objective was to increase consumption of milk products in general. Among individual programmes, there was a wide variety of objectives, ranging from the very specific objective of promoting particular cheeses or low fat milk products to the general and broad objective of relieving the milk market and developing new market potential. ‘Although proposing organisations appear to have a clear view of the strategies behind the programmes, such strategies are generally not made explicit in the programming documents nor are ex ante quantification of objectives provided.’ Also, whilst the different programmes tended to follow the specific and global objectives defined in the regulation, there was no prioritisation amongst the programmes. The programmes were in line with the objectives of the Regulation but presented several flaws: the objectives were very rarely quantified and presented no hierarchy. Global objective November 2011 Page 51 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Sector / Third country market India, China and South-East Asia Objectives of the programmes (global mainly) Increased sales of organic food Five objectives without ranking (global, specific) were defined : Increase the awareness of the products Inform on quality Increase awareness of methods of production and consumption Open new markets Strengthen the market shares Comments All programmes were consistent with regulation objectives, both in terms of their objectives and their activities. Most of the programmes focused on improving awareness of the quality and food safety of EU products. Several also focused on opening up new markets and on increasing demand. None were explicitly concerned with improving the image of EU products more broadly than on matters regarding quality. All the programmes generally complied with the European objectives. They did so to comply with the EU regulation and increase their chances of being Japan selected. However, they never built strategies adapted to the market or programme-specific “The lack of clarity in the global EU strategy limits the All three programmes had the possible added value”. Interviewees had trouble defining following objectives. what the objective of EU promotion was. If it was to compensate for the reduction of export subsidies, the Specific objectives view was that the promotion budget should be much higher. Increased demand for The programme documents all referred to similar European products objectives, in line with those of the Regulation (“to increase awareness”, “to develop exports”, “to enhance USA and Canada Consumers more willing to EU quality products”, “to consolidate the image of EU buy European products products”). The objectives of promotion programmes were therefore broad and encompassed many issues. The Global objectives programmes did not demonstrate the relevance of promoting a specific product in a given country. They Increased consumption of were neither “selective nor focused on a given European products challenge” and there was no order of importance. This hampered the development of synergies with other Increased exports of programmes European products “The formulation of the programme objectives was rather a formality than a genuine strategy”. The compliance of objectives with those of the Regulation is Russia supposed to ensure internal coherence of programmes. Thus, the role of the programme objectives was rarely achieved. Few programmes had original objectives. The programmes cover the objectives of the Regulation and hardly mention classical promotion objectives, other than those stipulated in the Regulation. The writing of To increase the demand Switzerland and the objectives appears as a formal exercise, hardly suited Norway To open new markets to reporting the real orientations of the programmes concerned. Source: ADE on the basis of the EU co-financed information and promotion actions between 2002 and 2008 evaluation reports Final Report November 2011 Page 52 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 3.2.2 EU guidelines 32 25F Guidelines for preparing proposals are provided only for internal market programmes in Annex II of Commission Regulation (EC) No 1071/2005. The guidelines are established by sector and suggest messages, target groups and channels that should have a central position in the I&P programmes for the different product categories. They also provide an overview of the situation of the sector and of the goals. As far as EU guidelines are concerned, the framework contract evaluations examined the extent to which these guidelines provided useful guidance to design and implement I&P programmes, and how well they were applied. In other words, this question assessed the extent to which the guidelines were accurate, comprehensive and clear to facilitating the design (i.e. defining channels, target groups, messages and objectives) and implementation of the specific I&P programmes, which were under the scope of the framework contract evaluations. Most evaluations were positive about the role of these guidelines in programme design, especially in guiding objectives and relevance. They were seen as a set of general indications and clear general orientations, which allow identification of channels, target groups, messages and objectives, while at the same time allowing latitude to adapt the I&P strategy to the specificities of the targeted markets. Regarding the implementation of I&P programmes, the guidelines were rather unclear or not useful. Some stakeholders considered them inflexible in terms of budget and calendar modifications (as underlined in the wine sector) or requested clarifications about how to re-use and duplicate the I&P material with a view to improving effectiveness and efficiency (as underlined in the organic sector). Most of the time, guidelines are considered as a useful tool by competent bodies to pre-select the programmes and by proposing and sometimes implementing bodies to understand the purpose and priorities of the I&P programmes in the sector. In most cases, the guidelines allowed proposing organisations sufficient flexibility to design programmes on the basis of their experience. However, a limited number of stakeholders, as mentioned in the wine evaluation, considered them either restrictive or too general. Globally guidelines are well applied. Although some variations were evident, most of the samples of I&P programmes under the scope of the framework contract evaluations pursued objectives, targeted audiences, conveyed messages and used channels as specified in the guidelines. In addition, it is interesting to note that health-related messages were ruled out due to the necessity to base such messages on generally-accepted scientific data and acceptance of national authorities. Indeed, health benefits to consumers were not claimed directly, because of the difficulties in gaining scientific certification. However, many programmes claimed benefits associated with food safety, nutrition, quality and ‘naturalness’. 32 EQ 1.2 of the framework contract evaluations: « To what extent do the guidelines give useful guidance and how well are they applied? » Final Report November 2011 Page 53 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC For example, the milk sector programmes were unable to claim health benefits, given the complex nature of the health impact, but were able to claim benefits associated with being a natural and nutritious product. The programmes of the organic sector that attempted to make the case for health benefits encountered difficulties and delays over obtaining scientific evidence to support this. As a result, attention was not drawn to health benefits and focused instead on environmental benefits. Regarding the wine sector, although actions related to health were included in the design, they were not undertaken in practice because evidence of the positive effects of an appropriate level of wine consumption on human health is more controversial. Processed fruit and vegetables included scientific evidence about the nutritional quality of processed fruit and vegetables in order to improve their image among consumers. All the programmes for fresh fruit and vegetables promoted the products as being fresh, natural and quality products contributing to a balanced diet. In addition to positive feedback about guidelines, some suggestions for improvements were made: The fruit and vegetables evaluation suggested that the guidelines should be extended to provide a common format for programme design including minimum requirements, an explanation and description of the programme strategy, provision of an intervention logic, and appropriate tools for measuring the results and impacts of the programmes, taking account of the types of measures implemented. Some stakeholders interviewed during the organic evaluation voiced demands for improving the methodological guidance for designing programmes and evaluating effects, impacts and efficiency of actions and programmes. In the milk sector, a number of stakeholders underlined the need to clarifying the nature of the guidelines: Are guidelines indicative or prescriptive? 3.2.3 Added value of EU cofunding 33 26F Under this heading about value added of EU co-funding, the contribution of EU cofunding to promotion activities that would not have been undertaken otherwise is summarized here as foreseen by the third standard evaluation question of framework contract evaluations. It considers the extent to which the programmes would have taken place in a different way if there had been no EU co-funding to the programme. Several evaluations found a clear effect on the scale of promotion activities, mostly because the EU co-financing allowed larger programmes with more ambitious activities covering a wider geographical area and including, notably, TV. The other effects of the EU co-funding were the multiplier effects (e.g. implementation of other campaigns) and the possibility to set-up a programme for countries without national funding for promotion and multi-country programmes as far as fresh fruit and vegetables are concerned. Promoting basic products (white liquid milk) which do less interest the private sector was evidenced by the milk evaluation, by this adding credibility to similar campaigns from Member States. The main facts regarding the added value of the EU co-funding are presented in the table below. No clear conclusions were observed from the Switzerland and Norway evaluations. 33 The third standard question of the framework contract evaluations, that was common to both internal market and third country programmes and asked whether the EU co-funding did contribute to promotion activities that would otherwise not be undertaken. Under this issue EU co-funding for multi-country programmes was discussed. Final Report November 2011 Page 54 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 17: Sector / Third country market EU value added Larger programmes Fresh and processed fruit and vegetables Multiplier effects and multi-country Grouping professionals and multiplier effects Wine Organic products Programme set up Multiplier effects Milk and milk products India, China and South-East Asia Covers basic products and adds credibility to MS campaigns Larger programmes Multiplier effects Final Report EU value added of the EU co-funding Comments EU co-funding contributed significantly to enlarging and reinforcing campaigns at national, regional or product level. In the absence of cofunding, programmes would have been much smaller and different in terms of messages (e.g. oriented to trademarks, brands and specific origins), TGs (larger) and information channels used (e.g. no TV-spots due to their high implementation cost). The added-value of EU cofunding lay in the capacity to carry out large scale programmes addressing a wide audience with expected high impact. It also contributed to generic campaigns (not linked to specific origins/brands). There was some evidence that co-funded programmes have facilitated implementation of other campaigns. Stakeholders had identified some multiplier effects. Multi-country programmes had by their very nature promoted cooperation between MS at EU level. However, some stakeholders added that the multiplier effect was also restricted owing to the fact cooperation between different programmes was rather difficult to implement, mainly due to management problems. In addition, it was considered that multiplier effects could be improved if synergies with private initiatives were allowed during the implementation of the I&P programmes. Overall, EU co-financing was considered to generate multiplier effects, notably for implementing large programmes including TV-spots. The EU co-financing had been a clear incentive for grouping together of professionals and for improving sector integration – even though the sustainability of this effect was in some cases questionable. In addition, at least some of the programmes had promoted or facilitated implementation of other (brand-oriented) campaigns. In particular, small producers felt encouraged to organize their own promotion campaigns outside their region of production. The EU co-financing allowed countries without national funding to set up promotion programmes. Indeed, without co-financing, the programmes either would not have taken place or would have been much smaller. Programmes presented leverage effects, grouping professionals, allowing larger programmes and creating incentives for further programmes without EU funds EU co-financing appears to set a relevant focus on products barely considered by private sector promotion strategies, such as white liquid milk. EU involvement in the promotion of such products adds credibility to similar messages conveyed in some Member States In cases where programmes would have taken place without EU funding, that funding allowed the programmes to be larger and longer: for organisations which would have conducted promotion anyway, EU funding enables longer programmes, more diverse instruments and countries or cities to take place. There was some evidence that EU and MS funding leveraged a larger contribution from private enterprise. November 2011 Page 55 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Sector / Third country market EU value added Comments The EU co-funding was an important opportunity for proposing organisations from Member States which do not support promotion. The case of a French & Danish pork promotion programme illustrates the possibility for a newcomer to benefit from the experience of one more experienced in the country. The added value resided more in the financial incentive than in the Programme set Japan “European message” which didn’t play an important role in the up programmes: in other words, the added value of EU backing is more obvious regarding financing schemes (without any EU programmes the implementation of many programmes would not have taken place) than regarding the content of the message (European labelling represents a minor added value, because the European logos are known very little even unknown by most of the interviewees). Programme set The added value perceived in EU co-funding are essentially in up encouraging campaigns which would never occur otherwise USA and Canada Multiplier The EC co-funding had a leverage effect on the programmes (economies effects of scale, bigger target audience, etc.). There were several cases of proposing organisations from countries Programme set Russia where no national funds are available which have had access to funding up thanks to the European programme (Lithuania, Finland) Source: ADE on the basis of the EU co-financed information and promotion actions between 2002 and 2008 evaluation reports 3.2.3.1 The European multi-country and multi-product approach 34 27F Although there was no standard question on multi-product and multi-country programmes in the framework contract evaluations, many of the evaluations did provide information on this under the EQ about promotion initiatives that would not be undertaken without EU co-funding. For internal market programmes, Council Regulation (EC) 2826/2000 mentions the possibility of having applications from more than one Member State. Council Regulation (EC) 2060/2004 introduces the concept that the EC gives preference to multi-country programmes. For programmes in Third Countries, Council Regulation (EC) 2702/1999 specifies that preference will be given to proposals from several Member States and this is sustained in subsequent regulations. Few programmes adopted a multi-country or multi-product approach (see the tables hereafter). The potential benefits of an EU approach were often recognised, including the possibility for newcomers or smaller organisations to join with larger promotion programmes to gain from their experience and to reach a critical mass. However, the evaluations reported that few programmes actually adopted a multi-product or multicountry approach, for several reasons: 34 A multi-product programme is a programme including at least information or promotion activities for two or more agricultural products. A multi-country programme is a programme either put forward by more than one Member State (which may be termed ‘multi-proposer’), or involving measures to be implemented in more than one Member State or in more than one third country (which may be termed ‘multi-target’). Final Report November 2011 Page 56 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC the difficulty of agreeing a single common strategy for all countries and/or products; the specific risk, with multi-product programmes, of confusing customers and diluting core messages; concerns about the administrative burden of the process, including the time, the cost and the administrative effort involved in collaboration amongst MSs, both in the design and delivery of programmes. The evaluations suggested that multi-country or multi-product approaches would not take place without the EU funding. There were several examples where a multi-country programme involved essentially separate activities that were bundled together to give a greater chance of selection. Table 18: Internal market Multi-country and multi-target programmes on the internal market Number Five multi-country Fruit and programmes were vegetables implemented No multi-country and multi-product programmes had been undertaken Wine Three multi-country programmes (of the Milk 39) and two multiproduct (of the 12 sample programmes) No multi-country Organic programmes had been products undertaken Source: ADE on the basis of the EU co-financed reports Final Report Comments The difficulties regarding the design and the costs of managing a multi-country programme are higher than those of managing a single country programme. In certain cases, it therefore appears that multicountry programmes are less efficient than single-country programmes, because it costs more to achieve the same results. Moreover, multi-country programmes for fresh fruit and vegetables have to tackle specificities in terms of content (cultural differences i.e. in food habits), national regulations (on health, access to schools, etc.) and efficiency (increased costs rather than economies of scale). Some stakeholders highlighted the high administrative burden of such programmes. To alleviate this burden certain stakeholders suggested allocating specific budgets for these tasks, that each MS be responsible for monitoring actions implemented within its territory, whatever the origin of the programme, and that the EC have overall responsibility for monitoring them. Stakeholders consider that cultural and sensitivity differences among consumers in different countries and additional management costs for such programmes discouraged Proposing organisations to submit such programmes. One programme explicitly invited other MS programmes to join into their action but the partners gave up due to internal organisation problems and no results came out from this invitation. information and promotion actions between 2002 and 2008 evaluation November 2011 Page 57 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 19: Third country market Multi-product, multi-country and multi-target programmes on the third country market Number Comments Three multi-target There is no multi-country programme and the framework contract India, China and and two multievaluation stresses the need for differences in approach to multiSouth-East Asia product were target countries. implemented All multi-country programmes involved France. There were certain Four multi-country cases of interesting multi-product programmes (promoting three and few multidifferent kinds of wines, or ham and cheese...) or of newcomers Japan product benefitting from the experience of a partner. However, the “multi” programmes were programmes did not necessarily share the same strategy and were implemented sometimes a bundle of separate programmes. There were a few Multi-product programmes were seen as very complicated because multi-product and the “partners do not engage in the same activities”. The bureaucracy USA and Canada six multi-country involved by these programmes has discouraged many proposing programmes organisations. 4 out of the 22 programmes There was one example of a real multi-product and multi-country promoted more programme, promoting wines and cheeses from Austria and Russia than one type of Germany. The other multi-country programmes were not product (in which considered to be a real multi-country programme, as there were not case they promoted many common activities of the countries involved. “product baskets”) There is no real Two programmes were launched by several MS at the beginning but Switzerland and "multi-country" they finally became single-country programmes when some Norway programme proposing organisations left the programmes. Source: ADE on the basis of the EU co-financed information and promotion actions between 2002 and 2008 evaluation reports Promotion programmes targeting more than one third country market. The list of third countries was defined in the Annex of Commission Regulation (EC) N° 67/2005 and N°1346/2005 and includes 19 countries 35 and 5 geographical areas 36. 28F 29F There are many examples where programmes have operated across different third country groups: 70% of the programmes that targeted Japan also targeted other countries (most often the USA) and 80% of the programmes that targeted Russia also targeted other markets (mostly the USA, China and Japan, but also, to a lesser extent, Ukraine). There is no evidence from the evaluations about whether targeting several third countries with the same programmes provided positive or negative results (see multi-country approach hereafter). 35 36 Switzerland, Norway, Russia, Japan, China, South Korea, India, South Africa, Australia, New Zealand, Turkey, Croatia, Bosnia and Herzegovina, FYR of Macedonia, Serbia and Montenegro (and Kosovo), Ukraine, Bulgaria, Romania Middle East, South-East Asia, North Africa, North America, Latin America Final Report November 2011 Page 58 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 3.3 Theme II – Actions and information channels used and their cost-effectiveness The second/fifth theme of the framework contract evaluations was addressing the actions and information channels used and their cost-effectiveness 37. The first question asked whether the types of actions and information channels used were efficient and their cost reasonable in relation to achievement. The second question asked which actions gave the best return in terms of cost versus impact. The third standard question asked whether management methods were appropriate and efficient 38. 30F 31F 3.3.1 Efficiency of the different information actions As defined by Article 7.2 of Commission Regulations (EC) 1071/2005 and 1346/2005, programmes must contain sufficient details to comply with the rules applicable and must enable their cost/effectiveness 39 to be evaluated. Cost/effectiveness may be evaluated at different levels according to the intervention logics of programmes. As they concern a unit cost analysis, they are mostly termed “cost/output” although they may compare the unit costs with outputs (of different activities), or the unit cost of results, such as target group reach (per thousand contacts or per mille contacts (CPT or CPM)) or even the quality of contact. The latter indicators are more cost/result or even cost/impact ratios (if the quality of contact is linked change of behaviour such as increased consumption 32F IM evaluations provided detailed elements about cost-effectiveness. Beside various cost/output ratios linked to various activities, cost/result ratios in terms of target group reach were analysed by most evaluations. They compared namely cost per thousand or mille contacts (CPT or CPM) of different channels. In 2008, the milk evaluation even made a comparison between Above-the-line and Below-the-Line advertising. Above-the-line is a type of advertising through media such as TV, cinema, radio, print, banners and search engines. Major uses include television and radio advertising, web and Internet banner ads. This type of communication is conventional in nature and is considered impersonal to customers. Below-the-Line uses less conventional methods than Above-the-Line strategies. These may include activities such as direct mail, public relations and sales promotions. Below-the-line advertising focuses on direct means of communication, most commonly direct mail and email, often using highly targeted lists of names to maximize response rates. 37 This was the second theme in the internal market evaluations and the fifth theme in the third country evaluations, although the questions were the same in both cases. There is no directly equivalent to the two first questions in the current evaluation. 38 The issues raised in the question are partially covered by EQ6, EQ8 and EQ9 in the current evaluation, which deal with the management framework, the evaluation reports and administration. 39 Although the English version of the Regulation mentions cost/benefit, it is actually cost/effectiveness that is evaluated as some of the benefits are non tangible. The French version of the Regulation always mentions cost/effectiveness so as the EQ of the framework contract evaluations. Final Report November 2011 Page 59 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Various cost ratios Most of the evaluations provided some assessment of efficiency and the evaluations generally concluded that the programmes were adapted to optimise efficiency (score 3.9). However, there were also many examples where reporting of cost-output ratios was partial or subjective and at risk of bias (cf. Table below). Table 20: Sector / Third country market Fresh and processed fruit and vegetables Use and appreciation of the cost-output ratio Appreciation The cost-output ratios are evaluated but present some limits Wine The cost-output ratios are evaluated but their relevance is questioned Milk The cost / output ratios are rarely evaluated and it is difficult to provide them for time horizon over a year Final Report Comments on the use Most programmes include cost-output ratios (or information allowing them to be established) in the programming documents. The costs of the outputs from the sample programmes are in compliance with the budgets for most programmes, and savings have been realised in a few programmes owing to economics of scale, negotiated discounts etc. Cost-output ratios are ensured through public tendering procedures and are in line with national market standards (state-of-art rules). However, a simple addition of cost-output relationships for individual activities might not indicate a clear contribution to cost-effectiveness, as the results of different actions are linked to implementation of other actions. Thus, stakeholders consider that efficiency should not be measured for each activity independently but should be examined within the wider context of a given programme. Cost-impact ratios are more difficult to calculate and interpret than cost-output ratios, because of the important external variables which also influence changes in demand and behaviour. The campaigns remained within the planned budget and respected the envisaged cost-output relation. Stakeholders consider that the costoutput relation has been satisfactory for the types of actions implemented and the information channels used. Nevertheless, the methods used for assessing the cost-output relation are often subjective (unreliable and probably biased) and lack rigour since programme coordinators do not follow established and broadly accepted methods for such assessments. For example, the cost of advertisements versus the number of articles written by a journalist after an event is not a valid method of comparison because (i) articles are a form of public relation whereas advertisements are purely promotional – they reach different audiences and since the receptiveness of the audiences will differ, their impact will also differ; (ii) it is not known whether the written articles were wholly favourable or not; and (iii) articles and advertisements may appear in different sections of a publication and have an entirely different look and feel. The regulation requires sufficient detail to be provided to enable the cost / effectiveness ratio to be evaluated, but there is limited evidence that ratios have been estimated. Several programmes did use estimates of the cost per thousand or per mille (CPT or CPM) and some used marketing indicators such as cost per 100 Gross Rating Points. Where information was available, it suggested that the proposed targets for cost-output ratios are in line with, or below, market benchmarks and November 2011 Page 60 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Sector / Third country market Appreciation Comments on the use the limited evidence from ex-post studies suggests that targets were achieved or surpassed. There were several reasons why proposals rarely provided estimates for cost-output ratios. Firstly, defining appropriate targets was expensive, required data collection and analysis, and would usually be done by the implementing agencies as part of their bid to proposing organisation, which is subject to the uncertainty of competitive tendering and then submission to the MS and EC selection process. Secondly, unit costs in the marketing business tend to be quite volatile, making it difficult to predict costs one or two years in advance. Thirdly, many of the programmes are designed with channels of communication that complement and reinforce each other and it is therefore not useful to try to attribute impact to any one channel. Fourthly, many of the programmes are explicitly designed to have multiplier effects through indirect contacts and it is very difficult to estimate the scale of the indirect contacts. Although there was limited use of cost-output ratios, two of the milk programmes use a scoring system to select the most efficient channels of communication. It was difficult to judge effectiveness of the measures because, although programmes had objectives, implemented measures didn’t. Strong Furthermore, it was practically impossible to compare different Organic difficulties to programmes because of their connections with other events and products assess campaigns, rendering the single effect of the implemented measures within one programme impossible to assess. Source: ADE on the basis of the EU co-financed information and promotion actions between 2002 and 2008 evaluation reports Channels The evaluations suggested that there is no general rule about what channels are most efficient and that all channels play a role for different products, countries, target groups and phases of information and promotion campaigns for agricultural products (cf. Table 21). Final Report November 2011 Page 61 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 21: Sector / Third country market Efficiency of Different Channels Appreciation TV seems very efficient in terms of target group Fresh and reach however the processed fruit combination of and vegetables different channels need to be considered to assess efficiency Comments Efficiency should not be measured for each activity and channel independently, but should be examined within the wider context of a given programme. Indeed, efficiency results from synergies between different measures within a programme. Measuring efficiency in terms of target group reach appeared difficult. However, there was some suggestion that TV was a relatively efficient channel. For most programmes the design favoured information channels involving direct contacts with consumers or opinion leaders as they are deemed to produce more durable impacts – like the creation of a “culture” around the product – and benefit from multiplier effects. Wine Direct contacts with consumers or opinion leaders more efficient Milk CPM (cost per mille) Organic products Qualitative comparisons of channels depend on the countries Working with India, China and professionals is South-East Asia more efficient Final Report Although stakeholders agreed that most of the activities and messages are well adapted to target group specificities and needs, they did not further justify that judgement with concrete arguments. Assessment surveys led by some implementing bodies showed a high degree of satisfaction after events and positive results in raising target group awareness of the product. In addition, programmes produced innovative outputs well suited for particular target groups, such as a website to reach internet users of age between 20 and 40 and training for caterers, restaurateurs and distributors. The analysis of a sample of promotion products showed that both content and layout are attractive and well designed in regard to the selected target group despite a few inconsistencies or lengthy texts. The milk evaluation found that when comparing the different efficiency ratios in terms of costs versus results, the CPM for electronic media and below-the-line channels appeared to be generally much higher than for above-the-line media. On the other hand these types of channel offer a higher quality of contact and in some case a secondary target group is reached, lowering the cost per impression (e.g. in schools). Qualitative comparisons gave different results depending on the countries, e.g. in France, supermarket actions are considered to be a good way to have a short term impact, whereas events are considered more interesting in Austria. There was a strong reliance on working through professionals and this was considered the most efficient option, in view of the very specific nature of the markets and the fact that contacts with professionals is relatively inexpensive. There is a strong reliance on contact with professionals, including use of fairs in China, and a strong focus on the quality message. Some programmes supplement these costs with other tools, like press relations, promotion at points of sale and fairs. Unit costs for the range of activities are roughly comparable across all programmes suggesting that they are consistent with standard market rates. Some activities which might be more costly are justified because November 2011 Page 62 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Sector / Third country market Appreciation Comments they complement and reinforce the more efficient contact with professionals and help to achieve credibility and a critical mass. The quality of the information and promotion activities is rated highly by local stakeholders. However, there was frequent criticism that the programmes are too passive and that there is a lack of business orientation and of participation by businessmen. There were difficulties in establishing good working relationship with key local partners and, especially, of working nationwide. Competitors (e.g. from the USA) are more involved in consumer-oriented activities, including TV campaigns and consumer events, such as tastings and vouchers. Most programmes used the participation in fairs as a first point of Limited but mixed promotion. It was not considered to be enough to increase shares on channels (events the Japanese market, where human relations are very important to Japan and direct build networks. The use of too many different channels was contacts) considered inefficient in the Japanese market. Actions targeting journalists generally had good impacts in the press, while tastings and fairs seemed to provide good short term results. USA and Canada Mixed channels However, professionals seem to be sceptical regarding the actual impacts of the campaigns. In-stores promotion represented 50 percent of the promotion budget Difficulties due to although the market structure of Russia (large variety of eclectic Russia the market shops) makes it very difficult to have an efficient in-stores promotion. structure Concerning fairs, it is easy to target the appropriate fairs, but it’s difficult, within a fair, to ensure that professionals will be reached. The activities allowing for direct contact between promoters and target audiences (tasting, trade fairs, and point-of-sale promotion) were more effective and had a better cost/effectiveness ratio. The programmes based on the promotion of the product itself, particularly effective in terms of improvement of the image, had used public relations actions, whereas the less efficient promotion programmes Switzerland and Direct contacts are based on the building and spreading of a European image had used a Norway more efficient wider range of tools, including among other things publicity (through billboard campaigns and the press). Publicity, especially the poster campaigns, had a relatively high cost when it was the main tool and not a support for direct promotion actions. When it conveyed an exclusively European message, or when it was not displayed regularly, it appeared to be a particularly expensive tool in relation to the lack of evidence of impacts. Source: ADE on the basis of the EU co-financed information and promotion actions between 2002 and 2008 evaluation reports Messages The choice of message reflects the specific objectives and the target group. Most of the evaluations provided conclusions on specific objectives (section 2.1.1) and target groups (2.3.2) and these are covered elsewhere in this synthesis. The importance of messages Final Report November 2011 Page 63 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC regarding nutrition, quality, safety and naturalness is also described elsewhere in the synthesis. Two of the evaluations provided additional insights that may be more generic. The organic sector evaluation highlighted that programmes concentrating on a single message had the greatest success. Moreover, more complex messages often resulted in confusion and recall of different messages to those intended. The USA-Canada evaluation stated that the North-American market was considered to be responsive to branding and labelling. The fact that EU co-funded programmes were limited to generic promotion had limited the beneficial impact of the programmes on consumption and awareness. 3.3.2 The overall management set-up of I&P programmes Article 7 of Regulation (EC) No 1075/2005 provide details about the overall management set-up behind the design and implementation of the I&P programmes. Proposing organisations (such as trade and interbranch organisations) prepare their I&P programmes for submission to the competent body. If at this stage they have already selected an implementing body (one or more communication/marketing agencies) through a competitive procedure, the implementing body contributes to the elaboration of the programme. Should this not be the case, the proposing body has to select an implementing body before the contract with the EC services is signed 40. Competent bodies provide support to proposing organisations and pre-select the best proposals, submitting these to the EC, who make a final selection. 3F During implementation, proposing organisations submit the required reports to competent bodies, who check and comment on them and, when they are approved, submit them to the EC for reimbursement of EC co-funding. If proposing organisation wish to change their work programmes during the year, they submit requests to the competent body, who have the authority to approve changes, providing a contract amendment is not required. The framework contract evaluations addressed the issue of the overall management set-up through the question dealing with management methods (question 2.3). The first level of this question dealt with the management overall set-up: Is the overall set-up, from the European Commission through national authorities and professional organisations down to the implementing bodies, appropriate and efficient? Based on lessons learnt from the framework contract evaluations, few comments on the overall management set-up suggested that there were few problems. Indeed, most stakeholders considered the overall management set-up and distribution of management roles appropriate, with the exception of: programme selection procedures (for further details, cf. section below devoted to the selection process of programme proposals); Several efficiency problems pointed out by the USA and Canada as well as Switzerland and Norway evaluations. Management was considered a bureaucratic 40 This procedure was introduced by EC Regulation No 1071/2005. Under EC Regulation No 94/2002 implementing bodies had to be selected during the programme design stage. Final Report November 2011 Page 64 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC burden and communication between proposing organisations and the Commission is rare; A need pointed out the organic evaluation for clarification of the roles of each party in the selection procedure, as well as a need for improved transfer of experience-based knowledge throughout the Member States. 3.3.3 Administrative requirements The administrative requirements are defined by the Commission Regulations (EC) N° 94/2002 and (EC) N° 2879/2000. The guidelines are detailed in the Annex II of Commission Regulation (EC) N° 1071/2005. They concern both the selection procedures and the implementation procedures. 3.3.3.1 Selection process of programme proposals The selection procedures are defined for those submitting programme proposals to the EC. These procedures govern the calls for proposals launched at Member State level, as well as the preparation of the list of programme proposals selected by Member States. Programme proposals should include: provisional detailed budgets and the types of expenditure; a strategic and marketing analysis; a coherent strategy with objectives, proposed measures with selected channels, messages, target groups and countries; duration and timetable for implementing measures; and monitoring and evaluation considerations. All the financial, economic and technical capacities of proposing organisation(s) and implementing organisation(s) should be demonstrated. Once the programme proposals have been elaborated, the proposing organisations may submit them once a year (e.g. 30 November for the programme proposals on the internal market) to the competent body, which evaluates the programmes’ quality and assesses whether they respect the Regulations. If the quality is judged sufficient and the programme conforms to the Regulations, the programme is sent to the Ministry. Before sending I&P programmes to the EC Services for approval, the Ministry pre-selects the best quality programmes for all agricultural products covered by Council Regulations No 2826/2000 and N° 2702/1999. The framework contract evaluations addressed the issue of selection process of programme proposals through the question dealing with management methods (question 2.3). The second level of this question dealt with the selection procedures: Are management methods, including selection and other implementation procedures, use of human resources, timeliness (etc.) appropriate and efficient? These framework contract evaluations suggested that there was a lot of criticism about the respective roles in the selection of proposals, where the Member States had generally failed to fulfil their responsibility of pre-selection and where the criteria from the European Commission were not very explicit. This had contributed to a feeling amongst applicants that selection decisions were not well explained and to calls for clearer evaluation criteria. Final Report November 2011 Page 65 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Although EU guidelines for designing programmes (cf. above Theme 1: Coherence between the implementation of the programmes and the objectives of the regulation for further details) were considered good, there was widespread uncertainty about how selection was done. In many cases, there were complaints about a lack of clarity and confusion about the selection and the use of other criteria than those mentioned in the Regulation. The criteria mentioned in the Regulation often appeared to be a formality which had little effect in the balance of decisions. As highlighted in the milk sector evaluation, there appeared to be little pre-selection among some Member States. Stakeholders stated that they were not confident that the selection criteria were being objectively applied and that they suspected that other issues may be influencing selection decisions. The evaluations of the wine and organic sectors as well as USA and Canada also pointed out that the reasons behind the final decisions of approval or rejection of programmes by the EC were not communicated to or understood by the Member States. The organic sector evaluation and the wine sector evaluation highlight that the approval process was a duplicate process: both the MS and the EC had to go through the programme details and make their own evaluation. The organic sector evaluation report suggests transferring the competence to the MS entirely. The fruit and vegetables sector evaluation considered that both stakeholders at national level (competent body and Ministry) and at EU level needed to be involved in programme selection. There was some evidence that involvement of producers in the design and implementation of programmes has strong advantages, for efficiency and effectiveness and to avoid duplication. In the milk sector, several proposing organisations included industry representatives on their boards, which helped to ensure that messages benefited from the latest marketing experience. 3.3.3.2 Implementation procedures The implementation procedures are defined for Member States during the implementation. Among others, these procedures provide for performance security, the format of the signed contracts, the issues of documents relating to advances and interim payments, the reports on annual checks, and the provision of information on checks carried out for multi-country programmes. As far as implementation procedures are concerned, Member States and their proposing organisations need to meet several requirements. According to the Commission Regulation (EC) No 94/2002 and (EC) N° 2879/2000: The annual evaluation report is to be submitted by the proposing organisation to the Member State within four months of completion of the annual programme measures. This report is mandatory in respect of payment of the balance of the co-funding of the programme. It must include a summary of the work carried out, evaluation of the results obtained and a summary financial statement showing all scheduled and incurred expenditure. Final Report November 2011 Page 66 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The quarterly monitoring reports have to provide clear and accurate information on implementation of the programme at measure or programme level. It shall cover: funds spent progress of measures, channels used, duration, delivered outputs and use of promotion material. The co-funding payments in several tranches are made on the basis of the quarterly monitoring reports. The framework contract evaluations addressed the issue of implementation procedures of programme proposals through the question dealing with management methods (question 2.3). The second level of this question dealt with the selection procedures: Are management methods, including selection and other implementation procedures, use of human resources, timeliness (etc.) appropriate and efficient? Monitoring and Evaluation of the programmes Most of the framework contract evaluations concluded that quarterly monitoring activity focused mainly on expenditure and that little information was produced on outputs. There was very little evaluation activity as part of the management of programmes and no reports of ex post evaluation studies for any of the programmes. Only three of the framework contract evaluations provided objective information on efficiency, such as costoutput ratios. Some details regarding the milk and fruit and vegetables sectors are presented below to illustrate these statements. The milk evaluation highlights that the quarterly and annual reports described only the bare minimum of information about activities undertaken and there was very little information on any of the additional issues (e.g. on timing, place, scale and outputs). Even when a milk programme had good information on cost-output ratios and other evidence on efficiency, this information was not included in the quarterly and annual reports. In several cases, the estimates of cost-output ratios were used to change activities, but this was not reported in the quarterly and annual reports. Stakeholders remarked that they never received any feedback from the EC on the content of these reports. It seems that, in the milk programmes, the quarterly and annual reports were not used for their intended purpose. The quarterly and annual reports of the fruit and vegetables sector rarely provided information about programme results and impacts. Rather, they presented mainly a summary of expenditures scheduled and incurred as well as some outputs at activity level. In this respect, most stakeholders mentioned a lack of common approaches and methodologies at the EU level for evaluating I&P programmes. Moreover, the organic products and the wine evaluations pointed out that there were no cases of sound or in-depth evaluations of the programmes, providing information on the target groups reached, the results or the impact of the programmes. Administrative burden The framework contract evaluations included limited information on the efficiency of the approval process. There were concerns about the confusion in roles between the Member Final Report November 2011 Page 67 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC States and the Commission (as explained above under the section Selection process of programme proposals). In addition, four of the nine framework contract evaluations mentioned that the details required for applications is a substantial burden. Indeed, in the milk sector, stakeholders found that it was hard to comply with the amount of detail to be submitted for the budget and schedule. Many of the savings obtained by high efficiency or by focusing on the most efficient channels were lost to the programmes because they occur towards the end of a budget year. Moreover, it was not possible to expand activities in other areas to disburse funds before the end of the programme. The main reason for this was that changes in activity and budget transfers have to be approved by the competent body. Because proposing and implementing organisations were aware that the rules on carryover meant that they would gain little benefit from making savings, they had little incentive to make efficiency gains. Likewise, some stakeholders of the fruit and vegetables sector found the information required for evaluating the programmes at the moment of sending them for approval to be too detailed (e.g. the accurate cost-output ratio of media actions over 3 years). The organic sector evaluation pointed out that the requirements were sometimes considered to lack flexibility and restrict alternative approaches and in some cases, the competent bodies’ interpretation of the guidelines resulted in delays. For the India, China and South-East Asia evaluation, there was almost unanimous agreement amongst proposing organisations that the administrative burden of the EC programmes was too heavy and not well adapted to the task. The requirement for detailed costing was considered inappropriate because prices for marketing activities are so volatile. This resulted in frequent amendments to the contract, since carryover was not allowed and budget transfer was limited. In preparing applications, there were particular problems understanding which actions would be eligible. The complexity is further increased by the need to meet MS requirements as well as EC requirements and by the fact that the level of knowledge about the programmes in MS Competent Bodies is very variable. Application deadlines Two out of the nine framework contract evaluations suggested that the programmes would be more efficient with 6 monthly application deadlines. Indeed, some national stakeholders of the fruit and vegetables sector suggested the possibility of proposing I&P programmes to the EC Services for approval twice a year instead of once, which would allow a more continuous presence in terms of communication 41. The India, China and South-East Asia evaluation highlighted that the infrequency of the call for proposals creates problems (for example, if a call is missed, organisations have to wait six months to present a new programme). 34F 41 In the first years of the Regulations (2000-2004) programmes could be introduced twice a year, both for the IM and for TC. Final Report November 2011 Page 68 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Funds adaptation Two of the nine framework contract evaluations recommended relaxing the rules that prevent carryover of unspent funds from one year to another. The milk sector evaluation suggested that more flexibility over transfer between budget lines would be useful, in order to help programmes adapt to lessons learnt during implementation. But there were five examples of changes to activities that had been made successfully within the existing regulations and had improved the performance of the programme by cutting or changing activities that were considered inefficient. Many of the India, China and South-East Asia programmes used contract renegotiation and this often resulted in larger than 10% savings. It would therefore have been very useful to allow transfer of more than 20% between budget lines. 3.4 Theme III – Coverage and content of the programmes The third standard theme in the framework evaluations examined the coverage and content of the programmes, focusing on the size target groups, and the duration of the programmes 42. 35F 3.4.1 Size 43 36F In terms of size, the I&P programmes showed important differences. The evaluations on internal market programmes were generally positive that the size of the programmes was sufficient to achieve objectives, bearing in mind the size of the target group. For the Third Country markets, all the evaluations except Switzerland and Norway reported that the budget was insufficient to achieve the objectives, even after providing a geographical focus on selected cities. Milk: The size of target group was reflected in the very wide range of budgets of the programmes, which ranged from €140,000 to €10.5m. Most programmes could be classified either as: small, in target group and budget, with highly focussed actions; or large, in target group and budget, relying on mass communication. However, two of the programmes were large but used focussed target group actions. The reasons for the variation in size of programmes seemed to be related to the number of proposing bodies involved in each MS: where MSs had several potential proposing bodies, the tendency was for a larger number of smaller programmes to be put forward. There was some evidence that larger programmes achieved the sort of strategic impact that was more appropriate for EU funding. There were no explicit selection criteria relating to size, but stakeholders believed that the EC preferred larger 42 There is no direct equivalent of these questions in the current evaluation. 43 « Are the programmes co-financed of sufficient critical mass to have effective coverage? » EQ 3.1 of the framework contract evaluations Final Report November 2011 Page 69 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC programmes, provided this did not result in excessive expenditure per capita for smaller countries. Smaller programmes were generally of shorter duration and larger programmes were longer, which seemed appropriate. However, the longer programmes were often made up of a series of shorter term actions and it was essential to be able to adapt the campaign through the period as it was not possible to foresee in the design what will be most effective in the second or third year. Fruit and Vegetables: It seemed that the programmes are all sufficiently adapted to the needs of the specific target groups to be effective. Indeed, they had sufficient critical mass to cover the TG’s in line with expectations. The budgets of the programmes range from €160,760 to €19.66m. Organic products: The co-financed programmes – as a combination of campaigns – had a sufficient mass to reach a significant proportion of the groups to hold their attention and build on changed behaviours in the long run. They could also have sufficient mass to ensure that a message is “absorbed”, but only by focusing on one message at a time, which must be simple and well defined. The budgets of the programmes range from €225,000 to €7.5m. Programmes were considered to have a sufficient size to reach the required critical mass to contribute to changes in the long run. In some cases, where the results were unsatisfactory, they were attributed to the quality of the programmes rather than their size or duration. India, China and South-East Asia: Most of the Asian programmes were relatively long and large, reflecting the importance of the market. They tended to focus on a few cities, which was a pragmatic approach. Exploiting the opportunities for large international fairs was appropriate in China. The budgets of the programmes range from €308,800 (sample programmes) to €3.9m. Japan: Given the costs of adapting the language and the message, it was considered necessary to have large-dimension activities in order to have an effect: small-scale actions were not efficient by themselves, unless they were anchored to larger programmes. In general, there was a lack of adequacy between resources available, the audience targeted and the objectives of the programmes. The budgets of the programmes in the sample (budget for Japan) range from €112,033 to €4.1m. Russia: The choice of measures was generally adapted to the budgets of programmes, but the objectives of the programmes were most of the time too optimistic given the budgets. The budgets of the programmes in the sample (budget for Russia) range from €36,630 to €2.1m. USA and Canada: Budgets were quite low compared to the objectives of the programmes. In other cases, such as programmes implemented in Canada, actions were considered to be too dispersed to reach a critical mass. The budgets of the programmes in the sample range from €300,000 to €4.5m. Final Report November 2011 Page 70 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Switzerland and Norway: The allocated budgets are sufficient to fulfil the main requirements of a promotion programme. The budgets of the programmes in the sample (budget for Switzerland and Norway) range from €90,000 to €645.030. 3.4.2 Target Groups 44 37F On the internal market, the target groups for I&P programmes are defined in the Guidelines in the Annex II of the Commission Regulations. These are mainly consumers, sometimes targeting specific subcategories of consumers or so-called secondary or intermediary target groups such as doctors, nutritionists, opinion leaders, etc. Some of the target groups are listed below: Organic products: consumers in general, consumer associations and specific subgroups of consumers, opinion multipliers, distributors (supermarkets, wholesalers, specialised retailers, caterers, canteens and restaurants), food processors, teachers and schools. Wine: distributors, consumers (excluding young people and adolescents referred to in Council Recommendation 2001/458/EC, opinion leaders (journalists, gastronomic experts), and educational establishments in the hotel and catering sector. Fruit and Vegetables: - Fresh: young households under 35, school-age children, adolescents and their parents, mass caterers and school canteens, doctors and nutritionists. - Processed: households, mass caterers and school canteens, doctors and nutritionists. Milk and milk products: children and adolescents (especially girls aged 8 to 13), women of different age groups, and elderly people. Target groups are not explicitly defined for third country promotion programmes. However, the evaluations did assess the extent to which the co-funded programmes targeted the appropriate size of groups given their budgets. This element is discussed in the “size” paragraph before. Most of the evaluations reported that the channels and messages were well adapted to target groups. This adaptation was most successful when working with experienced exporters and with local professionals and agents, both in third countries and in the internal market. Milk: The guidelines did not distinguish between primary and secondary targeting. Thus, in some circumstances, the target group was an intermediary (e.g. teachers or health professionals) and there was no specification of the primary target group of consumer. There was a wide range in the size of the target group, with some programmes involving only a few thousand school pupils or producers and others 44 « Do the programmes reach a sufficiently wide audience or target group to be effective » EQ 3.2 of the framework contract evaluations Final Report November 2011 Page 71 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC reaching millions of consumers. However, the size of the target group was, in fact, rarely specified in clear terms and there was often no explicit distinction between the total number of people in the target group and the number that were expected to be reached, either through primary or secondary contacts. Fruit and Vegetables: The evaluation concluded that the programmes are all sufficiently adapted to the needs of the specific target groups. They had sufficient critical mass to cover the target groups in line with expectations and, indeed, some stakeholders suggested that there was scope to expand the target groups without additional budget, if justified by the context analysis. Organic products: The analysed programmes showed important variations among target groups, more or less in line with the guidelines, but showing a broad scope. Several programmes included “producers”, “households”, or “consumers” in the target group. They also generally included more specific targets, such as doctors or school establishments. Japan: The evaluation showed that targeting not only to professionals but as well directly to consumers is relevant when, at the same time, products are already known on the targeted market and, and resources of the programme are large enough. USA and Canada: Given the size and fragmentation of the North American market and the budgets of co-funded promotion programmes, there was a need to target the actions. Programmes therefore generally focused on professionals and on a few dense regions (e.g. New England and the Mid-East). Russia: All programmes focused on two cities: Moscow and Saint Petersburg. This choice was considered appropriate, given most importers are based in those cities of highest population density and purchasing power. Nevertheless, other emerging cities could be interesting in the future. India, China and South-East Asia: The promotion activities undertaken correspond to the targeting of the programmes towards professionals rather than consumer targets. 3.4.3 Duration 45 38F EU co-funded programmes could last from one to three years. In most cases, the programmes were implemented for three years. Evidence on whether the duration of programmes was sufficient is mixed: internal market programmes were highly positive (score 4.5), whilst third country programmes were markedly negative (score 1.8) considering 3 years too short. The evaluations were clear that one year programmes are only useful for pilot activities. The narrow objective of improving awareness could be achieved in three years, except if products were totally new to a market, but there was less 45 « Are the programmes of adequate duration to have a significant impact » EQ 3.3 of the Framework Contract Evaluation. Final Report November 2011 Page 72 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC agreement about whether a significant and sustained change in consumption could be achieved in three years. Most of the third country evaluations pointed out the need for promotion activities to be sustained, if there is to be any long term impact. Milk: Whilst three years is sufficient to have an impact on awareness, a much longer programme of at least ten years is required to have a major and sustained impact on consumer behaviour. Some stakeholders suggested that there should be opportunities to extend programmes that are highly successful. However, competent bodies were concerned that this would limit the scope for new programmes to be funded. Fruit and Vegetables: Programme durations of 3 years (or 3 seasons) were considered adequate to achieve objectives (in terms of results). One-year programmes were only useful for pilot actions. Stakeholders stressed that uninterrupted presence over the long term was vital, in order to change bad food habits and to maintain reasonable food habits, when facing competition from the agro-food industry, especially for young consumers. Organic products: Stakeholders considered three years to be an appropriate duration for programmes. On the one hand, it allowed them to reach a sufficient critical mass and on the other, longer programmes would require to renew the content. However, three years were not sufficient to pass on complex messages or to provoke spectacular changes in demand. Unsatisfactory results were however rather attributed to the quality of the programme rather than its duration or size. Wine: Duration of three years was considered adequate for reaching the programme objectives. Shorter programmes would have risked being ineffective and longer ones would have had to be updated to take into account market evolution and changes in target group needs. USA and Canada: Three year programmes were too short for new products on the market. Quite often, EU programmes were part of ongoing actions of proposing organisations in the country. The large majority of programmes focused on professionals and few of them went through local sales points to reach consumers. Russia: According to promoters, the market was characterised by certain elements (complicated and variable custom rules and high concentration of food importers) which made it more difficult for them to penetrate within the three year scope of the programmes. If a product was already on the market, three years was considered to be an appropriate duration for the programmes. According to Russian professionals, products that don’t succeed in penetrating within the market throughout the programme were not well adapted to the market. Switzerland and Norway: One to three-year programmes had been implemented in Norway and Switzerland, although duration of five to seven years was considered by professionals as the minimum required, especially if the European programme was the only on-going campaign for the product concerned. The duration of the programmes was not long enough for objectives to be met, unless there was an extension. Final Report November 2011 Page 73 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 3.5 Theme IV – Impact and effectiveness of the measures The fourth standard theme in the framework contract evaluations addressed the impact and effectiveness of the programmes. The first question dealt with improvements in the image of EU products and themes, the second question asked whether it was possible to establish any link with the evolution of demand for European products. In the third country evaluations, the third question asked whether new markets and export opportunities had been opened up, whilst in the internal market evaluations, the third question asked whether the portfolio made a coherent whole 46. 39F 3.5.1 Reputation of EU products/themes and potential link with demand This section concerns the impact of the programmes on the image and reputation of EU products and at a second stage any potential link between the programmes and the evolution of demand of EU products. Sector- or country-specific comments are available in the table below. The impact of the programmes on the evolution of institutions is also described. Most of the evaluations reported that the programmes had an impact on awareness (score 3.4). The milk sector evaluation suggested that the impact on awareness might decline rapidly, but this was because general levels of awareness were high anyway. Some evaluations claimed that it can be inferred that increased awareness will have generated a change in consumption habits. Several evaluations (India, China and South-East Asia, USA and Canada as well as milk and wine sectors’ evaluations) stressed the importance of the quality of contact, as an indicator of the likelihood that awareness will be converted into changing consumption habits. Overall the evaluations suggested that it was not possible to identify the impact of the programmes on consumption, in general (score 2.1). However, it was possible to observe impact on consumption only for a few activities with specific target groups (milk, wine, fruit and vegetables and organic sectors) although this impact was sometimes temporary. As a side effect, there was some evidence that the EU programmes had an impact on strengthening professional organisations. One milk promotion programme illustrated the feasibility of having all stakeholders involved in the process. In the organic sector, programmes had a leverage effect on grouping professionals. 46 The current evaluation contains no questions about impact. Final Report November 2011 Page 74 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 22: Sector / market Impacts Awareness Fresh and processed fruit and vegetables Demand/ Consumption Awareness Wine Demand/ Consumption Awareness Milk Demand/ Consumption Evolution of institutions Organic products Final Report Impacts of the programmes Demand/ Consumption Comments Large majorities agreed that the promotional materials and activities contributed to increased awareness among pupils and their parents of the health benefits of fruit and Vegetables consumption. Nevertheless, evaluation points out that ex post surveys measuring more precisely the results and especially the impacts of the programmes in terms of awareness-raising and change of behaviour should be implemented after programme termination to report on achieved impacts on the long run. In terms of demand, no causality link between I&P programmes and the evolution of EU demand for fruit and vegetables could be provided, and it was not easy to establish the link with demand for fruit and vegetables in general, owing to the influence on demand of many external factors. However, there were indications that one programme contributed to reversing a falling trend in consumption, especially of fresh fruit among young consumers (<35). Another programme showed increased consumption by children at school and at home. Assessment surveys led by some implementing bodies showed a high degree of satisfaction after events and positive results in raising TG awareness of the product. Although the evolution of wine consumption in Europe is the result of a complex combination of factors and therefore difficult to relate to one single factor, it could be said that the programmes had a positive impact on demand for the promoted products Recognition rates were reasonably high (31% to 36% for most programmes, although one only achieved 10%), but fell rapidly after the end of the programme. Surveys for several of the programmes suggested that recall rates were 10% or less after only a few years. This may reflect the fact that many of the milk programmes promoted messages associated with naturalness, health and diversity of options that were already wellknown (with adherence rates of 85% to 90%). There was also some evidence that the impact of the EC funded programmes may have been mixed with other programmes, such as those associated with osteoporosis and with branded products. There was some variation in demand for milk and dairy products, but demand for milk and butter was saturated and falling in most EU member states, despite falling prices. Thus most of the programmes aimed to reverse this decline. There was, however, scope for expanded demand for dairy products and cheese. Three of the programmes reported changes in consumption amongst the target group (varying from 3% to 7% increase) and the synthesis evaluation did target groups surveys, suggesting that between 30% and 90% of target group consumers would consume more, for most programmes. Despite this evidence, there was little evidence of the impact on consumption at national level. The EU programmes had an important role in promoting greater stakeholder collaboration. It was rarely possible to detect the impact of the programmes because it was rarely foreseen in their set-up. In cases where it was measurable through surveys, the impact was considered greatest among people who were already well informed and buyers of the products. In general, the November 2011 Page 75 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Sector / market Impacts Evolution of institutions Awareness Demand/ Consumption India, China and South-East Asia Evolution of institutions Awareness Japan Demand/ Consumption USA and Canada Demand/ Consumption Russia Demand/ Consumption Final Report Comments sales of organic products have been increasing in all MS except for Italy, but the trends cannot be properly attributed to the programmes. The EU co-funding had a regrouping effect for professional organisations and allowed going beyond sector-specific programmes. The target group was mainly professionals and actions to raise awareness amongst professionals accounted for almost 85% of the budget. There was a strong impact on awareness of professionals, especially in the newer Chinese market, but also India. The experience with building professional networks was more limited. Impact on consumer awareness is also limited, with exceptions (especially for the Le Crunch ‘generic brand’ for apples). It was difficult to obtain evidence of the impact of programmes on sales. There is some evidence of growing sales, but this could be affected by other factors, such as harvests in competing producer countries and exchange rates. The impact on distribution patterns was also limited. Of the nine sample EU programmes, five were conducted by organisations that had no previous experience marketing in Asia and the EU programmes therefore had a major effect on exploring new markets. The EU programmes had a positive effect on the capacity of Proposing Organisations and associated producers that were new to marketing in Asia, or were new to certain promotional practices, but limited effect on experienced Proposing Organisations and producers. There is some evidence that, even with experienced Proposing Organisations and producers, the EU programmes helped to improve capacity in defining strategies and designing programmes, which has led to clearer programme strategies and targets and better adapted actions. There are few cases where the EU programmes have encouraged wider producer participation in joint promotion activities. One of the campaigns had important impacts (98% of surveyed consumers remembered the Beaujolais campaign – and most of them remembered it well) whereas most of them reached less than 50% of the surveyed consumers Promotional programmes did not lead to identifying or creating new commercial relations or markets and did not change perceptions of the promoted products. There were however more positive short term results: the participants in seminars showed interest in using the promoted products more often. Most programmes focused on maintaining market shares of products that were already well established on the market rather than opening new ones. The impact of programmes was highest for products already well known on the market or regularly consumed (high levels of remembrance of the Greek olive oil, the Parmigiano Reggiano and Beaujolais, Côte du Rhône and Port wines campaigns). The improvement of the image and level of knowledge did not necessarily lead to increases in consumption of products. Furthermore, the net effect of generic promotion could not be properly assessed (although a net positive effect was detected for some products). According to surveys, the programmes had a positive impact on business opportunities in the country, by creating new professional contacts and November 2011 Page 76 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Sector / market Impacts Comments increasing trust. The impacts in terms of image and consumption showed differences among programmes, but the most remembered programme was only remembered by 25% of the survey respondents. The consensus among professionals was that generic promotion is an efficient tool to raise awareness of European products but the evaluation points out that the European promotion programmes had little impact on Awareness the building and spreading of a European image of quality, for lack of identification of the products as European, and for lack of awareness of Switzerland and the quality label system. Norway It was impossible to draw a causal link between promotion programmes Demand/ and increases in sales except in the long term. At best, programmes had Consumption helped to boost sales but were by no means the only explanation for this upward trend. Source: ADE on the basis of the EU co-financed information and promotion actions between 2002 and 2008 evaluation reports 3.5.2 Knowledge of EU Logos The term “EU logo” is used to refer to the EU quality label logos (PDO, PGI, TSG and organic logos 47, represented below) and the EU flag. One of the objectives of the promotion policy is to increase public awareness regarding these logos and associate the European labels (and the European origin of products) as a quality guarantee for the consumer. This paragraph focuses on the quality logos, whereas the “EU image” paragraph assesses the latter issue of associating the EU origin with a certain quality of products. 40F Figure 12: European labels (former and current PDO; PGI, TSG, former and current organic logos) Former PDO Former organic PGI TSG Current PDO Source: DG Agri Website 47 Current organic The EU adopted new PDO and organic logos, respectively in 2009 and 2010. At the time of the past evaluation reports, the former PDO and organic logos were the one represented in promotion programmes for organic products. Final Report November 2011 Page 77 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The evaluations suggested that there were substantial challenges in promoting the EU logos (score 2.1) and it was rare for programmes to devote resources to promoting the EU logo. Knowledge of the EU logos in third countries seemed to be very low, even in Switzerland and Norway. Furthermore, promoting the EU logos was seen as one of the most expensive activities as measured in terms of cost per successful contact. There was some sign of success in improved recognition of the EU logos for fruit and vegetables in the EU10 and for organic products (Table 23). Table 23: Product/ Region Organic products Wine Fruit and Vegetables Milk Logos promoted by co-funded programmes and results Logos promoted Organic PDO / PDO USA-Canada PDO, PGI, TSG & organic products Japan PDO, PGI, TSG & organic products Final Report Results Only two of the programmes provided data that allowed measuring the knowledge and impact of the programme on the perception of the EU organic label. In France, far more consumers were familiar with the national organic logo (92%) than the EU one (less than 50%). In Denmark, the levels of recognition are closer, but confidence in the national logo remains higher. The survey in France showed that, whilst familiarity with the EU organic logo was high, 80% of people who were familiar with the logo did not change their opinions or their purchasing habits as a result of the logo. Although there had been programmes carrying out messages on EU labelling, the evaluation did not emphasize this point. It did however state that the high increase of wines with an indication of the geographical origin “weakened the value of the quality wine classification system and the credibility of the label”. Thus, the expansion of use of the label of geographical origin was reducing the exclusivity of the label and the advantage provided by the label. The evaluation concluded that, in the EU15, there was no EU image for fresh fruit and vegetables and that consumers recognised national and regional images. However, there was evidence that the EU10 did value the EU logo for citrus fruit. The milk evaluation report analysed two programmes promoting PDO milk products. However, there were was no assessment of the progress of the logo recognition due to the promotion actions. The EU labels were quite difficult to promote. Consumers did not know of them and they referred to notions consumers were unfamiliar with (e.g. “terroir”, which was somewhat known in Canada but not at all in the USA). European origin in itself is not actually considered as a relevant argument for promotion: indeed, the European nature was not considered to be a reason for buying certain products because “Europe is not selling on its name”. The EU guarantee could however be useful for newcomers from regions or countries unknown on North American markets and for products without pre-existing and identified labels. Overall, programmes did not give much priority to the European logos. Logos were rarely recognised on the Japanese market, and were seldom put forward by the programmes. The logos were often misunderstood or unknown to both professionals and endconsumers. There was a certain lack of adaptation of the November 2011 Page 78 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Product/ Region Logos promoted Russia PDO, PGI & organic products Switzerland and Norway PDO & organic products PDO, PGI, TSG & organic products India, China and SouthEast Asia Results programmes to the culture: logos were not translated although the public would be sensitive to such adaptations. The very low level of development of the organic sector in the country hampered the success of the organic promotion programme. The general awareness on logos is very low in Russia. Furthermore, none of the European logos were translated and virtually no one could read or understand them. Even people who had good knowledge of logos found them unclear. There could be potential for logos to influence consumers if they were adapted to the culture. Programmes promoting EU logos programmes had failed to develop either a high quality image of EU goods in Switzerland and Norway, or knowledge of European terms and related labels. In China, a case of synergy was highlighted due to the fact that the EAT campaign had already done promotion on the EU “quality” logos. Many other logos exist on the Chinese market, although the one most unanimously recognised is the ‘QS’ logo certifying product safety. In India and South-East Asian countries, knowledge of EU logos was very low although programmes did promote them. Source: Nine past evaluation reports 3.5.3 EU Image This section concerns the contribution of the programmes on the improvement of the EU image linked to agricultural products worldwide and the development of new markets. The internal market programmes had little impact on promoting the EU image: the image of milk is already high, but is not associated with the EU; the image of fruit and vegetables varies with different groups, but there is no clear EU identity in the image; the wine programmes focused on wines produced by specific regions and therefore had no impact on the EU image. However, programmes to support organic products have had more success in promoting an EU image associated with quality food (Table 24). Final Report November 2011 Page 79 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 24: Sector Contribution of the programmes on the improvement of the image of EU agricultural products on the internal market Comments Due to the diversity of fruit and vegetables in the EU, there was no common EU image of fruit and vegetables among consumers, but a combination of national or regional images at least in the EU-15. However, Fresh and in terms of image, there were indications from closed programmes and processed fruit surveys that I&P campaigns impacted positively on the image of fresh fruit and vegetables and vegetables among young consumers, as well as among teachers and doctors as primary TGs. In fact, few of the sample programmes were closed at the time of the evaluation. Also, clear baselines were rarely provided on the image of fruit and vegetables among a specific TG As a consequence of the campaigns, the image of the (promoted) products had improved among opinion leaders and the general public had been made familiar with a wider range of products than before. Since the majority of Wine the programmes promoted national products, it was unclear whether they have contributed to improving the overall image of European wines or quality schemes. The milk campaigns had a relatively strong impact on awareness and had satisfactory recognition rates (between 31% and 36%) with one exception. Milk For the people who saw the campaigns, the effects on their knowledge and perceptions of milk and milk products are noticeable in the sense that the messages have generally been well recognised. The results varied across the analysed countries. In France, the EU organic guarantee made a big difference (organic products certified outside the EU Organic products were much less trusted than those certified inside the EU), whereas the perception difference was much smaller in Italy Source: ADE on the basis of the EU co-financed information and promotion actions between 2002 and 2008 evaluation reports For Third Countries, the impact of I&P programmes on the EU image was mixed. In India, China and South-East Asia, USA and Canada as well as Russia, the EU image among professionals is already high and there was some reinforcement of this image. In Switzerland and Norway, the EU image seemed less highly valued, perhaps because local food products compete for the quality image on at least equal terms (Table 25). There was little evidence from the evaluations that new markets were developed. Some of the third country evaluations (India, China and South-East Asia, Switzerland and Norway, USA and Canada) stressed the importance of new business relationships as one of the most important long term impacts. Two evaluations (India, China and South-East and Russia) mentioned the positive impact of the programmes on professional and trade associations. Final Report November 2011 Page 80 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 25: Contribution of the programmes on the improvement of the image of EU agricultural products worldwide and on the development of new markets Third country market Comments There was a strong impact of the programmes on awareness amongst India, China and professionals. Although most professionals still identified primarily with South-East Asia the gourmet reputation of France, Italy and Span, the EU identity. For wines, the country origin was considered to be more important than the European one which didn’t act as a selling argument. The image of European products was not put forward, even in multi-country Japan programmes. “The tight focus on the product and the diversity of promoted products make it unlikely to attain any “critical mass” enhancing the European image in Japan”. Given that only 18% of the programmes were multi-country, the national characteristics were generally much more important than EU ones. The programmes generally focused on the quality of the product itself or its local origin, rather than the EU logo or characteristic. Promoting the EU USA and Canada was even considered counter-productive for products famous for their local or national characteristics (e.g. Champagne, Parmigiano Reggiano). For newcomers on the North-American market however, it is considered as useful to anchor on the EU image and logo. The programmes had not succeeded in promoting the EU identity. Russia National identity remains the main sales argument for South-Western EU MS. The promotion programmes contributed towards improving the image of quality of the promoted products when the campaigns were concentrated on the product itself. Campaigns which focused on the European identity had really poor impacts in terms of recognition of the product and improvement of the image. In conclusion, for the most part, programmes had failed to develop either a high quality image of EU goods in Switzerland and Norway, or knowledge of European terms and related labels. Nevertheless, the European image probably benefited from repeated messages about European products; however, this contribution Switzerland and would be perceptible in the long term only. Norway In the longer term, some programmes, well-designed and adapted to the market, contributed to the creation of new commercial outlets, mainly as a result of better information on the product for professionals, and effective targeting of professionals. Lastly, they enabled producers to compare their offer in a new market, which had sometimes led to an improvement of the quality of the products and an adaptation to the needs of consumers. Promotion programs targeting mainly consumers had no visible effect on creating new outlets but have more consolidated the existing outlets. Source: ADE on the basis of the framework contract evaluations Final Report November 2011 Page 81 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 3.6 Theme V – Complementarities between Member State programmes and those submitted by the professional organisations The first question in this theme assessed the extent to which the EU co-funded programmes developed synergies and complementarities with other programmes which were set up by the private sector or by Member States 48. The framework contract evaluations also asked whether the EC co-funded programmes had any multiplier effect in mobilising additional support from the private sector and Member States. The internal market evaluations had an additional question about the internal coherence of the programme. 41F Overall, the involvement of Member State governments in the programmes did not necessarily lead to the development of synergies with other measures led by the private sector or Member States (score: 2.4) A least where they quite difficult to identify. The evaluations found only a few cases of synergies. In some cases, synergies may have been coincidental. Two evaluations (USA and Canada and organic products) recommended that the EU provide an effective strategy to develop synergies. However, messages were mixed on whether or not resources should be allocated to develop such incentives (score: 3.5). 3.6.1 Involvement of the private sector 49 and the ‘multiplier effect’ 50 42F 43F The evaluations attempt to discover whether the EU co-funding encouraged other sources of funding to contribute more than they would otherwise have done towards promotional activities, either in the EU co-funded programmes or in other programmes. Overall, the private sector showed limited participation in the EU co-funded promotion campaigns. As mentioned in the India, China and South-East Asia report, it was difficult for proposing organisations to raise funds from the private sector. Most interest came from POs, typically trade organisations, some of them funded by levies and there was no evidence of competition amongst them. Although the EU co-funding acted as a catalyst for promotion activities, the effect on the private sector itself was difficult to identify. There was some evidence however of cooperation with certain supermarket chains in promotion activities. The fact that programmes could not focus on brands was invoked by some evaluations as a reason for the weak involvement of the private sector. This did however not prevent these programmes from benefiting the private sector nor did it prevent the promotion of high value products. 48 49 50 This question is related to EQ10 and EQ11 in the 2011 evaluation, which deals with the complementarities and added value of EU support, compared to MS and private sector activity. « How well the programmes complement those initiated by Synergy with MS/private sector?» EQ 5.1 of the Framework Contract Evaluation. « Are the programmes expected to have a multiplier effect ? » EQ 5.2 of the Framework Contract Evaluation. Final Report November 2011 Page 82 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The products promoted in co-funded campaigns were generally of high added value (PDO, PGI and TSG products such as cheeses, wines and meat; organic products, etc.). Large proposing organisations were usually already involved in the promotion of their products. In the case of smaller organisations, the EU measures clearly acted as a lever to encourage promotion. There is little evidence that EC funded I&P programmes led to new activity by Member States or by the private sector. The table below gathers information provided by the reports on the development of synergies and complementarities between EU co-funded programmes and private initiatives. Table 26: Assessment of the reasons for which synergies and complementaries were (or were not) developed with the private sector Product/ Region Reasons for difficulties in developing synergies Organic products The EU and national or private organisations presented different strategies. Wine Were never foreseen by the programmes and were considered to be difficult. Fruit and Vegetables Programme implementation realities prevent the development of synergies with the private sector: difficulties for the private sector to re-use materials (logos, slogans, messages) developed by co-funded campaigns and preventing the use of trademarks. Final Report Reasons for complementarities and synergies Other organisations benefitted from the increased awareness of the EU organic logo to benefit their own products. In Austria, one PO specifically developed an EU co-funded programme as a complement to a national initiative. Many proposing organisations already carry out their own generic promotion campaigns (wine promotion is generally not based on brands but on origin) and usually consider the EU co-funded campaign as part of their global strategy. Stakeholders consider generic campaigns complement and reinforce campaigns carried out by the private sector, the regions and MS. There are synergies with national campaigns about a healthy diet. Synergies exist with private sector activities in terms of goals (healthy food and diet), messages conveyed (5 November 2011 Conclusion Complementarities and synergies were generally unplanned or indirect, except in the case of the Austrian programme. Complementarities and synergies during the programmes were generally accidental or indirect. Global synergies (on the long term) were rather foreseen by the POs than by the EU. Concrete examples of synergies were identified but developing synergies with the private sector are hampered under the current regulation (e.g. re-use of logos and slogans and trademarks) Page 83 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Product/ Region Milk USACanada Reasons for difficulties in developing synergies Nothing is done by the private sector or MSs for generic milk product promotion. Lack of incentives. Restriction on brand promotion Preventing brand promotion seems to have discouraged the private sector Reasons for complementarities and synergies a day) and target groups (young people, schools) and channels. The absence of other stakeholders in such promotion ensures that there was no overlap. Messages regarding health benefited the sector in general. Nothing was foreseen to develop synergies Switzerland and Norway India, China and South-East Asia Mixed results. There were cases of complementarities but they were limited in many cases. No complementarities or synergies identified. Japan Russia Conclusion No redundancies or contradictions among programmes No deliberate search for complementarities or synergies No cases of joint promotion and no synergies identified. Cases of synergies were unplanned Few complementarities were identified Other initiatives have addressed similar themes or products but there is no evidence on whether these initiatives had a complementary or overlapping effect. Source: ADE based on framework contract evaluations 3.6.2 Coordination with national initiatives The development of synergies and complementarities with MS initiatives depended mostly on national contexts and policies. On the internal market, some sector particularities led to stronger interactions (e.g. for Fruit and Vegetables, where major organisations all agreed on the need to increase Fruit and Vegetables consumption for matters of public health). Sector-specific information on how national initiatives and co-funded programmes were articulated on the internal market is detailed below: Milk - Almost no information about other national initiatives in the sector: Most of the co-funded programmes focused on the country of the PO and were well adapted to the economic and policy environment in the country. A few programmes promoted PDO products in other countries and for export. There was little synergy with other programmes and there were few incentives for seeking this synergy. In Germany and Belgium the proposing body was common for all I&P programmes (with or without EC funding) and provided centralised tendering for greater efficiency, but did not provide functional synergies. Final Report November 2011 Page 84 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Fruit and vegetables - Mainly in line with national initiatives about healthy diet. In many countries, campaigns were conducted in schools to promote healthy eating. These campaigns embraced fruit and vegetables by promoting healthy diets and physical activities, and were mostly co-financed by Public Health organisations (ministries, civil society and sometimes private organisations). Some stakeholders thought that activities or programmes targeting schools or truly generic messages promoting healthy diets should be completely financed from public funds. Thus, the few potential synergies mentioned in the programming documents referred mainly to public health initiatives concerning a healthy diet. Despite the rather low awareness of synergies in programming documents, most stakeholders were able to identify potential synergies between co-financed programmes and MS initiatives and/or private initiatives. Concrete examples of such synergies were found in terms of goals (healthy food, food quality etc.) and messages conveyed (e.g. health message “5 al día”), but also in terms of target group (e.g. schools) and channels (e.g. common web-portal). Some problems were reported with respect to exploitation of synergies between co-financed programmes and national branch organisations (e.g. reinforcement of synergies between the PO and implementing body during programme implementation; developing synergies with other countries etc.). Wine - Most stakeholders consider co-financed generic campaigns as complementary to the traditional brand-oriented campaigns. This was reinforced by the fact that the immediate effects pursued by brand-oriented promotion were complemented by the long-term impact of generic promotional campaigns. However, there was little evidence that those synergies between EU co-financed programmes and brand-oriented campaigns were intended rather than merely accidental. No overlapping or duplication was identified between the EU co-financed programmes and other campaigns. In third countries, the results were less obvious or satisfactory. No cases of coordination were identified even though many national or regional institutions implement programmes in the countries analysed. On the other hand, few serious problems were pointed out by the evaluations. USA-Canada – Regional Chambers of Commerce are also engaged in promotion activities in many cases. Their actions and events were not coordinated with the EU co-funded ones. In some cases, the EU funding was seen as a threat by proposing organisations, as it induced a “competition” effect between national and EU programmes: potential national funds were being reduced and companies gave preference to EU programmes, reducing the visibility of national ones. This was the case at the New York Fancy Food Show, where companies chose to join the European Gourmet Project rather than the French Pavilion where they used to be. The EU co-funded programmes focused on specific messages which national, regional or private campaigns did not cover. The themes and products promoted differed as Final Report November 2011 Page 85 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC well. Although there was no overlap with private programmes, no mutual reinforcements were observed either. In the other evaluations in Japan, Switzerland and Norway, no or very few complementarities could be identified. In the expanding context of the Russian import market, it is considered difficult for promotion programmes to be “in competition”. Therefore, although no cases of complementarities or synergies were identified, no overlaps or contradictions appeared either. 3.6.3 Synergies among EU initiatives The other main EU promotion initiative carried out at the time of the framework contract evaluations on third country markets was the European Authentic Tastes (EAT) campaign. According to the reports that referred to it (the USA-Canada and the China-India-SouthEast Asia reports), results were mixed. In the USA, stakeholders were very critical of the lack of coordination between initiatives. At the New York Fancy Food Show, both the European Gourmet Project (EGP, a multicountry and multi-product co-funded programme involving Spain, France, Italy and Portugal) and the European Authentic Tastes programme (EAT, a programme implemented at the initiative of the EC on the European quality system) were present, in separate booths. This created a sense of confusion for attendees and prevented them from exploiting potential synergetic effects. According to the China-India-South-East Asia report, there was the case in China where the I&P programme was implemented after the European Authentic Tastes (EAT) programme. Stakeholders in China reported that the messages had reinforced each other in an effective way. Several PDO-PGI campaigns were also active in China and the reinforcing nature of the various programmes was successful. The focus on a few cities gives good potential for reinforcement of messages. However it was stated that, in general, cooperation amongst EU programmes at international market fairs offers scope and has not been sufficiently exploited. 3.7 Horizontal conclusions and recommendations This section presents the conclusions and recommendations reached by the nine evaluations. The section does not aim to reach independent conclusions and recommendations. This is done at the end of the study, taking into account all sources of evidence, including the framework contract evaluations. Final Report November 2011 Page 86 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Promotion programmes coherent with the objectives of the Regulation Most programmes are consistent with the Regulation objectives, but need more prioritisation and strategy Six of the nine evaluations (milk, fruit and vegetables, organic, wine, Japan, Russia) concluded that the programmes followed objectives that were in line with those in the EC Regulation. Three of the evaluations (fruit and vegetables, organic) concluded that the there was some lack of clarity in the objectives and two evaluations (fruit and vegetables, wine) recommended that objectives should be quantified. The Asia evaluation concluded that the objectives of the programmes are not appropriate because they aimed to have an effect on sales, which is unrealistic. Five of the evaluations (fruit and vegetables, organic, wine, USA and Canada, Japan) concluded that there was limited prioritisation or ordering of objectives and no distinction between specific and global objectives. Four of the evaluations (fruit and vegetables, organic, wine, Japan) recommended a better link between objectives and actions. The USA and Canada evaluation recommended that applications should not aim to match Two of the programmes (Switzerland and Norway, USA and Canada) concluded that the global strategy of the I&P programmes was ambiguous and should be made clearer. The Switzerland and Norway evaluation recommended distinguishing between programmes that support large market with generic promotion and those that support a collective high quality brand, often supplied by a small number of producers in a new market. Objectives are sometimes well based on market analysis and sometimes not Most evaluations drew conclusions on the relationship between objectives and market analysis: the fruit and vegetables evaluation was largely positive; the organic evaluation called for more consistency; the milk evaluation recommended that applications should specify the market failures that they addressed; and the Japan and Russia evaluations recommended more attention be paid to tailoring actions to local market conditions. The China, India and South-East Asia evaluation mentions that the majority of sampled POs undertook analyses (especially newcomers), which showed an improvement compared to the situation in previous evaluations. Globally guidelines – Annex II of the Commission Regulation (EC) No 1071/2005 – for design were adequate and well applied According to four framework contract evaluations (milk, fruit and vegetables, organic, and wine), guidelines for designing co-funded information and promotion programmes are satisfactory and well-applied by stakeholders. They provided all the main information needed to prepare a programme proposal while leaving room for adapting information and promotion actions to the specificities of the targeted markets. Nevertheless some comments to fine-tune guidelines were made. Two evaluations (milk, fruit and vegetables) recommended that the guidelines should include a requirement to specify the intervention Final Report November 2011 Page 87 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC logic in proposals. The milk evaluation recommended that the guidelines should require an explicit exit strategy with a view to sustaining obtained results and impacts after the end of the programme. Claiming health benefits is problematic One of the evaluations (organic) concluded that no programmes claimed health benefits, because of the difficulty of obtaining scientific evidence of the robustness required in the regulation. The milk evaluation found that the emphasis of the programmes was on nutrition and health, although the evidence in the body of the evaluation suggested that no programmes formally claimed health benefits. EU added value Several evaluations found a clear effect on the scale of promotion activities, mostly because the EU co-financing allowed larger programmes with more ambitious activities covering a wider geographical area and including, notably, TV. The other effects of the EU co-funding were the multiplier effects (e.g. implementation of other campaigns) and the possibility to set-up a programme for countries without national funding for promotion or even indirect effect on grouping of professionals of concerned sectors. Finally, multicountry and multi-product programmes were clearly implemented to respond to the specific Commission interest. The framework contract evaluations were ambivalent about multi-country and multi-product programmes Only three of the evaluations (milk, fruit and vegetables, Asia) drew conclusions on the success of multi-country programmes (26 on the all 212 implemented programmes were multi-programmes 51), and all stressed the challenges facing multi-country programmes resulting from additional costs, cultural differences and differences in suitability of messages. However, the fruit and vegetables evaluation commented that the European dimension of the processing industry has begun to facilitate a multi-country approach. Two evaluations (milk, fruit and vegetables) recommend caution in favouring multi-country programmes. 4F The Switzerland and Norway evaluation concluded that most of the programmes only had a limited EU dimension and barely justifying EU co-funding programmes or just enough to offset the absence of national financial support. The evaluations rarely draw conclusions on multi-product programmes. The USA and Canada programme concluded that synergy amongst products was not always sought, but recommended that it should be. The Russia evaluation also recommended more attention be given to multi-product programmes. 51 These figures come from the framework contract evaluations. There might be small differences with counting of programmes under EQ 4 that rests on more recent data from DG Agri. Final Report November 2011 Page 88 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Design and selection Stakeholders are unclear about how programmes are selected Four of the framework contract evaluations (milk, fruit and vegetables, organic, and wine) found that there was uncertainty about how the Member States and European Commission select programmes, despite the feedback provided by the Commission. The milk evaluation made suggestions for introducing a two stage scoring system. The USA and Canada framework contract evaluations concluded that the approval process was not efficient or carried out timely. The Asia evaluation recommended that applications be accepted throughout the year, especially as some producers face seasons when they are exceptionally busy. Two of the evaluations (organic, wine) found that there was some confusion over the role of Member States and the European Commission in selecting programmes. The milk evaluation found that most Member States were not fulfilling their obligations to exercise some pre-selection activity and suggested that Member States might be given an allocation for the number, or total value, of the applications they can submit. Management, monitoring and evaluation The overall management set-up and administration of the programmes is adequate According to four of the framework contract evaluations (milk, fruit and vegetables, organic, and wine), the overall management set-up was globally appropriate to manage co-funded information and promotion programmes, with the exception of the selection procedures (as explained above). However, some evaluations underlined few management issues. For example, the US & Canada evaluation found that administrative arrangements were considered a burden and that some proposing organisations had resigned as a result. Two of the framework contract evaluations (milk, and wine) found that the guidelines for implementation were less useful than for design. The milk evaluation found that Member State competent bodies were too often slow in responding to requests for amendment to programmes. In addition, the US & Canada evaluation found that there was insufficient contact from the European Commission, after selection. Arrangements for monitoring and evaluation are not effective, but there are mixed views on improvement All four framework contract evaluations carried for the internal market drew conclusions about monitoring. Two of them reported that the monitoring guidelines were followed in an adequate manner (organic, and wine). The milk evaluation called for more guidance to require monitoring reports to provide evidence on efficiency. The milk evaluation also recommended that the European Commission and Member States should provide more feedback on quarterly monitoring reports to proposing organisations. The Asia evaluation recommended more attention be given to learning lessons and making rapid adjustments to Final Report November 2011 Page 89 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC activities, as well as quarterly reports should be limited to one page documents and that monitoring reports should be submitted annually. Three framework contract evaluations (fruit and vegetables, organic, and wine) drew conclusions on the quality of evaluation, all of which concluded that the analysis was too superficial. The fruit and vegetables evaluation recommended that evaluation reports should be required to report on the intervention logic and on complementary activities funded by the European Commission, Member States or the private sector. The organic evaluation recommended that a budget should be assigned for ex post evaluation of all programmes. The Asia evaluation recommended that evaluation should be limited to a single report by independent evaluators in the final year. Efficiency of Measures Efficiency is rarely measured, but, when evidence is available, efficiency is normally high The evaluations on the internal market (milk, fruit and vegetables, organic, wine) drew conclusions about the efficiency of measures, all concluding that efficiency was good, but that it was not properly monitored. Two evaluations (milk, wine) recommended that more attention is given to efficiency in proposals, but the milk evaluation found that it is not possible to make projections for efficiency several years in advance and, thus, that work programmes for the second and third years should be considered as indicative. None of the third country evaluations drew conclusions about efficiency. Different types of actions and information channels are complementary and there are few general rules about which is best Four evaluations (milk, fruit and vegetables, organic, wine) found that it was not useful to draw conclusions about which measures are most efficient, because most measures are undertaken as part of a range of complementary and interdependent measures. This conclusion is supported by the wide range of conclusions on the most efficient measures: the wine and Switzerland and Norway evaluations concluded that direct contact with consumers was most effective; the fruit and vegetables programme concluded that TV and press campaigns gave low cost per contact but that they may still have had a role in a broader campaign; the USA and Canada evaluation found that press and public relations actions were the most effective and on-line actions were the least effective; the Russia evaluation found that ads in magazines were most likely to be effective; and the Asia evaluation found that public relations and fairs were cost effective. The efficiency of programmes based on the EU logo was very mixed, with the Japan evaluation concluding that the logo has potential as a marketing tool (although programmes had provided few results in terms of logo recognition, the use of logos in the country is making progress). The Russia evaluation concluded that the logo is ineffective because not adapted to Russian consumers, but still recommended that the EC support a major programme to make Russia more receptive to logos. The USA and Canada and Japan evaluations found that the EU logo can be useful in giving access for EU-10 to the EU reputation for quality. Final Report November 2011 Page 90 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Working with professional agents in third countries is usually effective and efficient Four of the five third country evaluations (USA and Canada, Japan, Russia, Asia) found that working with experienced regional and local professional agents is usually the most effective way of managing programmes. The Switzerland and Norway evaluation recommended working with professional agents where new products and/or new market are involved, while the Japan and Asia evaluations found that direct contacts with customers could also become important, when working with well-established names and markets. The Asia evaluation found that programmes were not sufficiently businessoriented to suit the expectations of local agents. Targeting, Size and Duration The definition of target groups is in line with the Regulation, but could be improved in some cases Almost all the evaluations drew conclusions about the level of targeting, with three (organic, Russia, India, China and South-East Asia) being positive and the rest (milk, fruit and vegetables, wine, USA and Canada) suggesting that improvements in targeting were possible: in most cases this involved clearer targeting, although the fruit and vegetables evaluation argued for a broadening of the target groups. Five evaluations recommended more targeting, including four third country evaluations. Most of the evaluation concluded that measures were well adapted to the target group. The size of programmes is appropriate on internal market, while insufficient for third country market programmes Three of the evaluations (organic, wine) reached positive conclusions that the size of the programmes was sufficient to achieve objectives of increasing awareness, bearing in mind the size of the target group. However, all three also concluded that the size was insufficient to have an impact on consumption. The milk evaluation found examples of countries in which programmes had become too fragmented. Although only one evaluation of third countries drew conclusions about the size of the programme, there is evidence in the body of all the third country evaluations except Switzerland & Norway that the budget was insufficient to achieve the objectives, even after providing a geographical focus on selected cities. The duration of 3-year programmes is appropriate for the IM less for third country markets, but more attention is needed on what happens after the programme Five of the evaluations drew conclusions on whether the duration of the programmes was sufficient to have an impact on awareness, with three positive (milk, organic, wine) and two negative (USA and Canada). Six evaluations (milk, organic, wine, Japan, Russia, Asia) concluded that it was necessary for three year programmes to be part of a longer term set of programmes, but supported three year programmes, with the potential to have a second phase. The wine evaluation recommended giving preference to second phase applications, to facilitate continuity and longer term impact. The Asia evaluation recommended that Final Report November 2011 Page 91 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC approval of second phase programmes should be dependent on an independent evaluation at the end of the first phase, and that second and third phases should receive a declining EC share of match funding. Two of the evaluations (fruit and vegetables as well as organic) concluded that one year programmes are only suitable for pilot activities. Impact The impact on the EU logo has been limited and recommendations are mixed Regarding the EU logo, two evaluations on the internal market (organic, wine) concluded that impact on awareness of the EU logo is limited, although the organic evaluation concluded that some progress is being made, especially where the EU logo built on the reputation of national organic logos. All third country evaluations drew conclusions on the impact on the EU logo; with most concluding that the impact was limited and the Russia evaluation concluding that the situation is complicated by distrust of the quality of products from Eastern European Member States. Many of the third country evaluations recommend giving higher priority to promoting the EU image, and this was especially true for Japan and India, China and South-East Asia. The impact on awareness and the image of EU products is positive Three of the evaluations (milk, fruit and vegetables, Switzerland and Norway) reported that the programmes had a good impact on people’s awareness about the promotion messages and the image of the product. The milk evaluation concluded that this awareness has tended to decline quite rapidly after the end of the programmes and thus recommends sustained activity. The milk evaluation found that impact on primary contacts (e.g. nurses and teachers) was generally very high. The Japan evaluation recommended that improving awareness should be the indicator of effectiveness and that no attempt should be made to measure impact on consumption. The Asia evaluation was positive about the impact of the programmes on the EU image. The impact on consumption is unclear Five of the evaluations (milk, fruit and vegetables, Switzerland and Norway, USA and Canada, Japan) concluded explicitly that it was difficult to establish a causal link between the programmes and statistics about consumption. However, there were a significant number of examples where, in the body of the reports, impact on consumption could be observed for programmes with small target groups (fruit and vegetables, organic, wine). Furthermore, in the body of the reports, some evaluations claimed that it was legitimate to assume that a change in awareness will lead to changes in consumption, although the milk evaluation concluded that consumers who claim that they will change consumption behaviour did not always do so. Two evaluations (milk, fruit and vegetables) stressed the importance of the quality of contact as an indicator of whether changing awareness will be converted into changing consumption habits. Final Report November 2011 Page 92 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Many programmes have a strong positive impact on collaborative promotional institutions Five evaluations (organic, wine, Switzerland and Norway, USA and Canada, Asia) found strong positive impact on the institutions for collaborative promotion. In third countries, the reports suggest that programmes were successful in creating and/or strengthening business relationships which were considered to be valuable, although the reports themselves do not draw this conclusion. The Switzerland and Norway and Asia evaluations found that the I&P programmes gave some smaller producers their first opportunity to access international markets and the USA and Canada and Asia evaluations recommended that priority and special assistance should be given to newcomers. In contrast, the Japan evaluation concluded that the I&P programme was only effective in safeguarding existing market shares. The Asia evaluation recommended that building the capacity of producers to market in Asia should be the primary objective of the programme, in place of the objective of increasing sales. The EU funding leveraged some additional funding, but the picture is quite complex Six of the evaluations (milk, fruit and vegetables, organic, wine, USA and Canada, Asia) concluded that I&P programmes leveraged substantial additional funding. The conclusions are less clear on whether the EC funding stimulated additional MS funding or partially replaced MS funding. In the body of the reports, there is evidence that, in many cases (fruit and vegetables, Russia), I&P programmes would not have taken place without EC funding and the EC funding was therefore successful in leveraging additional financing. In those countries where I&P activities would have taken place, the EC funding was considered as additional funding that allowed larger programmes to take place and also encouraged a structured approach to generic funding that would not otherwise have happened. Complementarity The synergy with other programmes was very limited, but there is little redundancy and incoherence Most programmes reported limited synergies with other measures, whether funded by the EC or by MSs or the private sector. Four evaluations (organic, wine, Japan, Russia) reported that, on the few occasions when with other EC programmes existed, they were coincidental, rather than planned. Most evaluations reported that there was no redundancy or incoherence between programmes, although the Japan and Russia evaluations concluded that there is a risk of competition in some programmes, notably for wine and at fairs. The fruit and vegetables programme reported some complementarity with publicly funded health promotion activities, but found no examples where this was planned. Six of the evaluation drew conclusions on the level of synergy with the private sector, of which four (fruit and vegetables, organic, wine) concluded that there was little planned synergy, largely because private sector interest is focused on brands, and two were mixed. Final Report November 2011 Page 93 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC However, the wine evaluation found that there was unintended complementarity with brand campaigns. In the reports on wine and fruit and vegetables, the main explanation of this was associated with the restrictions on brand identity and on intellectual property rights of produced material. The body of the reports suggested that private sector interest was greatest for high added value products, such as cheeses, wines, meat and organic products, because these products are closest to having a brand identity. The milk evaluation recommends clearer guidelines to allow the private sector to use promotion material produced in the I&P programmes. Five evaluations recommended activities to improve exchange of experience and encourage synergy: the milk and fruit and vegetables evaluations recommended that fact sheets should be produced on all programmes; the organic evaluation recommended more exchange between programmes; the wine evaluation suggested the more meetings and a website would help with exchange of experience; and the Asia evaluation recommended info days, meetings and an internet instrument. 3.8 Good practices Good practices applicable to both the internal market and third countries This section of the report presents the evidence of the framework contract evaluations. None of the nine evaluations explicitly identify best practices. Therefore, the following best practices are selected as the strongest recommendations of the evaluations themselves, without adding additional interpretation. These best practices concern the MS level, not the EC. Design of I&P programmes 52 There are no given recipes of actions to undertake. Programmes should consider a mix of actions; Include a market situation description justifying the reasons of the proposal; Define quantitative (SMART 52) objectives; Describe the appropriate strategy; Define the target population(s); Select the appropriate communication channels and messages in accordance with the market description; Carefully describe the activities as well as the correspondent budget; Define clear efficiency criteria to select the most appropriate communication channels; Take into account the fact that most expensive channels may produce better quality contacts Define intended results and impact Define appropriate monitoring and evaluation tools to assess these results and impacts Prepare exit strategies to be implemented at the end of the programme 45F SMART objectives : Specific, Measurable, Attainable, Relevant and Time-bound Final Report November 2011 Page 94 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Implementation of I&P programmes Reinforce the implementation of information and promotion programmes through: - A competitive tendering procedure for selecting implementing agencies - The use of a unique central purchasing agency allowing discount on the purchasing of media spaces Use results of the monitoring and the evaluation of the I&P measures to fine-tune them with a view to improving their impacts Good practices specific to Third country markets The following good practices are to be considered in addition to the ones mentioned above in the case of third country markets. Design of I&P programmes Work directly with regional and local partners (implementing agencies) to design information and promotion campaigns; Focus mainly on professionals with direct contacts through fairs, exhibition, tasting and other public relation activities; it is very important to select the right sites where appropriate professionals are present and to look for complementarities/ synergies with other European products; Eventually complement promotion activities to professionals with promotion activities to consumers when promoted products are known on the targeted market; Implementation of I&P programmes Work directly with regional and local partners (implementing agencies) to implement information and promotion campaigns Final Report November 2011 Page 95 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 4 Replies to the evaluation questions 4.1 Intervention logic Figure 13 represents the intervention logic of promotion and information actions for agricultural products. The architecture of the intervention logic is formed by Council Regulation 3/2008 and Commission Regulation 501/2008. It covers the information and promotion policy for agricultural products and their method of production as well as for food products based on agricultural products. It includes the internal market and third countries. Global objective of Council Regulation (EC) N°3/2008 The perceived global objective of the Community support – which is not explicitly specified in the regulation – is to support consumption and demand of EU agricultural products, by this also supporting producers’ income. This global objective is consistent with the objectives of the CAP, specified under article 33 of the Treaty of the EU, and supporting measures to help attain these objectives. Article 35 especially mentions joint measures to promote the consumption of certain agricultural products. The support to consumption is being pursued to strengthen demand and so bring improved sales and income to producers. Specific objectives of Council Regulation (EC) N°3/2008 The specific objectives identified in the introductory considerations and art. 2 of Regulation 3/2008 and in the annexes of Regulation 501/2008 concerning each product were gathered as follows: to increase the knowledge about EU agricultural products ; to boost/enhance the image of EU agricultural products, both in the eyes of consumers in the EU and in third countries; and to open up new markets in third countries. Indeed, as mentioned under recital 4, the policy is articulated around the specific quality, nutritional value and safety of products and their methods of production. It seeks to supplement and reinforce activities developed by Member States, and should have a multiplier effect on national and private initiatives. The operational objectives, specified in Article 2 of Regulation 3/2008 linked to these three specific objectives are the following: Final Report November 2011 Page 97 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC On the internal market and in third countries: - To inform about and promote the quality, the nutritional value and the safety of EU agricultural products and foodstuffs; - To inform about the EU systems of PDOs, PGIs, TSGs, organic farming and other EU schemes for quality standards and labelling of agricultural products and foodstuffs; - To inform about the EU system covering wines with a PDO or geographical indication, wines with an indication of wine grape variety and spirit drinks with geographical indication; - To identify needs in terms of promotion of EU agricultural products In addition, within third countries: - To inform about the EU system for table wines - To identify new markets for EU agricultural products. The inputs used to reach these objectives (based on Regulations 3/2008 and 501/2008) are the co-funded I&P programmes and high-level trade visits in third countries. Studies of new markets are also possible for third countries. The various eligible I&P measures defined in art. 2, shall be part of an I&P programme. Most programmes are co-funded by the EU, proposing organisations and the Member States, however there might be programmes that provide from the Initiative of the Commission only (see art.10). I&P programmes are made up of a series of measures with determined messages to be used and channels through which the messages are transmitted. These include notably public relations contacts with the media and advertising (through electronic channels, visual media (cinema, TV), radio spots, leaflets and brochures); point-of-scale actions to consumers; contacts with specialists (nutritionists, doctors, etc.), with schools, with retailers and businesses etc. They may also concern participation in events, fairs and exhibitions of national/EU/international importance both on the internal market or in third countries. According to annex I of Regulation 501/2008, the messages may include themes, in agreement with the operational objectives: Intrinsic features and advantages of EU products, such as quality and safety of food, specific production methods, nutritional and health aspects, labelling, animal welfare and respect for the environment; Specific characteristics associated with PDOs (particular geographical environment), PGIs (quality/reputation linked to geographical origin at the stages of production, processing or preparation), TSGs (particular traditional methods), information on logos and information on other quality aspects (safety, nutritional value, taste and traceability) and Community legislation strictly regulates production, quality indications, labelling and marketing; Very wide selection of EU wines of different origins, EU wine cultivation and links with regional/local conditions, customs and tastes. Final Report November 2011 Page 98 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Other inputs contributing to reach the objective are: High level trade visits in third countries that result from the initiative of the Commission; Studies of new markets also regarding third countries. The evaluation of I&P programmes is also foreseen among the inputs under the item of “impact assessments”. As introduced in chapter 1.2, information and promotion of agricultural products is not only financed by Council Regulation (EC) No 3/2008 through EAGF but also by other CAP measures, both under the first or the second pillar. These cover mainly the wine sector (reformed in 2008) and fruit and vegetables sector (reformed in 2007). Council Regulation (EC) No 73/2009 regarding direct support to farmers includes under its article 68 the possibility of specific support measures that could include promotion, but it is almost not used and thus not included here. Council Regulation (EC) No 814/2000 covers information measures on the CAP (see 1.2.5). These are of a much wider scope than the promotion policy, but there are some very specific activities targeted towards schoolchildren and healthy eating that may be considered complementary. In addition, the second pillar through the Council Regulation (EC) No 1698/2005 included since 2007 the possibility to promote regional quality products within the rural development programmes. The specific promotion measure aims to inform consumers’ about the existence and specifications of quality schemes and to raise consumers’ awareness about the availability of these products. Rural development policy, the second pillar of the CAP, includes a possibility to support promotion since 2003. Article 20 and 33 of Council Regulation (EC) No 1698/2005 on support for rural development by the European Agricultural Fund for Rural Development (EAFRD) provide for support to producer groups for information and promotion activities concerning products covered by food quality schemes. The promotion measure aims to inform consumers about the existence and specifications of quality schemes and to raise consumers’ awareness about the availability of quality agricultural products. Objectives regarding the fruit and vegetables sector are also concomitants to those of the Council regulation. Its approach to promotion, quality and marketing standards aims to encourage fruit and vegetable consumption. In the wine sector, information and promotion measures about EU wines in third countries pursue the broad aim to improve competitiveness of EU wines in those countries, through increased outlets. Final Report November 2011 Page 99 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Figure 13: Final Report Reconstructed intervention logic of promotion and information measures for agricultural products November 2011 Page 100 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The Regulations governing the I&P programme state that the objective is to support consumption. Despite this, many of the 2008 evaluations commented on the lack of purpose for the programme and this conclusion also dominates the Court of Auditors 2009 report. However, it remains unclear what are the intended benefits from increased consumption and who are the intended beneficiaries. Indeed, from an economic perspective, benefits arise in three ways: a) increases in consumer surplus (defined as the extent to which some consumers pay a price that is lower than the maximum price they would be prepared to pay); b) increases in producer surplus (defined as the extent to which some producers sell for a price that is higher than the minimum price they would be prepared to accept – a concept that is roughly associated with profitability); and c) ‘externalities’ associated with benefits that are not reflected in market prices (impact on health, environment). Reminder: the evaluation questions below are answered based on the following sources of information: statistical analysis (from DG AGRI data and online databases), documentary analysis (from EC Regulations, previous framework contract evaluations, programme proposals, monitoring and evaluation reports, and promotion material), an online questionnaire sent to competent bodies in Member States, and face-to-face interviews with EC representatives, competent bodies, proposing organisations and implementing bodies. Cases where the information is specifically linked to a given source are mentioned in the individual evaluation questions. 4.2 Replies to evaluation questions of theme 1 – Policy relevance and effectiveness This section presents replies to the evaluation questions (EQs) related to theme 1 Policy relevance and effectiveness. Each question is structured as follows: A summary of the rationale and approach to the question (including judgment criteria and indicators); Reply to the question based on judgement criteria developed in the data collection grid; An evaluative judgment box which highlights main elements of the reply to the evaluation question. The same structure is adopted for theme 2 and 3. Final Report November 2011 Page 101 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 4.2.1 Evaluation Question 1 To what extent has the list of themes and products, which may be covered by promotional measures, been relevant to achieving the objectives laid down in the Regulation? Rationale and approach This question aims at verifying the extent to which the list of themes and products is relevant to increase knowledge, to enhance the image of EU agricultural products and to open new markets (specific objectives) in order to increase the consumption of EU agricultural products (global objective). Overall this may be seen as an EU market development perspective. The scope of the analysis covers the list of themes and products, taking into account EU market trends and statistics from the FAOSTAT 53 and EU Export Helpdesk 54 databases, as well as document reviews and stakeholder interviews. The evaluation question is covered by six judgment criteria (Table 27). The two first judgment criteria provide qualitative elements from document reviews (J.1.1) as well as stakeholders’ interviews and the on-line survey (J.1.2). Judgment criteria J.1.3, J.1.4 and J.1.5 provide quantitative elements and appraise the relevance of the list from a statistical perspective. The last judgment criterion, based on export data from the FAOSTAT database focuses on the most important products for the EU in terms of export value (J.1.6). 47F 48F The main limitation of this approach is the data availability. Indeed, regarding the judgment criteria J.1.3 to J.1.6, five of the 17 themes and/or products (table 28) cannot be analysed by using FAOSTAT and EU Export Helpdesk databases because such data is not available. In this case, proxy indicators are used and the analysis is completed by qualitative elements. Unfortunately, it is not possible to estimate the degree of error of these proxies at an EU level; the situation being quite different amongst Member States. Moreover no data is available in the above-mentioned databases for EU quality schemes, organic farming products and graphic symbol for the outermost regions. No proxy could be used for these products, which are only covered by qualitative elements. This is a limitation as these themes concern a major focus of EU promotion policy. In addition, some specific data from FAOSTAT is missing for two products: the trade data for live plants and ornamental horticulture products as well as consumption data for fibre flax. However, the data from EU Export Helpdesk database is available, which enables a partial analysis. Regarding the geographical coverage, the data is analysed for the EU as a whole and may well hide important differences among Member States. 53 54 http://faostat.fao.org/ http://exporthelp.europa.eu/ Final Report November 2011 Page 102 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 27: EQ 1 Judgment criteria and indicators J.1.1 Documents regarding promotional measures validate the list of themes and products on the basis of identified market trends of agricultural products on both internal and thirdcountry markets Programming I.1.1.1 Documents validate the list for promotion on the internal market documents and I.1.1.2 Documents validate the list for promotion on third-country-markets evaluation reports J.1.2 Stakeholders validate the list of themes and products on the basis of identified market trends of agricultural products on both internal and third-country markets I.1.2.1 Stakeholders validate the list for promotion on the internal market Interviews, questionnaire I.1.2.2 Stakeholders validate the list for promotion on third-country-markets J.1.3 The EU market trends for the products covered by promotional measures over a recent period and its position into the worldwide market highlight important and strategic issues at stake for the EU I.1.3.1 Trends in (2000-2008) EU export values and EU export part (%) of world exports I.1.3.2 Trends in (2000-2008) EU import values and import part (%) of FAOSTAT world imports I.1.3.3 Trends in (2000-2008) EU exports and imports of the product as part (%) of EU agricultural commodities exports and import values J.1.4 The EU internal (intra-EU) market trends of the product covered by promotional measures over a recent period highlight important and strategic issues at stake for the EU national strategy I.1.4.1 Trends in (2002-2009) intra-EU export values and part (%) in the total EU Export Helpdesk EU export values I.1.4.2 Trends in (2000-2007) total and per capita EU consumption FAOSTAT J.1.5 The trends from EU to third-country market (extra-EU) of the product covered by promotional measures over a recent period highlight important and strategic issues at stake for the EU I.1.5.1 Trends in (2002-2009) extra-EU export values and part (%) in the total EU Export Helpdesk EU export values I.1.5.2 Trends in (2000-2007) total and per capita worldwide consumption FAOSTAT J.1.6 The list of themes and products which may be covered by promotional measures contains the most important products in terms of EU exportations values I.1.6.1 Products and groups of products classification according to the EU FAOSTAT export value in 2008 Table 28: Themes and products which cannot be analysed with the usual statistical sources and proxy indicators used Products / Themes Fresh, chilled or frozen meat, produced in accordance with a EU or a national quality scheme Marking of eggs for human consumption Wines with a PDO or a PGI, wines with an indication of the wine grape variety Quality poultry meat Spirit drinks with a geographical indication or a reserved traditional description Source: ADE Final Report Markets targeted Proxy products Internal Fresh, chilled or frozen meat Internal Internal and third country Third country Eggs Wine Third country Spirit drinks November 2011 Poultry meat Page 103 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Detailed answer J.1.1 Reviewed list inherited from the past more than the result of a genuine strategy The origin of the list of products and themes covered by promotional measures and its evolution since the mid nineties (1994) can be established on the basis of the Regulations reviewed. The promotion policy was initially covered by 12 Regulations during the 19942000 period and was defined by agricultural sectors of the internal market: citrus fruit, live plants and floricultural products, nuts, olive oil, table olives, grape juice, milk and milk products, flax, apples, quality bovine meat and PDO, PGI or TSG products. With the purpose to establish a coherent overall promotion policy that covers different topics and food products, these Regulations were grouped together at the beginning of the current decade into two Regulations, respectively for the internal and the third country markets. The sectors already covered in the nineties were included and sometimes enlarged. On the internal market, the list of sectors and products which may be covered by the actions was first defined in the Commission Regulation (EC) n°94/2002 by including the twelve previous sectors covered. The list was subject to revision every two years (art. 3). Further products have been integrated over time according to two different criteria: typical or quality agricultural products with high added value, such as PDOs, PGIs, TSGs, and the need to handle difficulties in individual sectors like in the milk or fruit and vegetables sectors or the avian influenza crises. Today, the list as defined in Annex I of Commission Regulation (EC) No 501/2008 includes 12 products and three themes (figure 14). Products/themes have been added (highlighted in grey in the table below) without removing any product from the list. Table 29: Evolution of the list of themes and products covered by promotional measures on the internal market since 1994 1994-2000 period Milk and milk products Citrus fruit Nuts Apples Grape juice Live plants and floricultural products Internal market Reg (EC) n°94/2002 Reg (EC) n°1071/2005 Milk products Milk and milk products Reg (EC) n°501/2008 Milk and milk products Fresh fruit and vegetables Fresh fruit and vegetables Fresh fruit and vegetables Processed fruit and vegetables Live plants and floricultural products Processed fruit and vegetables Live plants and products of ornamental horticulture Fresh, chilled or frozen meat, produced in accordance with a EU or a national quality scheme Fibre flax Processed fruit and vegetables Live plants and products of ornamental horticulture Fresh, chilled or frozen meat, produced in accordance with a EU or a national quality scheme Fibre flax Olive oil and table olives Olive oil and table olives Quality bovine meat / Flax Olive oil Table olives / / / Final Report November 2011 Page 104 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 1994-2000 period PDO, PGI or TSG products / / / Internal market Reg (EC) n°94/2002 Reg (EC) n°1071/2005 Information about PDOs, PGIs, TSGs and the PDO, PGI or TSG graphic symbols Information about organic Organic farming farming Quality wines psr, table wines with a geographical indication Quality wines psr, table wines with a geographical indication Information on the quality and safety of food and nutritional and health aspects Information about agricultural production systems that guarantee product traceability and the labelling of such products / / / / / / / / / / Source: ADE based on the concerned Regulations. Reg (EC) n°501/2008 PDO, PGI or TSG Organic farming Wines with a protected designation of origin or a protected geographical indication, wines with an indication of the wine grape variety / / / / Labelling of eggs for human consumption Seed oils Honey and beekeeping products Graphic symbol for the outermost regions / Marking of eggs for human consumption Seed oils Honey and beekeeping products Graphic symbol for the outermost regions Poultry meat Regarding third country markets, sectors and products which may be covered by the actions were established in the Commission Regulation (EC) n°2879/2000, based on two criteria. The first one is the same as for the internal market: typical or quality agricultural products with high added value. The second one concerns exports opportunities or the potentialities of new markets in third countries. Some products have been added over time (highlighted in grey in the table below) without removing any product from the list. Note that between 1994 and 2000, only flax and olive oil were considered for the promotion activities in third countries markets. Today, the list as defined in Annex II of Commission Regulation (EC) No 501/2008 includes 10 products and two themes (figure below). Both for the internal and third country markets the list has mainly been enlarged over time, some products being added without removing any product from the list. The list and its evolution show the responses given to priorities but do not reflect a real policy strategy. Final Report November 2011 Page 105 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 30: Evolution of the list of themes and products covered by promotional measures on third country markets since 1994 Third country markets Fresh, chilled and frozen beef, veal and pig meat; food preparations based on these products Quality poultry meat Cheese and yoghurt Olive oil and table olives Reg (EC) n°67/2005 Reg (EC) n°1346/2005 Fresh, chilled and frozen beef, veal and pig meat; food preparations based on these products Quality poultry meat Milk products Olive oil and table olives Quality wines psr, table wines with a geographical indication Quality wines psr, table wines with a geographical indication Reg (EC) n°2879/2000 Spirit drinks with a geographical indication or a reserved traditional description Fresh and processed fruit and vegetables Products processed from cereals and rice Spirit drinks with a geographical indication or a reserved traditional description Fresh and processed fruit and vegetables Products processed from cereals and rice Fibre flax Live plants and products of ornamental horticulture PDO, PGI or TSG Organic farming Source: ADE based on the concerned Regulations. Reg (EC) n°501/2008 Fresh, chilled and frozen beef, veal and pig meat; food preparations based on these products Quality poultry meat Milk products Olive oil and table olives Wines with a protected designation of origin or a protected geographical indication, wines with an indication of the wine grape variety Spirit drinks with a geographical indication Fresh and processed fruit and vegetables Products processed from cereals and rice Fibre flax Live plants and products of ornamental horticulture PDO, PGI or TSG Organic farming A large diversity of (types of) products and themes Having a look at the products and themes on the basis of a documentary review (J.1.1) and the stakeholders (both competent bodies and proposing organisations) views (J.1.2) one notices the high diversity of products at several levels. Indeed, the list includes groups of products, single products, fresh products, processed products, themes, “standard” products and products produced under EU quality schemes. Groups of products and single products: promotion can concern single products like fibre flax as well as groups of products (e.g. “fruit and vegetables”) that gather a wide range of single products. Some stakeholders state that having large product categories guarantees greater flexibility to the proposing organisations in the selection of the products for which they wish to implement the promotional activities. Final Report November 2011 Page 106 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Fresh and processed products: for some products, promotion may cover processed products at the first stage of processing. Processed products usually generate more added value and represent a larger share of the EU exports – in terms of value – than fresh products. Moreover, some stakeholders stressed the importance of the food industries as the largest outlet for agricultural products. Themes and products: promotion activities may concern products such as milk products as well as themes such as organic food. Therefore, promotion can be differentiated by the production process, with a potential contribution to the specific objectives of the Regulation, namely the EU image and the knowledge about the EU products. “Standard” and quality products: products like “standard” olive oil on one hand and products produced under EU quality schemes such as PDO or PGI wines on the other hand are both on the list. For some types of consumers, they do not have the same impact in terms of visibility and impact on the EU image and knowledge of EU products. J.1.2 Stakeholders have no opinion about the list or find it relevant Quality schemes are generally considered particularly appropriate by the surveyed stakeholders (competent bodies) to achieve the objectives of the Regulation as is illustrated in the online survey results regarding both the internal (figure 14) and third country markets (figure 15) 55. However, some of them underline that for a given product, promotion should not be limited to those that are originating from a quality scheme (i.e. meat on the internal market) or to those with a system of labelling (i.e. eggs on the internal market). Indeed, these restrictions limit the promotion opportunities for important EU agricultural products. Moreover, some stakeholders consider that all European products are high quality products due to the various cross-compliance requirements for the direct CAP payments. 49F Figure 14: Extent to which competent bodies consider the listed themes and products relevant on the national or EU market Source: On line survey (ADE) 55 These results are also discussed under the next section. Final Report November 2011 Page 107 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Figure 15: Extent to which competent bodies consider the listed themes and products relevant on third country markets Source: On line survey (ADE) As shown in Figures 14 and 15 a large share of stakeholders have no opinion about the list of products (average of 46%) with peaks above 60% for the graphic symbol for the outermost regions and fibre flax, especially for third country markets (fig 15). Half of the respondents, however, consider milk and milk products (54 %) relevant to some or to a large extent, as well as fresh fruit and vegetables (46 %), organic farming (43 %), and meat in an EU/national quality scheme (40 %). Processed fruit and vegetables (31 %), quality wines PSR, table wines with GI (29 %) as well as PDO, PGI and TSG (31 %) are also considered relevant to a large extent by around one third of the respondents. J 1.3-1.5 The statistical analysis shows a variety of distinct market and consumption trends, depending on the product and the market The analysis conducted through the judgment criteria J.1.3 to J.1.5 based on FAOSTAT and EU Export Helpdesk data reveals a large diversity of findings. These are synthesised per market categories hereinafter. EU market trends in the worldwide context To analyse EU market trends in a worldwide context (J.1.3), four categories of market trends have been established for different products by means of three indicators. These four trend categories are the following: 1. Growing and significant market trend for the EU: when there is an increase of the EU export and import values (positive market trends) and the EU exports of the Final Report November 2011 Page 108 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC product represent more than 5 % of total agricultural commodities export value in 2008 2. Growing but small market trends for the EU: when there is an increase of the EU export and import values (positive market trends) but EU exports of the product represent less than 5 % of total agricultural commodities export value in 2008 3. Declining and small market trends for the EU: when there is a decrease of the EU export and import values (negative market trends) and the EU exports of the product represent less than 5 % of total agricultural commodities export value in 2008 4. Declining but significant for the EU: when there is a decrease of the EU export and import values (negative market trends) but the EU exports of the product represent more than 5 % of total agricultural commodities export value in 2008. Table 31: Product classification at the EU level in the worldwide context J.1.3 EU market in the worldwide context 2000-2008 market trends % in total EU Market category (exports, imports) Exports (2008) Positive >5% 1. Growing and significant for EU Positive <5% 2. Growing but small for EU Negative <5% 3. Declining and small for EU Negative >5% 4. Declining but significant for EU Source: ADE Product example Fruit and vegetables Honey Fibre flax / For instance, the fruit and vegetables sector can be identified, at a global level, as a growing market with positive export and import trends. Moreover, this sector represents a significant share (16 %) of the total EU agricultural market in 2008 and is accordingly identified as a “growing and significant market for the EU”. Stakeholders confirm this statement by considering the fruit and vegetables sector as one of the most relevant of the list, both on the internal (figure 14) and on third country (figure 15) markets. Conversely, from the statistical point of view, the fibre flax sector is identified as a small declining market by the statistical analysis and as a market with little interest by a majority of stakeholders (figures 14 and 15). Note that no product on the list was identified as a “Declining but significant for EU” market. Internal market trends (intra-EU) The internal market trends (intra-EU J.1.4) analysis identifies seven relevant issues at stake based on intra-EU export 56 trends over the 2002-2009 periods (I.1.4.1) and the EU consumption trends over the 2000-2007 periods (I.1.4.2) (Table 32): 1. Consumption issues: consumption issues are identified for a given product when the intra-EU trade is stable or increasing (positive or stable) although EU consumption decreases over time (negative). The issues are thus obviously related to the decrease of consumption. 50F 56 The term intra EU trade is used under this section concerns intra EU export trends and intra EU import trends are not covered. Intra EU trade concerns exports from Member States to other Member States. Final Report November 2011 Page 109 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 2. Trade issues: these are identified when EU consumption is increasing or stable (positive or stable) while intra-EU trade is decreasing (negative). The issues are thus related to the support of EU products in order to keep the EU market share on its own market. 3. Trade opportunities: these are identified when EU consumption is increasing (positive) and the intra-EU trade is stable. There is thus an opportunity to increase intra-EU trade in response to consumption growth. 4. Consumption opportunities: these are identified when the intra-EU trade is growing (positive) while EU consumption is stable. There is thus an opportunity to increase consumption in response to the positive trend observed at intra-EU trade level. 5. Trade and consumptions opportunities: both EU consumption and intra-EU trade are stable over time. There are thus possible actions on trade and consumption in order to stimulate the market. 6. Declining market: the market is characterized as declining when both intra-EU trade and EU consumption are decreasing. 7. Growing market: the market is characterized as growing when both intra-EU trade and EU consumption are increasing. For instance, the fruit and vegetables sector on the internal market is characterised by a decrease of EU consumption over the last four years while the internal market represents the most important outlet for EU trade (around 85 %) with stability over time: the issues are obviously related to the decrease of consumption of fruit and vegetables. The olive oil and table olives sector shows an increase of its consumption in the EU while the internal market is observing a slight and recent decreasing phase. The issues regarding these products thus concern the support of intra-EU trade in order to follow the consumption growth and to keep EU market share on the internal market. This sector is thus identified as a “trade issues” sector. Note that no product on the list was identified as a “Growing” market. Table 32: Product classification at the internal market level J.1.4 Internal market Market category 1. 2. 3. 4. 5. Consumption issues Trade issues Trade opportunities Consumption opportunities Trade and consumptions opportunities 6. Declining market 7. Growing market Source: ADE. Final Report EU consumption (2000-2007) Negative Positive or stable Positive Stable Intra-EU trade trends (2002-2009) Positive or stable Negative Stable Positive Fruit and vegetables Olive oil and table olives Milk Honey Stable Stable Wine Negative Positive Negative Positive Fibre Flax / November 2011 Product example Page 110 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC It should be mentioned that some products not included in the list for the internal market but eligible for third country markets were identified as strategic issues on the internal market by the statistical analysis and the stakeholders. This is the case for products processed from cereals where “consumption opportunities” were identified on the internal market. Indeed, this sector is characterized by the stability of EU consumption while the growing intra-EU market is the most important outlet (75 %). In order to achieve the objectives of the Regulation, the focus should be on consumption of these products considering the aforementioned elements. The products processed from cereals are also the most often mentioned products to be added to the list by the stakeholders. EU to third countries (extra-EU) market trends The analysis of extra-EU market trends (J1.5) identifies six relevant issues at stake on third country markets according to the related indicators, namely the extra-EU export trends over the 2002-2009 periods (I.1.5.1) and the worldwide consumption over the 20002007 periods (I.1.5.2) (Table 33): 1. Trade opportunities: these are identified when worldwide consumption is increasing 2. 3. 4. 5. 6. (positive) and extra-EU exports are stable or increasing (positive or stable). There is thus an opportunity to benefit from this favourable context (essentially influenced by the consumption growth) by boosting EU exports. Market shares opportunities: these are identified when worldwide consumption is stable and extra-EU exports are stable or increasing (positive or stable). In this context of consumption stability, there is thus an opportunity to gain market shares or at least to continue to raise the value of extra-EU exports. Declining market: the market is characterized as declining when both extra-EU exports and worldwide consumption are decreasing. Trade issues: these are identified when worldwide consumption is increasing (positive) while extra-EU exports are decreasing (negative). The issues are thus related to the support of extra-EU trade in order to benefit from worldwide consumption growth and to reverse the decreasing trend of extra-EU exports. Market shares issues: these are identified when worldwide consumption is stable while extra-EU exports are decreasing (negative). The issues at stake are thus related to the support of extra-EU products trade in order to keep EU market shares on the third country markets. Evolution to an EU niche market : the market is considered as evolving to a niche market for EU products when the worldwide consumption is decreasing (negative) while the extra-EU exports are stable or increasing (positive or stable). Final Report November 2011 Page 111 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 33: Products classification at the extra-EU level J.1.5 Extra-EU market Market category 1. 2. 3. 4. 5. 6. Trade opportunities Market share opportunities Declining market Trade issues Market shares issues “Evolution to an EU niche market” Source: ADE. Worldwide consumption (2000-2007) Positive Stable Negative Positive Stable Negative Extra-EU export trends (2002-2009) Positive or stable Positive or stable Negative Negative Negative Positive or stable Product example Spirit drinks Wine Fibre Flax / / / For instance, the worldwide consumption of spirit drinks is growing while extra-EU market shows a slight growth: the issue for these products is thus at the trade level in order to benefit from the increase of worldwide consumption. This sector is thus identified as a sector with “trade opportunities” in the third country markets. Note that no product on the list was identified as belonging to the “Trade issues”, “Market share issues” and “Evolution to an EU niche market” categories. As shown for the internal market, some products with strategic issues are not in the list of products for third countries markets although strategic issues have been identified at the extra-EU level by the statistical analysis and the stakeholders. For instance, honey and beekeeping products on the third country markets are characterised by a relative stability of worldwide consumption and the extra-EU market: the issue identified for this sector is to increase EU market share on third country markets and the category is thus “market shares opportunities”. Consequently this sector could be added to the list in order to achieve the objectives of the Regulation. Summary of the main facts The statistical approach shows a diversity of issues for the different products and themes. Indeed, various market and consumption trends were identified depending on the product, first in the worldwide context, as well as from the internal market perspective and finally from the sole third country market perspective. Moreover, some products are not addressed on the internal or third country markets although they seem to be a strategic issue on these markets, as it has been identified with the statistical analysis and confirmed by stakeholders. Finally some stakeholders argue that it may be useful to harmonise the list of themes and products for the internal and the third country markets. J.1.6 The list contains the most important products in terms of EU export values According to the analysis of EU agricultural product exports, all the products on the list are among the most important products in terms of value, except for table olives (highlighted in grey in the table below). Note however that some products are not analysed as individual products but are taken into account under groups of products (i.e. honey is Final Report November 2011 Page 112 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC part of the sugar and honey group) due to the nature of available data. The themes (graphic symbol for the outermost regions, PDO, PGI, TSG and organic products) as well as quality products have not been taken into account in the analysis because of the lack of data. Table 34: Ranking Products and groups of products classification according to the EU export value in 2008 Products % 1 Wine * 2 Crude Materials51F57 ** 3 Food preparations ** 4 Cheese of whole cow milk * 5 Beverages distilled and alcoholic * 6 Wheat 7 Chocolate preparations 8 Pastry 9 Cigarettes 10 Pig meat * 11 Beverage non-alcoholic 12 Meat and cattle boneless(beef and veal) * 13 Beer of barley 14 Food wastes57 15 Pork * 16 Chicken meat * 17 Pet food 18 Fruit preparations * 19 Cattle meat * 20 Olive oil, virgin * Source: ADE based on FAOSTAT. 4,8% 4,6% 4,4% 4,4% 3,5% 2,9% 2,9% 2,9% 2,6% 2,5% 2,0% 1,6% 1,5% 1,4% 1,3% 1,3% 1,2% 1,1% 1,1% 1,1% Groups of products (non exhaustive list) Fruit and vegetables * Beverages and tobacco ** Cereals and preparations ** Dairy products and eggs ** Meat fresh, chilled and frozen ** Crude materials (large)57 ** Coffee, tea, cocoa and spices Wine, vermouth and similar ** Animal and vegetable oil ** Sugar and honey ** Meat prepared ** Hides and skins Offals edible fresh** Textile fibres ** % 16,3% 16,1% 12,5% 10,0% 9,4% 7,4% 5,9% 4,9% 4,8% 2,4% 1,4% 0,8% 0,5% 0,3% * Product which may be covered by promotional measures ** Group of products that contains one or more products which may be covered by promotional measures It should be noted that some important products in terms of EU export value are not contained in the list, such as wheat and pastry. A large number of stakeholders, would like the list to be broadened in order to ensure the best possible coverage of European products, with the final purpose of achieving the objectives of the Regulation. This is the case for products such as those processed from cereals that do not appear in the list targeting the internal market (although they are in the list targeting third country markets) and vice versa. In addition, products that do not figure in any list but present interest according to some stakeholders include: delicatessen/cold cuts, spices, essential oils, cork, sheep, goat and rabbit meat, herbal and aromatic plants etc. Finally, the presence of some products with small export values (for example honey) could be justified due to their potential in terms of increased consumption of EU products or to enhance the EU image and the knowledge about these products. 57 These definitions are based on the FAO classification. For more details, see glossary at www.faostat.org Final Report November 2011 Page 113 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC To what extent has the list of themes and products, which may be covered by promotional measures, been relevant to achieving the objectives laid down in the Regulation? Evaluative Judgement The list covers a wide range of different types of agricultural and food products (e.g. including fresh and processed products at the first stage of processing; single products (such as milk) or group of products (such as fruit and vegetables or organic products); “standard” products (such as olive oil) versus products under EU quality schemes). The list is mainly the heritage of the past, resulting from the merger of 12 sectors covered by promotion mainly on the internal market. Since 2000, three criteria have guided the integration of new products: (i) the fact that highly differentiated and valued products are typical or produced in a quality scheme, (ii) the need to handle market or consumers’ confidence crisis in individual sectors and (iii) the potential export opportunities in third country markets. Although the revisions responded to priorities or crises, there is a lack of overall strategy in this list. The statistical approach showed the high diversity of issues, in terms of market and consumption trends (growing, stable or declining), depending on the product and the market taken into consideration. It should be noted that the statistical analysis also confirmed that most the important products in terms of export values are included. Almost half of the competent bodies considered milk and milk products, fruit and vegetables, meat in an EU/national quality scheme, organic farming relevant to a large or to some extent. Almost a third of the competent bodies had the same appreciation for processed fruit and vegetables, wines produced under quality schemes and PDO-PGITSG. A large share of competent bodies did not express any opinion about the list of products, especially for some more marginal products (fibre flax, outermost regions) and for third country markets. They also considered it useful to harmonise the list of themes and products targeting the internal and the third country markets. The list of themes and products is broadly relevant. Thanks to a large scope of eligible products they do altogether contribute to enhancing the image and improving knowledge of EU agricultural products. The large scope does also allow responding to a high heterogeneity of situations and various trends (growing/stable or declining markets, etc). However, the list of products and themes alone is not enough to achieve the objectives of the regulation. Although the principles of the policy are defined, an overall strategy is missing. Final Report November 2011 Page 114 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 4.2.2 Evaluation Question 2 To what extent has the list of third-country markets, in which promotional measures may be carried out, been adequate to achieve the objectives laid down in the Regulation? Rationale and approach The question aims at assessing to what extent the list of third countries or geographical areas is adequate in a market development perspective (e.g. open up new markets) outside the EU for the different products/themes covered by the Regulation. The methodology applied to answer the question is based on an analysis of statistics from FAOSTAT 58, the EU Export Helpdesk 59 and the World Bank 60 databases completed by document reviews and stakeholders’ interviews. The answer to the EQ is organised according to six judgement criteria (Table 35). The first criterion is based on documents reviewed. There are four judgment criteria that provide quantitative elements: judgment criteria J.2.2, J.2.3 and J.2.4 appraise the relevance of the listed countries to achieve the objectives of the Regulation from a figure-based perspective. There are two judgment criteria that regard qualitative elements namely stakeholders’ interviews and the on-line survey (J.2.5) in order to complement the issues identified in the statistical analysis. The last judgment criterion enables determining if the list includes the most important EU trade partners in terms of EU export values (J.2.6). 52F 53F 54F The main limitation in this approach is the data availability for each theme and product which may be covered by promotional measures in the third countries as already mentioned under EQ 1 especially for EU quality schemes which are an important aspect of the promotion policy. There is no data on Kosovo available in the FAO database, so the country is not included in the analysis although the export data for this country is available on the EU Export Help Desk database. Table 35: EQ 2 Judgment criteria and indicators J.2.1 Documents regarding promotional measures validate the list of third countries on the basis of identified markets for EU agricultural products I.2.1.1 Documents validate the list of third countries for promotion on the third Programming documents countries market and evaluation reports J.2.2 The EU exports to the third countries of the products covered by promotional measures over a recent period present high values and consequent growths I.2.2.1 Trends in EU exports to the third countries of the products which may be EU Export Helpdesk covered by promotional measures Number of products with high export growth (>100%) over the 2002-2009 period Number of products with high EU export values (> 100 €m) in 2009 Number of products for which the country is in the top ten of the EU export destinations in 2009 58 59 60 http://faostat.fao.org/ http://exporthelp.europa.eu/ http://data.worldbank.org/ Final Report November 2011 Page 115 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC J.2.3 Third countries consumptions levels and/or growth of the products covered by promotional measures over a recent period are higher than the world and the regional average I.2.3.1 Trends in third countries consumptions of the products which may be FAOSTAT covered by promotional measures Number of products with higher consumption per capita than the worldwide average in 2007 Number of products with higher consumption growth than the worldwide average over the 1995-2007 period J.2.4 Third countries GDP per capita and its growth are higher than the worldwide average I.2.4.1 Trends in GDP third countries World Bank GDP per capita vs. worldwide average in 2009 GDP per capita growth vs. worldwide average growth over the 2000-2009 period J.2.5 Stakeholders validate the list of third countries on the basis of identified markets for EU agricultural products I.2.5.1 Stakeholders validate the list of third countries for promotion on the third Interviews, questionnaire countries market J.2.6 The list of third-country markets, in which promotional measures may be carried out measures contains the most important third countries in terms of EU exports values I.2.6.1 Third countries classification according to the extra-EU exports value in EU Export Helpdesk 2009 for each product which may be covered by promotional measures Detailed answer Documents reviewed: a rather stable list over time The list of third-country markets, characterized by the presence of both countries and geographical areas (Table 36), has evolved little over time. It includes 18 countries and 5 geographical areas. The main changes concern the integration of five countries in the framework of Commission Regulation (EC) n°67/2005 as well as the focus on three countries (Ukraine, Romania and Bulgaria), initially included in the geographical area of Central and Eastern Europe. These latter two countries joined the EU in 2007 and were thus removed from the list. Table 36: Third-country markets in which promotional measures may be carried out Reg (EC) n°501/2008 Switzerland India Norway South Africa Russia Australia Japan New Zealand China Turkey South Korea Croatia Source: ADE based on the concerned Regulation. Final Report Bosnia and Herzegovina FYR of Macedonia Serbia Montenegro Kosovo Ukraine November 2011 Middle East South-East Asia North Africa North America Latin America Page 116 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC From the statistical perspective: according to the scoring system proposed, some listed third countries seem more relevant than others Quantitative judgement criteria are used in order to cluster third countries from a statistical perspective. The first criterion concerns EU export trends to third countries for the eligible products and themes (J.2.2). The second criterion concerns the consumption trends in the targeted countries, both in terms of consumption quantities and growth (J.2.3). The last criterion concerns GDP trends per capita and growth (J.2.4). In order to sort by relevance the countries eligible for promotion activities, an indicative scoring system has been developed. This should help to reflect their relevance in achieving the objectives of the Regulation, from a statistical perspective. Each judgment criterion is subdivided into several indicators. For each indicator (taking into account the importance in absolute value and the evolution over a recent period) a mark has been attributed to each country. Table 37 presents the scoring system with the example of China. The sum of all marks leads to a ranking of the countries. Scoring system approach The scoring system adopted is supported by indicators and sub-indicators for the three criteria, as presented in the table below. Judgment criterion J.2.2 “EU export trends to the country”, has one indicator I.2.2.1 “Trends in EU exports to the third countries of the products which may be covered by promotional measures” subdivided into three subindicators: Number of products (maximum ten) with high export growth (>100%) over the 20022009 period on ten products : 0 to 10 points are allocated to this specific sub-indicator in the scoring system Number of products (maximum ten) with high EU export values (> 100 €m) in 2009 61 : 0 to 10 points Number of products (maximum ten) for which the country is in the top ten of the EU export destinations in 2009 : 0 to 10 points Criterion J.2.3 “country’s consumption trends” the indicator I.2.3.1 “Trends in third countries consumptions of the products which may be covered by promotional measures” is divided into two sub-indicators with the following scoring: Number of products (maximum nine) with higher consumption per capita than the worldwide average in 2007 : 0 to 9 points Number of products (maximum nine) with higher consumption growth than the worldwide average over the 1995-2007 period: : 0 to 9 points Indicator I.2.4.1 “Trends in GDP third countries” related to the judgment criterion J.2.4 “GDP trends”, two sub-indicators are defined with the following scoring: GDP per capita vs. worldwide average in 2009 (8.581 US$) : 0 to 10 points 5F 61 The value of 100 €M allows to identify if the analysed country is important for the EU in terms of export values and the most recent year 2009 was chosen for this sub-indicator. The evolution of EU exports is reported under the previous sub-indicator. Final Report November 2011 Page 117 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC If the GDP per capita value for the analysed country is lower than the worldwide average in 2009 (8.581 US$), no point is allocated. If the values are equal, 1 point is given. If the country value is higher, the points given to it are calculated by dividing the country value by the worldwide average 62. GDP per capita growth vs. worldwide average growth over the 2000-2009 period (+ 1.3%) : 0 to 10 points If the GDP per capita value growth for the analysed country is lower than the worldwide average over the 2000-2009 period (+ 1.3%), no points are allocated. If the values are equal, 1 point is given. If the country value is higher, the points given to the country are calculated by dividing the country value by the worldwide average 63. 56F 57F Table 37: The scoring system applied to the China case J.2.2 The EU exports to China of the products covered by promotional measures over a recent period present high values and consequent growths I.2.2.1 Trends in EU exports to the third countries of the China Facts Scoring products which may be covered by promotional measures Number of products with high export growth (>100%) Eight products with high exports 8/10 over the 2002-2009 period growth (>100%) Number of products with high EU export values (> 100 Four products with high EU exports 4/10 €m) in 2009 values (> 100 €m) Number of products for which the country is in the top Seven products in the top ten of the 7/10 ten of the EU export destinations in 2009 EU exports countries J.2.3 Third countries consumptions levels and/or growth of the products covered by promotional measures over a recent period are higher than the world and the regional average I.2.3.1 Trends in third countries consumptions of the China Facts Scoring products which may be covered by promotional measures Four products with higher Number of products with higher consumption per capita consumptions per capita than the 4/9 than the worldwide average in 2007 worldwide average Number of products with higher consumption growth Six products with higher consumption 6/9 than the worldwide average over the 1995-2007 period growths than the worldwide average J.2.4 Third countries GDP per capita and its growth are higher than the worldwide average 0B I.2.4.1 Trends in GDP third countries GDP per capita vs. worldwide average in 2009 > : number of times higher ; = : 1 point ; < : 0 point GDP per capita growth vs. worldwide average growth over the 2000-2009 period > : number of times higher ; = : 1 point ; < : 0 point Total scoring Source: ADE China Facts The GDP per capita is lower than the worldwide average The GDP per capita growth is 7.1 times higher than the worldwide average (2000 to 2009) 63 0/10 7.1/10 36.1/68 1B 62 Scoring 2B The maximum of points attributed in the framework of this analysis is 9.2 points for the Norway. The maximum of points attributed in the framework of this analysis is 7.1 points for the China. Final Report November 2011 Page 118 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The final ranking list of countries sorted by relevance can be viewed in the Table 38. Please note that geographical areas have not been handled in JC 2.2 to 2.4. The main conclusion that can be drawn from this table is the large range of relevance among the listed countries with respect to issues related to EU exports, consumption and GDP. Note that interviewed stakeholders (particularly the proposing organisations) tend to choose the countries for their promotional measures on the basis of market studies covering the production, consumption, trade and GDP trends of the targeted countries. This statistical approach shows that some of the countries such as Russia, Norway and China appear to be more relevant than others like India, Serbia and Bosnia and Herzegovina. Table 38: Country Russia Norway China Switzerland Australia Ukraine Source: ADE. Scoring of the listed third-country market in which promotional measures may be carried out regarding their relevance Total points 42.3 36.2 36.1 34.4 32.3 27.1 Country Japan South Korea Croatia New Zealand Montenegro Macedonia FYR Total points 26.6 21.8 21.3 18.5 18.2 16.8 Country Turkey South Africa India Serbia Bosnia and Herzegovina Total points 16.8 16.2 16.1 15.6 13.1 With few notable exceptions, stakeholders’ views on individual country relevance coincides with the statistical analysis Most of the countries identified as most relevant to achieve the objectives of the Regulation with the above statistical analysis are also those most often considered by the stakeholders as “relevant to a large extent” (figure 16), as shown in the on-line survey. This is the case for China (43 %), Russia (40 %), Japan (34 %), Switzerland (26%), Norway (23 %), Ukraine (20%) and South Korea (20%). Final Report November 2011 Page 119 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Figure 16: Extent to which competent bodies consider the listed third-country relevant to achieve objectives of the Regulation Source: On line survey (ADE) However, the statistical analysis and the stakeholders view are not unanimous regarding Australia and India. Australia seems to be very relevant from a statistical perspective although few stakeholders consider this country to be relevant “to a large extent”. On the contrary, India, which appears as not very relevant from a statistical point of view, is considered very relevant by over 30% of the stakeholders. These differences might originate from the selective perception of stakeholders. Indeed, India is characterised by important differences in social classes and some stakeholders have underlined the great interest of the upper classes in European traditions. However, such facts, even if they are true, are too minor in proportion to be reflected in the statistical analysis. Another reason why India is perceived to be more important by stakeholders is the large size of its market. However, this perceived potential is quantitatively weakened when broken down per capita in the statistical analysis. That being said, it is true that additional indicators like “total GDP” could have been included in the analysis. Regarding Australia, it is likely that stakeholders underestimate the potentialities of this country highlighted by the analysis of EU exports, consumption and GDP trends, but possibly due to the long distance. Geographical areas The geographical area level is too large to identify and appraise the relevance in achieving market developments. Indeed, there are sometimes strong disparities regarding EU exports, consumption and GDP indicators between countries in the same geographical areas. Final Report November 2011 Page 120 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC For instance, table 38 shows that Australia and New Zealand (that are in the list as countries located in the same geographical area) do not have the same relevance degree (respectively 32.3 and 18.5) to achieve the objectives of the Regulation. Likewise, the geographical areas defined by the Commission Regulation (EC) n°501/2008 (Middle East, South-East Asia, North Africa, North America and Latin America) include countries that can be very different from this perspective. Moreover, some stakeholders state that their choice of extra-EU markets is essentially made at the country level rather than the geographical level. Some of them even establish the list of priority countries based on “country-product” couples and they also underline that no proposing organisation should normally target a country without getting informed on the market possibilities at the country level, or even for large countries like China, at the city level. In any case, it seems that there is a general consensus on the limited relevance of the geographical areas level. The only positive point is that it gives stakeholders the freedom to choose the countries in the geographical area. The list includes the most important third countries in terms of EU export value A list with a high level of coverage The analysis conducted under the judgement criterion J.2.6 ranked third countries according to their importance (%) in the extra-EU exportations value in 2009 for each product which may be covered by promotional measures. The results of this analysis show that the list, with both its countries and geographical areas, includes the most important ones in terms of EU export values. However, some of the EU major trading partners such as the United States are not explicitly listed as individual country but only indirectly eligible thanks to the geographical areas. If the individual countries listed reflect some degree of priority, then the list might be reviewed in this perspective. The list covers a large part of the globe thanks to the geographical areas. Africa is one of the geographical areas that are not entirely covered, with the exception of the North Africa area and South Africa. There are also other regions that are not included in the list such as Asia (with the exception of the South-East Asia area) and Oceania but some countries belonging to these areas are in the list; for instance, Japan, China and India in Asia and Australia and New Zealand in Oceania. Although stakeholders (J.2.5) generally agree that the countries and the areas of the list cover almost all countries that are relevant to achieve the objectives of the Regulation, some of them suggest adding other countries to the list that present particular interest for their Member States, such as Belarus, Iraq, Iran, the Commonwealth of Independent States, some countries in Central Asia and the emerging countries of Africa such as Angola. Nevertheless, the relevance of these countries requires further investigation. Final Report November 2011 Page 121 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC To what extent has the list of third-country markets, in which promotional measures may be carried out, been adequate to achieve the objectives laid down in the Regulation? Evaluative Judgement The list of third countries and regions covers almost all regions of the world except some of the poorest countries namely in sub-Saharan Africa. In order to assess the relevance of this list of countries compared to the objectives of the Regulation, an indicative scoring system has been developed based on EU exported product trends to third countries, third countries consumption of covered products and GDP trends. This indicative scoring system shows a large range of relevance among the listed countries. To arrive at the list of eligible third country markets the statistical approach needs to be completed by more qualitative criteria. The most important trading partners in terms of EU export values are covered in the list, in an indirect way for some of them, thanks to the geographical areas, with the notable case of the United States. The considerations above suggest that a revised prioritisation of countries could be established on the basis of indications such as statistical data, complemented by qualitative criteria. Regarding geographical areas, this level of coverage does not allow precise identification of their degree of relevance as they include countries with strong differences in terms of macroeconomic indicators used. Moreover, stakeholders highlight that their choice of third country markets is not made at the geographical area level, which does not plead for their usefulness. However, their presence in the list leaves a large degree of flexibility to MS. Final Report November 2011 Page 122 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 4.2.3 Evaluation Question 3 To what extent have the information and promotion measures implemented at the Commission’s initiative been relevant to achieve the objectives laid down in the regulation? Rationale and approach The question seeks to verify the extent to which information and promotion initiatives undertaken at the level of the Commission are relevant to achieve the objectives in Regulation (EC) No 3/2008. The response to this question is based on available documentary information and interviews in the EC and in Member States involved in programmes submitted since 2008. The documentary sources are: the report from the Commission to the European Parliament and the Council on the application of Council Regulation (EC) No 3/2008; framework contract evaluations in third country markets; the organic food and farming promotion website; and an evaluation report on the European Action Plan for Organic Food and Farming. Three judgment criteria are used. The first one (J.3.1) assesses the relevance of issues addressed by the Commission initiatives in terms of Community interest, both on the internal market and in third country markets. The second criterion (J.3.2) concerns complementarities between Commission initiatives and co-funded promotion programmes. The third criterion (J.3.3) concerns high level trade visits and their potential effect on facilitating and enhancing further business opportunities. Table 39: EQ 3 Judgment criteria and indicators J.3.1 The products and themes covered by promotion measures carried out at the initiative of the Commission are an issue at stake for the EU= of Community interest I.3.1.1 The Commission has carried out promotion measures on Regulations, expenditure data, EC products and themes relevant for the EU on the internal market policy statements I.3.1.2 The Commission has carried out promotion measures on Regulations, expenditure data, EC products and themes relevant for the EU on third country markets policy statements J.3.2 Complementarities between Commission initiatives and co-funded programmes can be identified I.3.2.1 Evidence that co-funded programmes have been implemented Interviews for the products and themes carried out at the initiative of the Commission on the internal market I.3.2.2 Evidence that co-funded programmes have been implemented CD-ROM & DG Agri website for products and themes carried out at the initiative of the Commission data, past evaluations on third country markets J.3.3 High level trade visits facilitate and enhance business opportunities for EU agricultural products I.3.3.1 EU organisation of high level trade visits and participation in Local press campaigns linked to promotional events creates a positive image of EU agricultural products visits, visitors of EU stands… I.3.3.2 High level trade visits facilitate and enhance further business Interviews with/or questionnaire opportunities sent to former participants Source: ADE Final Report November 2011 Page 123 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The main limitation regards the fact that the evaluator did not have specific documentation on the EAT campaign which took place from 2005-2007, except for some comments in the framework contract evaluations of third country markets. The EAT campaign took place more than 4 years ago and few of the interviewees had a good record of it. Another limitation is related to the high level visits. In order to evaluate their impact on the creation of new business opportunities ex post surveys would be required among participants after 6 months- 1 year but this follow up is currently not implemented. Detailed answer Background and rationale of Commission initiatives Commission initiatives for promotion and information of agricultural products are explicitly foreseen in article 10 of the current Council Regulation (EC) 3/2008 on promotion for agricultural products. Under the former regulations (2000-2007) these initiatives were initiated in third countries (Council Regulation 2702/1999, article 7.2) where the Commission was enabled to entirely and directly finance and manage information campaigns, in particular on the EU systems of PDO, PGI, TSG and organic farming and high level trade visits. This possibility was extended to the internal market at the end of 2004, and in terms of activities to all promotion measures, as long as these measures are of Community interest or no appropriate proposal has been submitted. According to Article 10 of Council Regulation (EC) No 3/2008, Commission initiatives can be carried out on the following themes, after having informed the Management Committee for the common organisation of agricultural markets: on internal and third country markets: - information campaigns, especially on PDOs, PGIs, TSGs, organic farming and other Community schemes for quality standards and labelling of agricultural products and foodstuffs, as well as on the graphic symbols laid down in the applicable Community legislation; - information campaigns on the EU system covering wines with a protected designation of origin or geographical indication, wines with an indication of the wine grape variety and spirit drinks with a protected geographical indication; on the internal market, participation in events, fairs and exhibitions of national or European importance by means of stands aimed at enhancing the image of EU products; on third country markets: (i) high-level trade visits; (ii) public relation work, promotion and advertising, especially on intrinsic features and advantages of EU products (such as quality and safety of food or labelling); (iii) participation in events, fairs and exhibitions; (iv) studies of new markets. The current promotion regulation is characterized by a bottom-up approach, in which the proposed programmes rest especially on the dynamics and the interests of the professional trade and inter-trade organizations of the EU Member States who respond to calls for Final Report November 2011 Page 124 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC proposals. It may happen that some products or themes of Community interest especially at European level are not sufficiently addressed by the programmes proposed. Commission initiatives intend to improve the effectiveness of the whole promotion policy by enabling the Commission to initiate information or promotion campaigns for specific products/themes not sufficiently addressed. Commission initiatives that have been implemented In practice, the following initiatives have been carried out so far: An information and communication campaign on PDO, PGI and TSG in third countries under the “EAT” slogan (European Authentic Tastes) (2005-2007) in the USA, Japan and China; An information campaign on organic farming on the internal market (2006-2008); and Eight high-level trade missions and promotional events in third countries. J.3.1 Products and themes covered by promotion measures carried out at the initiative of the Commission are an issue of Community interest As introduced before, Commission initiatives aim to improve the effectiveness of the whole promotion policy, by enabling the Commission to initiate information or promotion campaigns for specific themes or topics of Community interest, where no such proposals are submitted by proposing organisations from the Member States. The initiatives focused on information campaigns on quality schemes, as foreseen by the Regulation. As soon as the possibility was given to the Commission to operate on the internal market, the EU-wide campaign on organic food and farming was launched, prepared in 2005 and successfully implemented in 2006-2009. This campaign was a follow up of the European action plan for organic food and farming. In third country markets the EAT campaign focused on PDO, PGI and TSG. Most high level trade missions were also focused with EU quality schemes, mainly PDO, PGI and TSG. EU-wide campaign on organic food and farming on the internal market On the internal market, the first initiative - on the promotion of organic food and farming was launched to follow up on Action 1 of the European Action Plan for Organic Food and Farming (EUOAP). This plan was released in June 2004 with the intention of laying down the basis for an organic farming policy development in Europe. The action plan outlined 21 points including consumer information and promotion campaigns. The 2004 EUOAP stated that “the Commission analysis has shown that more emphasis should be put on facilitating the development of the market. The current market share is on average about 2 % in the EU-15. In order to increase this figure or even to maintain the current figure on the long run, more focus on consumer expectations is needed. Consumers need better information on the principles and objectives of organic farming as well as the positive impact on, for example, the environment.” Final Report November 2011 Page 125 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The preparation of an EU wide promotion campaign started in 2005 when DG Agri launched a call for a promotion project (€3M for a three-years period) covering the EU25. A media agency was contracted in 2006 64. The Commission was advised by a group of experts, including leading specialists in the sector, on proposals from Member States. 58F The outputs were made public in 2008. A website was developed in 22 languages to provide information on organic products and farming to farmers, organisations and final consumers. The website http://ec.europa.eu/agriculture/organic/toolbox_fr also provides a free toolbox which can be used by other campaigns promoting organic farming. It includes key messages and slogans developed during the European campaign, marketing material (brochures, posters, banners, leaflets…), radio and TV material etc. The EU wide campaign on organic food and farming has shown very positive results in terms of content and coverage. Many marketing and information tools have been developed and are available on the website and can be reused by all Member States. The website is one of the most visited of all DG Agri sub-sites, as mentioned by the Report from the Commission to the European Parliament and the Council from November 2010. Information campaign on PDO, PGI, TSG and organic farming on third country markets: the EAT campaign on third country markets On third country markets, the Commission launched an information campaign on PDO, PGI, TSG and organic farming under the European Authentic Tastes slogan. The programme intended to raise awareness about the European quality designations for food. The PDO, PGI, TSG and organic farming information & communication programme was implemented under the “EAT” slogan (European Authentic Tastes) in three countries: In the USA: for 3 years - € 3m budget In Japan: for 3 years - € 2m budget In China: for 2 years - € 1m budget. The campaigns took place between 2005 and 2007. No specific evaluation of the EAT campaign has been undertaken. All the framework contract evaluations on third country markets, except for the Japan evaluation, refer to the EAT campaign. According to the USA evaluation, several northern European countries felt excluded from the EAT programme as only products from southern countries were presented (France, Italy, Spain), with the exception of the UK. Explicit reference is made to one of the activities, namely the large presentation stand at the Fancy Food Show in New York in July 2006. For this specific event, confusion was generated with the co-funded “European Gourmet Project (EGP)” as both projects were not grouped together in one single “European Pavilion”. The authors write: “The fact that both EU booths, the EAT programme and the Plan Gourmet were not placed next to each other may have caused confusion and did not exploit potential complementarities and synergy effects”. In contrast, the evaluation of promotion programmes in China noted a real case of synergy between promotion programmes and the EAT programme. Professionals targeted by promotion programmes were already aware of EU logos and quality systems thanks to the EAT programme. 64 Source : « Synthesis and recommendations » of the Specific Support Action Project N° CT-2005-006591 « European Action Plan of Organic Food and Farming – Development of criteria and procedures for the evaluation of the EU Action Plan for Organic Agriculture ; Sixth Framework Programme for European Research of the European Commission Final Report November 2011 Page 126 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The priority given to these EU quality schemes is in line with the provisions of the Council Regulation (EC) No 3/2008. However, especially on third country markets, a question can be raised about extending activities to other products that do not necessarily belong to EU quality schemes but are important in terms of market shares and export opportunities. J.3.2 Complementarities between Commission initiatives and co-funded programmes It is not specified in the Council Regulation (EC) No 3/2008 that the Commission initiatives are supposed to develop complementarities with co-funded programmes. However, given the fact that EC initiatives did interact – due to their content and coverage – with co-funded programmes, it is relevant to analyse the potential benefits that could be drawn for co-funded programmes. Complementarities between the EAT campaign and PDO+ co-funded programmes on third country markets Over the period 2004-2010, 37 programmes related to PDO+ have been implemented by the Member States, with 20 in the internal market and 17 on third country markets (table below). Table 40: Number of co-funded PDO, PGI and TSG programmes implemented on the internal market and on third country markets 65 59F Year Internal Market 2004 2005 2006 2007 2008 2009 2010 Total 2 4 2 6 2 0 4 20 Third Country market 0 8 1 3 2 2 1 17 Total 2 12 3 9 4 2 5 37 Source: ADE, based on data from DG Agri Third country markets concern mainly North America with 68% of the programmes (10 out of 17). Other markets targeted (once or twice) are China, India, Japan, Russia, Switzerland and South East Asia, Middle East It would be interesting to compare the EAT campaign, that began in 2005, with similar promotion programmes financed in the same countries (USA, Japan, China) in order to identify potential complementarities, gaps or overlaps. This detailed information is not available. 65 The count is based on the number of contracts signed. One count thus corresponds to a programme lasting during (at most) the three following years. Final Report November 2011 Page 127 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC From the framework contract evaluations, synergies were clearly identified for China, and a lack of complementarities for the USA between the EAT campaign and co-funded programmes. Currently, an Italian programme whose message is based on the EAT acronym, for “European Art of Taste” is being implemented and shows a willingness to build synergies. Complementarities between the EU information campaign on organic food and farming and organic co-funded programmes Since 2004, 24 co-funded programmes have been implemented on organic products, of which 19 are on the internal market and five target third country markets (Table 41). The Commission initiative took place from 2006-2009 with the website available since 2008. Table 41: Number of co-funded programmes on organic food and farming implemented on the internal market and on third country markets Year Internal Market 2004 2005 2006 2007 2008 2009 2010 Total 3 4 4 4 1 1 2 19 Third Country market 0 0 2 1 1 1 0 5 Total 3 4 6 5 2 2 2 24 Source: ADE, based on data from DG Agri In the case that co-funded programmes took place during or after the Commissions’ organic campaign, there have been cases of synergies. Interviewees in Spain, Italy and the Netherlands stated that the material developed by the Commission initiative made it possible for other organisations in the sector to carry out campaigns very efficiently. J.3.3 High level trade visits facilitate and should enhance business opportunities High level trade missions often involve a Commissioner, accompanied by a delegation of representatives of the European agri-food sector. They also include participations in international events, mainly trade fairs. These activities aim at the promotion of commercial relations by enlarging existing markets or opening up new markets. Participation in international events and fairs intend to present a positive European image of agricultural products facilitating further relations among national and private stakeholders. They should facilitate and enhance increased business opportunities for EU agricultural product exporters. On average, the EC organised one high level trade visit per year since 2004. Evaluation reports have been made by the agencies in charge of the organisation of these missions. Final Report November 2011 Page 128 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The main destination for high level missions was China, with 5 out 8 missions. One concerned India and another South-Africa, the latter at the occasion of the World Meat Congress. One promotional event was organised in Russia in 2010 with a stand presence at World Food Moscow trade fair. The objectives of these missions, as they are described in the evaluation reports, are in line with the objectives of the promotion policy. The visits aim at “promoting EU products and image”, “increasing the awareness about European products and quality”, and “increasing visibility” during international events. Some visits explicitly mentioned the goal of “creating business opportunities”. The visits are organised by the promotion unit, in cooperation with other units of DG Agri. The choice of themes and products addressed and promoted during the visit are based on political priorities and the context of the event. The participants are chosen by the EC on a case-by-case basis: depending on the type of visit, the participation can be open to different types of stakeholders or restricted. For this year’s visit in China, a website provides all detailed information about events, participants and development of the mission. Some stakeholders (competent bodies and proposing organisations) felt that they were excluded or not concerned by visits because their Member States were less involved in European Quality schemes. They questioned whether Commission initiatives in third countries should have such a strong focus on quality schemes. The principle of these visits and their themes appear relevant, promoting EU agriculture in general with a focus depending on the theme of the event. The missions generally ended with a satisfaction survey among participants. The results of these surveys were generally quite positive. However, no follow-ups are performed after the mission (e.g. 6 months after the visit), in order to assess the impacts of the missions in creating new business opportunities. The Table hereafter presents the visits undertaken since 2004, together with participation in promotional events. Final Report November 2011 Page 129 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 42: Year High level trade missions and promotional events implemented at the Commission’s initiative since 2000 Location 2011 China Shanghai & Beijing 2010 Russia Moscow 2009 2009 Themes & Products GI products General (PDO+, organic, F&V, wine & spirits, olive & oil, meat, dairy) Event World Food Fair 50 representatives of producers and producer organisations from 15 MS Commissioner + 11 businessmen from 8 MS from the main agro-food China General (wine, SIAL Shanghai sectors (About 15 Community Shanghai & spirits, olives & food and drinks exporters at the EU stand in Beijing oil, meat, dairy…) fair Shanghai) Working group made up of all trade China - Hong General, and in HOFEX trade representatives from EU MS Kong particular GIs fair pressent on the territory 2007 2004 China 2008 Actions Commisionner + Business delegation of 32 participants from European week - 15 MS mainly wine and spirits; Various conferences, workshops and events on geographical cheese; meat and delicatessen; olive PDO, PGI and TSG (see http://www.gi-visitorigin oil press.eu) South Africa - Meat (quality and 17th World Cape Town safety of EU meat) Meat Congress SIAL Shanghai General (PDO+, food and drinks organic, wine & fair, and China spirits, dairy, "European biscuits, pasta) Shanghai Weeks" General (PDO+, organic, F&V, India - New wine & spirits, Delhi & olives & oil, meat, Mumbai dairy) Aahar trade fair 2008 Participants Stand at the World Food Fair, conference, promotional retail actions in outlets First promotional part in Shanghai followed by a participation of the EU in a stand in the 10th edition of the SIAL Shanghai food and drinks fair, inaugurated by Fischer Boel & Political dialogue in Beijing EU stand and several events (opening cocktail, seminar on GIs and "European Delicacies festival") Commissioner EU stand, cocktail, meeting with minister Commissioner, EU delegation, working group of trade advisors from 14 MS, local importers and distributors of EU food Stand at the SIAL Shanghai food and drinks fair, seminar on EU quality labels within the fair, and "European Weeks" promoting Community products Commissioner, delegation of 28 representatives from EU food and drinks companies from 15 MS representing most of the sectors Trade mission led by the Commissioner Participation in the trade fair, press conference, buffets, visits to villages & universities, and seminars Source: ADE based on EC documentation Final Report November 2011 Page 130 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC To what extent have the information and promotion measures implemented at the Commission’s initiative been relevant to achieve the objectives laid down in the regulation? Evaluative Judgement The concept of Commission initiatives is relevant in the context of the promotion policy which depends on the initiatives of trade and intertrade organisations. Indeed, the EC can launch initiatives on themes and products that are of Community interest which are not undertaken by proposing organisations. Furthermore, the Commission may undertake campaigns at EU level which concern all 27 Member States. Commission initiatives have mostly concentrated on the organic sector and on PDO, PGI and TSG products. The themes covered are of Community interest and correspond to the main priorities of the Regulation. On the internal market, the Commission launched a promotion campaign on the organic sector (2005-2009) that was coherent and linked with the European Action Plan for organic food and farming. This promotion campaign has provided good results and impacted positively on other promotion campaigns which can reuse its promotion material. On the third country markets, the EC launched the EAT (European Authentic Taste) campaign covering PDO, PGI, TSG and organic farming on three third country markets (USA, Japan and China) from 2005 to 2007. The evaluative judgement on this campaign is restricted by the limited amount of information available on the campaign. Few of the programme documents referred to the campaign and views were mixed. Real synergies with promotion programmes were observed in China, but the USA campaign missed some opportunities, according to the framework contract evaluation (2006). Northern European MS felt excluded because the programme included very few products from their countries. There was limited complementary with co-funded promotion programmes. However, a current Italian programme reused and adapted the name EAT (European Art of Taste) for promoting their products on third country markets, which demonstrates concrete synergies. High level trade visits aim to value a positive European image of agricultural products and facilitate further relations for national stakeholders and enhanced business opportunities for exporters. These visits have been conducted mainly in China. The principle of the visits and their themes and destination appear relevant regarding export opportunities. The immediate follow-up of the visits provided generally positive feedback. However, an assessment of the business opportunities created would require performing evaluations after a longer delay. To conclude, the Commission initiatives are relevant to promote EU agricultural products, both on the internal market and in third country markets. They can act as a lever for the promotion of EU agricultural products. To do so, coordination (through effective communication) between EC initiatives and co-financed programmes or future programmes is an issue at stake. Final Report November 2011 Page 131 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 4.2.4 Evaluation Question 4 To what extent has the EU promotion policy framework been conducive to encouraging multi-product and multi-country programmes? Rationale and approach The question aims at verifying the extent to which Council Regulation (EC) No 3/2008 and Commission Regulation (EC) No 501/2008 encourage and support Member States in implementing programmes promoting more than one agricultural product, programmes put forward by more than one Member State and programmes involving measures in more than one Member State or in more than one third country. Specific attention is paid to the extent to which the Regulations cover the three types of programme, the extent to which the stakeholders (both proposing organisations and competent bodies) are satisfied with the guidance and support given by the Regulations and the Commission and the extent to which the design of multi-product and multi-country programmes presents a feasible and coherent strategy for integrating different national elements into one programme. The answer to the question distinguishes findings related to programmes implemented on both the internal and third country markets. In a final section the advantages and disadvantages of multi-product and multi-country programmes are highlighted. The judgement criteria (Table below) seek to address the extent to which the guidelines have promoted and the Member States have implemented multi-product and multi-country programmes and to highlight the advantages and disadvantages of multi-product and/or multi-country programmes as perceived in MSs. The main limitation in the data is that it does not highlight the contents of the multiproduct or multi-country programmes but concentrates on the extent to which the Regulations support such programmes, the guidance provided by the Commission, and whether such programmes are considered useful. The information does not give much insight into the integration of national elements into one strategy, e.g. cultural similarities or differences, national policies, funds available for such promotion, existence of promotion of agricultural products by the private sector, etc. Table 43: J.4.1 I.4.1.1 I.4.1.2 I.4.1.3 Final Report EQ 4 Judgment criteria and indicators Member States have increasingly implemented programmes promoting more than one agricultural product The Regulations explicitly refer to programmes promoting Regulations, more than one agricultural product and provide guidance and guidelines incentives for implementing such programmes on both internal and third country markets EU co-funded programmes promoting more than one Expenditure data agricultural product have been implemented on both internal and third country markets Stakeholders are satisfied with EU guidance and incentives to Questionnaire, Key implement programmes promoting more than one agricultural informant interviews product on both internal and third country markets November 2011 Page 132 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC J.4.2 I.4.2.1 I.4.2.2 I.4.2.3 J.4.3 I.4.3.1 I.4.3.2 I.4.3.3 J.4.4 I.4.4.1 Member States have increasingly implemented programmes including more than one Member State as beneficiary of EU co-funding The Regulations explicitly refer to programmes including more Regulations, than one Member State as beneficiary of EU co-funding and guidelines provide incentives for implementation of such programmes EU co-funded programmes including more than one Member Expenditure data State have been implemented on both internal and third country markets Stakeholders are satisfied with EU guidance and incentives for Key informant implement programmes including more than one Member interviews, State as beneficiary of the EU co-funding on both internal and questionnaire third country markets Member States have implemented programmes including measures in more than one Member State or in more than one third country The Regulations explicitly refer to programmes including Regulations, measures in more than one Member State or in more than one guidelines third country and provide incentives to implement such programmes EU co-funded programmes including measures in more than Expenditure data one Member State or in more than one third country have been implemented on both internal and third country markets Stakeholders are satisfied with EU guidance and incentives for Key informant implementing programmes including measures in more than interviews, one Member State or in more than one third country questionnaire Advantages of multi-product and/or multi-country programmes are perceived by stakeholders Stakeholders are able to identify specific aspects/ elements Key informant which favour implementation of multi-product and/or multi- interviews, country programmes questionnaire Detailed Response J.4.1 Programmes promoting more than one agricultural product Regulations Commission Regulation (EC) No 501/2008, Art. 10(1) states that in programmes aimed at third countries, preference is given to those covering a group of products and placing particular emphasis on quality, nutritional value and food safety aspects of EU production. This is the only reference in the Regulations to preference being given to multi-product programmes. There is no extra financial incentive to have multi-product programmes. Final Report November 2011 Page 133 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Occurrence of multi-product programmes According to DG AGRI expenditure data, in the case of the 2008-2010 programmes, out of 100 effectively contracted66 programmes, 4 (4%) were multi-product. Out of the four, two were on the internal market and two on third country markets. In the case of the 2001-2007 programmes, out of 336 programmes, 28 (8%) were multiproduct. This means that the number of multi-product programmes has actually decreased by half between the two periods. Around 70% of multiproduct programmes concern third country markets. Almost 20% are also multi-country programmes. Competent bodies see a value added in multi-product programmes According to the online survey, over 68% of the respondents saw a value added in multiproduct programmes. Multi-product programmes are seen as a good tool for the promotion of the image, knowledge and the increased consumption of a basket of products. They can lower costs and have synergetic effects. Some respondents consider them only useful for specific combinations of products only. Satisfaction with guidance provided by the European Commission for multiproduct programmes Regarding levels of satisfaction with the guidance provided by the European Commission as far as multi-product programmes are concerned, from the online survey, altogether 57% of the respondent competent bodies are satisfied to some or to a large extent with EC guidance and incentives to implement multi-product programmes on both internal and third country markets. The information from the interviews corroborates that the majority of competent bodies and proposing organisations were satisfied with the EC guidance and incentives to implement programmes promoting more than one agricultural product on both internal and third country markets, even those who did not choose to implement such programmes. Advantages of multi-product programmes came from their larger size (in financial terms) which allowed an increase of their potential impact, by reaching wider target groups. 66 The document distinguishes between total number of effectively contracted programmes and total number of contracted programmes including those abandoned. We use the number of effectively contracted programmes. Final Report November 2011 Page 134 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC J.4.2 Programmes including more than one proposing Member State as beneficiary of EU co-funding Regulations Council Regulation (EC) No 3/2008, Art 8 states that “Priority shall be given to the programmes proposed by several Member States or providing for measures in several Member States or third countries”. The EC’s preference for programmes put forward by more than one Member State is clear, both in the Regulations and to the stakeholders in all Member States. There is also a financial incentive to carry out multi-country programmes. The fees of the implementing body/bodies should be no more than 13% of the actual cost of implementing the actions for programmes proposed by one Member State and 15% for programmes proposed by more than one Member State67. Moreover, additional 2% of overhead costs are allowed for multi-country programmes. Occurrence of programmes including more than one proposing Member State 11 of the 100 programmes effectively contracted in 2008-2010 are put forward by more than one Member State. Eight of these programmes promote products on the internal market and three promote products in third countries. Of these multi-country programmes, one is also a multi-product programme on the internal market. In comparison, 30 (9%) of the 336 programmes effectively contracted in 2001-2007 were put forward by more than one Member State. This means that there has been a slight increase in the percentage of programmes put forward by more than one Member State in relation to the period 2001-2007. Competent bodies also see a added value in added in multi-country programmes In the online survey competent bodies purported to see added value in multi-country programmes. In view of the low occurrence of multi-country programmes, the perception of their added value in the online survey is surprisingly high68. Satisfaction with guidance by the Commission for multi-country programmes The following table based on the online questionnaire shows that the majority of respondent competent bodies are satisfied with the guidance and incentives for implementing programmes including more than one Member State as beneficiary of the EU co-funding on both internal and third country markets. Only three responses said not at all and four to a very limited extent. Overall, although the EU guidance in the Regulations is rather limited, it appears to be enough. 67 Application form for promotion programmes part-financed by the EU, ref : ARES(2009)347563-27/11/2009 68 24 respondents out of 35 (68%) said that they saw a value in multi-country programmes. Five (14%) said they did not the remaining 6 (17%) did not respond Final Report November 2011 Page 135 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 44: Satisfaction with guidance provided by the Commission for multicountry programmes This evidence is corroborated by the interview results. Most interviewees were satisfied with the guidance given by the EU but this did not automatically mean that they were convinced of the usefulness of the multi-country programmes. The overall picture is that the Member States were sceptical of the real added value of submitting programmes by more than one Member State due to the high administrative burden and the lack of conviction that the multi-country approach was more effective than the single country approach. The high perceived costs of multi-country programmes were seen to stem from the financial and human resources needed to prepare and administrate the programmes: these include identification and contact with the partners in other Member States, gathering market information about the country in question, meetings, translation of documents, waiting times for things to be agreed (as Member States may have different interpretations of the Regulation and therefore different requirements, adapting processes to different administrative procedures. The effort involved is perceived to be greater the more Member States are involved. Interviewees from 8 of the 9 visited MSs stated that the multi-country programmes were sometimes submitted to raise the chances of getting EC funding rather than because the cooperation raised the intrinsic value of the information and promotion campaign. The overall opinion of the competent bodies and proposing organisations in all Member States is that multi-country programmes can have added value in terms of exchange of experience, synergies and leverage effect69, but that they are very difficult and expensive to manage. The additional 2% of overall costs for overheads or for the implementation of the measures for programmes proposed by more than one Member State were not considered a sufficient incentive. However, in the case of the FR-BIV programme, the PO felt that the 69 For example, a multi-country programme involved the EC, 3 EU Member States and 4 Proposing organisations, the 4 PO sharing 20% of the costs. Final Report November 2011 Page 136 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC benefits outweighed the administrative burden. This is corroborated by the proposing organisation in the FR-INT programme. A further hindrance is that the agri-businesses involved in the programmes ultimately want to open up new markets for their products and do not consider it effective to campaign with their potential competitors in other Member States. Some Member States felt that this preference for multi-country programmes showed a lack of understanding of the situation on the ground where producers or sector associations often do not have counterparts in other countries and where the time required finding partners and establishing multi-country proposals is too long. Identifying appropriate partners was one of the most frequent highlighted hindrances to multi-country programmes. A further hindrance was the administrative effort needed both to prepare and to run multi-country programmes. The preparation of such programmes takes a lot longer and costs more on account of the need to translate documents, coordinate with others, etc. These costs accrue to the proposing organisations and although the chances of being accepted are perceived to be greater, the PO still carries the risk of no return on investment. The perceived risk is increased by the potential dropping out of a partner from another Member State which is a real danger to the success of the proposal. J.4.3 Programmes including measures in more than one Member State or in more than one third country Regulations Council Regulation (EC) No 3/2008 emphasise that the results of the programmes so far have been very encouraging and that the promotion of certain EU agricultural products on EU and third country markets should be continued. Council Regulation (EC) No 3/2008, Art 8 states that “Priority shall be given to the programmes proposed by several Member States or providing for measures in several Member States or third countries”. Occurrence of programmes promoting measures in more than one Member State or in more than one third country 26 (26%) of 100 programmes running between 2008 and 2010 promote measures in more than one Member State or in more than one third country. 9 (9%) promote measures in more than one third country and 17 (17%) promote measures in more than one Member State on the internal market. In comparison, 74 (74%) programmes target only one Member State or third country. In comparison, 91 (27%) of the programmes in the period 2001-2007 were targeted at more than one country. The proportion of programmes targeted at more than one country has therefore remained consistent. Final Report November 2011 Page 137 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Satisfaction with guidance by the Commission for programmes promoting measures in more than country Generally, the stakeholders (both proposing organisations and competent bodies) are satisfied with the guidance provided for the implementation of programmes in more than one Member State or third country. Usually, the proposing organisations are familiar with these markets and/or hire implementing bodies which are. The main criticisms regarding the EU guidance and incentives for implementing programmes including measures in more than one Member State or more than one third country concern the restrictions on using brand names or details of origin except in the case of the PDO, PGI, and TSG. This makes it difficult to find private co-financing for the measures. J.4.4 Advantages and disadvantages of multi-product and/or multi-country programmes The following table summarises the identified advantages and disadvantages of the multiproduct and multi-country programmes with competent bodies and interviewed proposing organisations. Table 45: Type of programme Multi-product (promoting more than one agricultural product) Advantages and disadvantages of the multi-product, multi-country and multi-target programmes Advantages Disadvantages Synergies between products leads to greater potential impacts A good combination of the products increases the impact of the promotional programme). Baskets of products are an attractive offer. Possibility of reaching wider target groups On account of their larger size and reach, such programmes potentially allow the reaching of larger and more diversified target groups. Difficulty of conveying different messages for different products in multi-product programmes Promoting different agricultural products through the same co-funded programme is challenging as the messages are not the same from one product to another, especially as the magazines and the trade press that promote one type of product are different from those that promote another. Difficulty of coordinating different seasonality of products Promotion activities are more difficult to plan as the agricultural products do not have the same seasonality. Possibility to promote intrinsic values of European quality products To convey for all categories of Final Report November 2011 Page 138 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Type of programme Advantages products the same messages related to the intrinsic values of European quality production and to promote the ‘Made in Europe’ brand and indirectly the brand of each country. Disadvantages Potential lack of recognisability of individual products The products may be less recognisable if promoted alongside other products, especially if these are not really compatible, e.g. linking milk with other products would dilute the message on milk. Different distribution channels for different products It is also not always straightforward to define the trade and distribution channels for the different kind of products. Increased chances of being accepted One of the main advantages of multicountry or multi-product programmes is the increased probability of being selected for co-funding. Competent bodies mentioned programmes which were only turned into multi-product and multicountry programmes for this reason. Multi-country (put forward by more than one Member State) Final Report November 2011 Non-eligibility of promoting brands The fact that brands cannot be promoted in the framework of the EU co-funded programmes is considered a disadvantage of the whole regulation. The promotion of private brands would be particularly necessary in third countries because it is a precondition for opening new markets. In the view of the MSs, this would also benefit European companies not covered by the promotion campaigns. High administrative burden Taking into consideration the various viewpoints, it appears that there are more difficulties in designing multi-country programmes than single-country programmes. The administrative burden of getting approval from all the MS competent bodies is considerable. The high level of detail required in the application forms for all proposing organisations doubles or triples the workload. There is also a danger that the competent body in the other Member State may pull out at the last minute. Also, the more producers from different countries are involved, the more difficult the preparation of the programme and activities. In some cases contacts have to be established which is very time consuming and there is a lack of experience in managing multi-country programmes. Page 139 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Type of programme Advantages Disadvantages Increased resources Multi-country programmes usually have a higher budget thus allowing organisations to undertake more actions in more countries and penetrate markets in a more effective way (increasing the acknowledgement of consumers, importers, retailers). This may entail lower costs for individual participants. They definitely promote actions which would not be possible otherwise. Greater impact is expected as a result Increased costs The costs of designing (coordination, translation of the documents, agreement on the programme in detail, etc.) and managing a multi-country programme are higher than those of managing a single country programme. In certain cases, it therefore appears that multi-country programmes could be less efficient than single-country programmes. Cooperation and exchange of experience Multi-country programmes encourage the exchange of experience and synergies as well as sharing market information, regrouping strengths of several MS to attack markets, economies of scale thanks to common actions. Newcomers in the information and promotion trade benefit from the know-how of the more experienced members of the partnership. Increased competitiveness Multi-country generic programmes and those aiming at increasing competitiveness in relation to the new producer countries may raise the awareness for EU products and convey for all categories of products the same messages related to the intrinsic values of European quality production. Final Report November 2011 Cultural and regulatory differences Multi-country programmes have to tackle specificities in terms of content (cultural differences i.e. in food habits), national regulations (on health, access to schools, etc.) and efficiency (increased costs rather than economies of scale). There are also differences in levels of expertise and technical standards between Member States. There are also differences in terms of interpretation of the EC Regulation. Non-eligibility of celebrity campaigns Both the British and the German proposing organisations considered it a disadvantage that the use of celebrity campaigns was not allowed by the current regulations. They felt that the use of celebrity based campaigns could open up opportunities for a multi-country approach, since many of the celebrities have a strong identity across the whole EU. Competition between producers At the end of the day, the aim of the producers involved in promotional programmes is to promote (and sell) their products on new markets. Promoting the same product from different countries in a third country (e.g. the apple) is ultimately not considered effective as it may entail high competition between the producers on the same market. Page 140 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Multi-target (involving measures70 in more than one Member State or in more than one third country) New markets and economies of scale The co-funding provides the possibility to create new demand on new markets and to ensure a wider presence in the selected markets. It also allows the opening up of new markets to private companies. There are advantages in terms of visibility if different producers from different Member States take part in a fair in third countries. All of them provide different products that altogether carry the image of European products. Differences in the messages in multitarget programmes Programmes promoting products in different countries are faced with the challenge of adapting the communication regarding the products to the different characteristics of each country. Regulatory restrictions The German competent body would like to see both third country and EU markets being the object of individual campaigns (e.g. Austria and Switzerland for German products). To sum up, the multi-product, multi-country and multi-target programmes were seen to have advantages in terms of critical mass and synergies but distinct disadvantages in terms of administrative burden and logistics. In particular, multi-country programmes were unpopular as they demanded a lot of preparation and were of uncertain benefit. Multitarget programmes were less problematic as targeting more than one country is common practice in marketing campaigns. To what extent has the EU promotion policy framework been conducive to encouraging multi-product and multi-country programmes? Evaluative Judgement Programmes promoting more than one agricultural product: only 4% of the programmes (2008-2010) are concerned and the share has decreased by half compared to 2001-2007 (8%). In the case of programmes targeting third countries, the Regulation gives preference to those covering a group of products. Guidance given by the EC to promote multi-product programmes is considered appropriate. Most surveyed competent bodies see a value added in multi-product programmes. They have a larger potential impact on the image and the possibility of reaching wider target groups. They are appropriate for some specific combinations of products (e.g. basket of products, combination of products as cheese and wine). However, some stakeholders (proposing organisations and some competent bodies) are not convinced by the increased effectiveness of multi-product versus single product programmes (conveying different messages, diluting the message, lack of recognisability of individual products, they may have a different seasonality, different channels). They see the preference given to multi-product programmes as counter-productive as some proposing organisations put forward multi-product programmes more in the hope of receiving co-funding than because it raises the intrinsic value or impact of the information and promotion measures. Programmes put forward by more than one Member State (genuine “multi-country programmes”): 11% of the programmes being implemented in 2008-2010 are concerned. 70 As defined by Council Regulation (EC) No 3/2008, they are measures that promote and provide information on agricultural products and their method of production, as well as food products. These measures must not be brand-oriented. Final Report November 2011 Page 141 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC There is a slight increase in share compared to 2001-2007 (9%). The Regulation gives clear preference to these programmes including financial incentives. The priority given to these programmes has not had much success on the programmes submitted in spite of efforts by some proposing organisations to submit multi-country programmes in order to raise their chances of being selected. Multi-country programmes demand considerable administrative effort and know-how of other Member States’ systems and cost more to prepare and manage than single-country programmes. Therefore, there is a lack of enthusiasm for these types of programmes among the majority of proposing organisations. A large part of surveyed competent bodies purported to see a value added in these programmes. Programmes involving measures in more than one Member State or in more than one third country (“multi-target programmes”): 26% of the programmes being implemented in 2008-2010 are concerned. Generally, the guidance provided by the EC for the implementation of such programmes is considered appropriate. Usually, the proposing organisations and the implementing bodies are familiar with these markets and are aware of which ones are good targets. The multi-product and multi-country programmes were perceived to have advantages in terms of critical mass and synergies but also disadvantages in terms of administrative burden and logistics. In particular, programmes involving more than one proposing Member State were considered difficult to implement as they represent a huge administrative burden. Programmes targeting more than one country were seen as less problematic as targeting more than one country is common practice in marketing campaigns. In conclusion, although the EU promotion policy framework has encouraged all types of multi-programmes, they do not have the expected occurrence. The multi-product programmes are rather marginal. Compared to single product programmes, they allow reaching wider target groups with a larger potential impact. However, certain products (PDO, PGI, wines, cheese, olive oil...) lend themselves better to this approach than the others (milk). Programmes involving measures in more than one Member State or in more than one third country have the advantages of entering new markets and/or providing economies of scales and a leverage effect. The programmes put forward by more than one Member State favour the cooperation and exchange of experience. These latter programmes, by promoting the cooperation between Member States, appear to be highly relevant regarding EU global objectives in the framework of the CAP. The difficulties in implementing such programmes (cultural, languages, coordination ...) are a reason of their current limited occurrence and require a specific support from the Commission. Final Report November 2011 Page 142 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 3.2.5 Evaluation Question 5 To what extent has the effectiveness of the programmes implemented since 2008 been improved in terms of achieving the objectives laid down in the regulation? Rationale and approach The question addresses the extent to which the experience gained up to and including the framework contract evaluations has led to improved effectiveness in achieving the objectives of the Regulation (i.e. improved knowledge, enhanced image and increased consumption). The focus is on the extent to which good practices identified in the framework contract evaluations have been used to improve the design and implementation of subsequent programmes. The evidence was gathered primarily from documents and interviews about the sample of 15 programmes, supplemented with more general discussions with competent bodies for the Member States in which the sample of programmes took place. The EQ is answered in two parts, the first dealing specifically with the use of good practices and the second with the performance of programmes since 2008. The analysis of good practices distinguishes between: a) those associated with the design of programmes; and b) those associated with implementation. In each case, the analysis considers awareness of the good practices, use of the practices and impact of that use on attainment of objectives. The analysis of performance since 2008 considers: a) whether awareness and image has improved (both on the basis of objective evidence and the views of interviewees); b) whether Member States are intending to follow-up EC programmes with their own; and c) whether demand and sales have improved. The change in awareness, demand and sales has been assessed based on the evidence in the documents, with additional interpretation provided during interviews. In some cases, proposing organisations and competent bodies have been asked for additional information from their working documents. No primary data collection or survey work has been done. The assessment of MS intentions regarding possible follow-up funding was based on interviews with competent bodies. It is not straightforward to assess the impact of the good practice guidelines. In many cases, the majority of good practices were followed, although there were no cases where all the good practices were followed. Although it was rarely explicitly recognised, most of the identified good practices are now included in EC guidance documents. Nevertheless, it appears that there is a growing body of lessons across the whole range of institutions involved and that the guidelines on good practice provide an important reference point, even if they are not always the primary source of lessons learnt. Programme documents were examined for evidence from surveys and statistics. However, as many of the programmes have only been operating for a year, there is limited evidence of the achievement of objectives. As a result, the main evidence comes from interviewees’ opinions about the importance of the lessons learnt from the 2008 evaluations. There were several situations where the interpretation of questions was imprecise. In particular, many of the programme documents and interviewees were positive about the Final Report November 2011 Page 143 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC good practices, but it was not always possible to determine whether this was because they simply agreed with the practices or because they had read the 2008 evaluations. In addition, it was also not always clear whether the pursuit of good practices had changed since 2008 and, if so, whether this was due to exposure to the lessons from the evaluation. Regarding the improved image in J.5.3, it was not always clear whether this was referring to recognition of the logo, or of the EU’s image more generally, or of the specific image of the products or themes being promoted. Finally, the I&P programmes work on various themes around awareness raising and the information on EU agricultural products. Supporting the consumption is only the global objective of the regulation, to which these programmes contribute. All the more, it is neither easy nor immediate to establish a link between the programmes and the changes at the level of the consumption of products. This fact had already been underlined by the sector-based evaluations of the framework contract evaluations. Table 46: EQ 5 Judgment criteria and indicators J.5.1 The quality of the design of the co-funded programmes has been improved between the period 2002-2008 and that from 2008 onwards I.5.1.1 Good practices highlighted in the evaluation reports covering Evaluations, recent programme 2002-2008 co-funded programmes have been taken into account in applications and annual reports, improving the design of the programmes (2008 onwards) minutes of Advisory Groups, key informant interviews I.5.1.2 Stakeholders are aware of good practices for designing co-funded Key informant interviews, programmes and have integrated them when designing the programme questionnaire proposals I.5.1.3 Stakeholders consider that good practices highlighted in the Key informant interviews, evaluation reports covering 2002-2008 co-funded programmes have questionnaire helped enhance attainment of the objectives of their programmes and of the Regulation J.5.2 Implementation of the co-funded programmes has been improved between the period 2002-2008 and that from 2008 onwards I.5.2.1 Good practices highlighted in the evaluation reports covering Evaluations, recent programme 2002-2008 co-funded programmes have been taken into account in applications and annual reports, implementation of the programmes (2008 onwards) minutes of Advisory Groups, key informant interviews, questionnaire I.5.2.2 Stakeholders are aware of good practices for implementing co- Key informant interviews, funded programmes and have integrated them when implementing the questionnaire co-funded programmes I.5.2.3 Stakeholders consider that good practices highlighted in the Key informant interviews, evaluation reports covering 2002-2008 co-funded programmes have questionnaire helped enhance attainment of the objectives of their programmes and of the Regulation J.5.3 The co-funded programmes implemented since 2008 have attained their objectives and those of the Regulation I.5.3.1 The level of knowledge and the image of EU agricultural products Post 2008 evaluation reports have improved as a consequence of the co-funded programmes I.5.3.2 The co-funded programmes have impacted on the demand for Post 2008 evaluation reports EU agricultural products Final Report November 2011 Page 144 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC I.5.3.3 Member State schemes to promote agricultural products have been supplemented and reinforced as a consequence of the co-funded programmes I.5.3.4 Stakeholders consider that demand for and sales of EU agricultural products on internal and third country markets have increased or diversified as a consequence of the co-funded programmes Documents for Member State programmes, key informant interviews Key informant interviews, questionnaire Source: ADE. Detailed Response The main good practices were identified in the recommendations of the nine previous framework contract evaluations (see also § 3.8 of the comprehensive synthesis) and are listed hereafter. Most of them have been included in EC guidance through application form models. In terms of design: Include a description of the market situation justifying the reasons of the proposal; Define quantitative (SMART) objectives with clear intended results and impact and appropriate monitoring and evaluation tools; Describe the appropriate strategy; Define the target population(s); There are no given recipes of actions to undertake. Programmes should consider a mix of actions and messages in accordance with the market description; Carefully describe the actions as well as the corresponding budget; Define clear efficiency criteria to select the most appropriate communication channels; In third countries, work directly with regional and local partners (implementing agencies) to design information and promotion campaigns; and In third countries, focus mainly on professionals with direct contacts through fairs, exhibitions, tasting and other public relation activities, carefully selected to reach the appropriate professionals. Look for complementarities/synergies with other European products. Eventually complement this with promotion activities to consumers when promoted products are known on the targeted market In terms of implementation of programmes: Use of competitive tendering for selecting implementing agencies; Using a central purchasing agency for media space; Using the results of monitoring and evaluation to revise the programme; In third countries, work with local partners for third country programmes. Final Report November 2011 Page 145 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC J.5.1 Good practice for programme design The present judgment criterion assesses the use of these good practices and their appreciation by competent bodies and POs in the design of the 15 sample programmes of the current period (2008-2010). Describe the market situation justifying the reasons of the proposal. Ten out of the fifteen sample programmes included a market analysis and interviewees were aware that this is required as part of the standard contact of applications. Only five did not include such market analysis (PL-SRW, PT-, PT-FEN, DE-TAG, EL-SEK). The analysis generally includes information about sales trends and some breakdown for target groups. Define quantitative (SMART) objectives with clear intended results and impact and appropriate monitoring and evaluation tools. The objectives described in eight of the programmes could be defined as SMART (i.e. all programmes excluding the four that did not recognise good practice and ES-OIA, FR-INT and IT-INT). Most applications included some tools for monitoring and evaluating results and impact, but two did not. The level of detail varied, with six programmes having detailed tools with some form of dedicated tracking survey. Two of the programmes provided objectives but failed to provide quantitative indicators of achievement. In one case, the quantification of results was limited to the outputs arising from actions and the objectives (in this case, improving knowledge) were not quantified or monitored. One of the competent bodies (UK) reported that the need to define and monitor results has always been clear and that the good practice from the framework contract evaluations has not changed this much. One of the programmes (IT-UIV) defined a composite index that combined output measures for all actions and provided a score of achievement for the whole programme. Describe the appropriate strategy. Nine of the programmes presented a strategy (i.e. all programmes excluding the four that did not recognise good practice and EL-SEK and EL-EAS).Two of these were reported as being detailed and based on a context analysis. One proposing organisation suggested that programmes should be required to define a clearer strategy using SWOT analysis and the market analysis. Define the target population(s). All of the programmes that recognise good practices had clearly defined target groups. However, one competent body stated that the definition of target groups had always been clear, even before the good practices identified in 2008. There are no given recipes of actions to undertake. Programmes should consider a mix of actions and messages in accordance with the market description. All the interviewees of the 15 sample programmes that were aware of good practices reported applying this principle and considered that it helped attainment of objectives. The range of actions were very wide across the programmes, reflecting the wide range of circumstances for different countries, products/themes, target groups and stage of development in market structure and marketing activity. The actions are generally well developed and are in accordance with the market description. Two of the programmes (FR-BIV and ITINT) reported that the actions were described but that there was no justification and no link to the analysis of the market situation. Another programme reported that the reasons for including a more sophisticated range of activities were more associated with Final Report November 2011 Page 146 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC improved experience in competent bodies and POs than as a result of the good practice guidelines. Carefully describe the actions as well as the corresponding budget. This is a requirement for a successful application and there were no reports of programmes that had failed to describe activities and detailed budgets associated with each action. There were no examples where this did not take place. Two of the competent bodies (UK, PL) reported that the quality of definition of actions and budget had improved since 2008. Define clear efficiency criteria to select the most appropriate communication channels. The experience with efficiency criteria was mixed: five of the evaluations included adequate criteria, whilst four did not. One of the POs delegated responsibility for maximising efficiency to the implementing agency, without engaging actively in the process. Another PO commented that efficiency criteria cannot be measured for one action alone since many of the actions are interdependent. In this respect, most programmes selected the actions on the basis of cost effectiveness, defined by a variety of indicators and with varying levels of quantification. The need to consider quality of contact was rarely mentioned explicitly in the programme applications. Only one of the programmes indicated that the actions chosen were not necessarily the cheapest because they were expected to produce better quality results, with stronger multiplier effects. In third countries, work directly with regional and local partners (implementing agencies) to design information and promotion campaigns. There was some confusion about the interpretation of this practice, which was sometimes taken to imply wide engagement in the design and sometimes taken to refer to engagement of the implementing agency. Five programmes reported that the implementing agency was involved in the design and one competent body (UK) reported that this was now happening more often. In third counties, focus mainly on professionals with direct contacts through fairs, exhibitions, tasting and other public relation activities, carefully selected to reach the appropriate professionals. Look for complementarities/synergies with other European products. Eventually complement this with promotion activities to consumers when promoted products are known on the targeted market. Only three of the programmes worked primarily with professionals and all of these then followed up with some contacts with consumers. It seems likely that good practice that tries to provide advice on actions is problematic since the circumstances are so different from one programme to another. There is rather mixed evidence about the impact of good practices on attainment of objectives. Because most programmes agreed with the good practices, there was a positive feedback regarding their usefulness in attaining the objectives of the regulation. The Italian programmes were particularly positive about the importance of having SMART objectives. However, many of the programmes stated that the guidelines had little impact as design is based on growing experience with what works and the guidelines are used to satisfy the EC rules. Several programmes reported that the good practices were overly detailed and had become a burden. In general, it seems that POs refer carefully to the guidance provided. POs are generally aware of the good practices contained within the guidelines, but they are not aware of the origins of these good practices and whether they have been based on the framework Final Report November 2011 Page 147 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC contract evaluations. Indeed, few POs are aware that the evaluation had taken place. This may be partly because the guidance is spread out over many documents with different dates and sources, in addition to the Regulation and the application form. In most cases, the POs rely on the Regulation and application form for guidance and good practice and are not aware of the other sources. The Italian programmes provide an exception to this and most were aware of the evaluations and of the findings and recommendations. Some of the competent bodies (e.g. FR, UK, EL) provide continuous guidance and support to POs to try to make sure that proposals respect the guidance. However, the extent to which POs make use of the guidance varies. Few competent bodies or POs distinguish explicitly between the guidance that was available before and after the framework contract evaluations and are therefore unable to comment on the extent to which the lessons learnt have affected their proposals. Only one programme (FR-UGP) explicitly referred to the framework contract evaluations. Although most of the programmes reported that the design had taken into account the guidance, incorporating good practice, there were also four (FR-UGP, EL-SEK, EL-EAS, UK-MMF and IT-AIA) that reported that the primary basis for design decisions was their own experience on previous programmes. In some cases, the PO reported that it felt partly responsible for the good practice because the lessons learnt at EU level coincided closely with the lessons learnt from earlier programmes. The continuity of staffing in proposing organisations and competent bodies seems likely to make a large contribution to the steadily improving quality of programmes. One of competent bodies (EL) suggested that there was also a negative legacy from the 2002-2008 period, arising from the fact that POs tended to stick with the countries, regions and markets that they were already familiar with, which were not always those that would provide the biggest impact. J.5.2 Good practices for programme implementation The present judgment criterion assesses the use of these good practices and their appreciation by competent bodies and POs in implementation of the 15 sample programmes of the current period (2008-2010). Use of competitive tendering for selecting implementing agencies. Eleven of the programmes provided information on this and all of them used competitive tendering. Using a central purchasing agency for media space. There were no examples of a central agency operating across different programmes and the implementing agency was normally responsible for buying media space. In one programme, this task was sub-contracted to a specialised media buying agency. Using the results of monitoring and evaluation to revise the programme. Six of the programmes reported use of the monitoring to refine programmes, although some programmes achieved this through routine management monitoring that takes place weekly or monthly and so do not use the quarterly report. Nine of the programmes reported use Final Report November 2011 Page 148 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC of evaluation reports of their own programmes and two reported referring to evaluation reports of past programmes. Working with local partners for third country programmes. There are no reports of this. One of the programmes used SOPEXA, a French agency that was considered to have enough experience to operate without local partners. Two of the programmes reported that the quality of the implementing agency was critical in ensuring the successful implementation of programmes. This experience may arise from past involvement in EC I&P programmes, but a high quality agency with other relevant experience may also be effective. Ten of the programmes reported that the good practice on implementation helped to achieve objectives, largely through the use of active monitoring and evaluation. Two said that the requirements can sometimes be overly detailed, which deters the richer trade organisations. There was one suggestion that implementation of multi-country programmes would benefit from simplified reporting requirement and, in particular, from allowing a lead country to manage all reporting. J.5.3 Performance since 2008 Awareness and image The impact on awareness and image was generally positive. Five of the programmes reported a strong impact, although for two of these the primary impact was for the image of the product or theme, rather than of the EU. In one country (Greece) it was claimed that the EU image is already so widely respected that no further improvement is needed. Two of the programmes reported weak or no impact. Of the remaining eight programmes, five had not yet started generating results and three provided no evidence. For several programmes where results were not yet available or were not reported, it was claimed that there was evidence of the outputs from actions and that, based on past experience; it is possible to infer that there would be an impact on image. There was a suggestion that the EU image in third countries is very difficult to promote and the rules against revealing national identity are very restrictive. A major programme would be required to have any impact on creating an EU identity. While the documentary evidence was only partial, the opinions of POs and competent bodies were almost universal in claiming that projects either had improved the EU image, or would do so in the future, when the full impact of the programme was felt. One reported that this would only be achieved if actions were sustained. Another reported some confusion over the image of the logo for the outermost regions. There was a suggestion in two of the programmes, that the improvement in the image of EU quality was particularly strong for wines. For milk, the impact on EU image was limited because the image of milk as a quality product is already high. One of the programmes reported a strong impact on EU image amongst distributors, but thought it likely that the impact on consumers would take much longer to achieve. Final Report November 2011 Page 149 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Member State follow-up The prospects for national funding of information and promotion programmes are not good, except for third country promotion. This reflects the increasing pressure on national budgets. In some countries (e.g. UK and Denmark), there has been a longstanding political decision that the promotion of agricultural products should be left to the private sector, unless there is clear sign of market failure or clear health benefits. It is not clear whether these political opinions are gaining ground in other Member States. Germany, Greece, Portugal and the UK have no national funding and no plans to introduce it. In Poland, national funding is limited but may be stimulated by the EC programmes. In France and Italy, national funding is declining and becoming more difficult to obtain. In Italy, it is now limited largely to certified and organic products. Only in Spain are the political prospects for national funding considered good, with the national programmes generating demand for EU co-funded programmes and the experience with EC co-funded programmes stimulating national funding. In many countries, it is likely that the private sector will build on the EC co-funded programmes, but this will happen largely through brand promotion. Promotion activities of EU Member States in third countries is different as many MSs do support trade promotion. Agricultural products generally promoted as part of broader programmes, rather through dedicated programmes. There is likely to be good cooperation and complementarity amongst these programmes, but the nature and extent of this cooperation is fluid and subject to fast changing economic and political pressures. Demand and sales Most of the programmes reported that it was impossible to isolate the impact of the programmes on demand and sales. This is quite understandable, as most programmes work on the image/awareness and are generic by nature. Nevertheless, seven of the programmes (FR-BIV, FR-INT, PL-SRW, UK-MMF, IT-UIV, GR-SEK, and IT-INT) recorded positive trends in sales and concluded that it was likely that the promotion programmes influenced this trend. Two of the programme recorded declining sales of milk, but still considered that it might be possible that the rate of decline had been slowed by the programme. There were some programmes where the link from promotion to sales was possible. A few programmes were sufficiently large, compared to the market they were supporting, that it was possible to have some confidence of their impact on sales. For example, sales of specific products to some export markets could be monitored either through trade data (e.g. apples in IT-INT) or through corporate data (e.g. wine in IT-UIS) and sales from organic farms could also be monitored (e.g. IT-AIA). The UK-MMF programme operated in only part of the UK and could therefore compare sales with other parts of the country. Final Report November 2011 Page 150 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Where data on demand and sales was not available, POs in three of the programmes felt that there was strong subjective evidence to suggest an impact on sales. A fourth thought that there would be an impact on sales, but only if the relatively small EC co-funded programme was supplemented by a wider programme over a longer period. A fifth reported that there were clear signs of increased private sector interest in joining the programme, which was a reliable indicator of whether it was affecting sales. One programme dismissed the possibility of having a noticeable impact on sales because it was too small. One programme reported that, although it might not be possible to observe an impact on total sales, it should be possible to observe greater segmentation of the market, with more sales of specialist products. To what extent has the effectiveness of the programmes implemented since 2008 been improved in terms of achieving the objectives laid down in the regulation? Evaluative Judgement The judgement of improvement of programmes implemented since 2008 is based on the 15 sample programmes chosen for in depth analysis. However, as many of the programmes have only been operating for a year, there is limited evidence yet of the achievement of objectives. The question is answered in two parts, the first dealing specifically with the use of good practices which were identified in the recommendations of previous evaluation reports (i.e. good practices to design and implement programmes, listed above and in § 3.8 of the report) and the second with the performance of programmes since 2008. There is widespread appreciation of good practice amongst proposing organisations. The importance of market analysis, SMART objectives, clear targets, well adapted actions and good monitoring are all widely supported. This is reflected in an improving standard of selected proposals. However, until 2010, there has not been evidence of this in the rejection rates of programmes at EC level. The appreciation of good practice is obtained mainly from the experience of proposing organisations, from competent bodies and from the Regulation and the application forms for these programmes on DG Agri website. The additional guidance documents produced to reflect the lessons learnt in the framework contract evaluation are not known by most of the proposing organisations interviewed and by many of the competent bodies interviewed, although they are made available through the guidance provided for applicants. More specifically, although most good practices are generally applied, there was a lack of application of the use of efficiency criteria to select communication channels. Furthermore, there is little explicit recognition of the importance of assessing the quality of contacts. Regarding the choice of promotional partners in third countries, there is limited application by the analysed programmes of the good practice recommending to focus on professionals (and not consumers). Regarding improved performance since 2008, competent bodies and proposing organisations were positive that performance is improving, as a result of good practice, and Final Report November 2011 Page 151 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC that this is leading to better attainment of objectives. However, the link between good practices and improvement of programmes and other factors (e.g. accumulated know-how) are very important as well. The impact on image and awareness was quite mixed at the early stage of implementation for the 2008-2010 programmes, but about half the programmes had documentary evidence of a positive effect and almost all the stakeholders were confident that awareness was improving markedly. Regarding impacts, most programmes reported that it was impossible to isolate their influence on demand and sales. This is plausible as programmes mainly work on image and awareness for generic products. However, seven of the sample did have statistics showing an increase in sales and some of these could be attributed to the I&P programme. Where no quantitative evidence was available, stakeholders were generally positive that a positive impact could be inferred from the successful impact on awareness. To conclude, good practices are identified in the application forms available to applicants and largely applied by selected programmes. Stakeholders believe that the quality of proposals is improving, mainly because of growing experience, supported by the identification of good practices. However, the Commission still identifies significant problems in the quality of proposals. There are some good practices that are less well adopted, notably concerning the measurement of efficiency. Most stakeholders believe that the achievement of objectives is improving. The factual evidence for this is limited and mixed, but tends to support the view. 4.3 Replies to evaluation questions of theme 2 – Management of information and promotion programmes 4.3.1 Evaluation question 6 To what extent has the overall framework for managing promotion programmes, from the Commission, through Member States competent authorities, proposing organisations and implementing organisations, been appropriate to achieve the global objectives of the Regulation? Rationale and approach The question concerns the appropriateness of the management of I&P programmes at the level of the European Commission, competent national authorities, proposing organisations, as well as implementing bodies. The answer to this question investigates the structure and functioning of the management framework. Issues related to selection or management of information and promotion programmes are discussed. This question also analyses to which extent responsibilities at each level are clearly defined, as well as the repartition of tasks and responsibilities with identification of complementarities, gaps or overlaps. The questions of agenda and time management are also taken into consideration. Some specific elements related to management are also handled under evaluation questions 8 and 9; this last one focusing in details on selection procedures and implementation requirements. Final Report November 2011 Page 152 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 47: EQ 6 Judgment criteria and indicators J.6.1 The overall framework from the European Commission down to the implementing organisations enhances attaining global objectives and efficiency I.6.1.1 The programme management procedures applied from the European Commission down to the implementing organisations (and vice versa) are clear and explicit I.6.1.2 Stakeholders consider that the overall framework establishes a clear and optimal distribution of stakeholders’ roles in order to achieve the objectives at EU and country levels I.6.1.3 Stakeholders consider that the overall framework leads to the establishment of appropriate selection procedures in order to achieve the objectives at EU and country levels I.6.1.4 Stakeholders consider that the programme management procedures applied from the European Commission down to the implementing organisations (and vice versa) are clear and explicit Source: ADE Detailed response JC 6.1 The overall framework from the European Commission down to the implementing organisations enhances attaining global objectives and efficiency What is the overall management set-up (framework) from the European Commission down to the implementing organisations? The overall framework in place, from the European Commission down to the implementing bodies are detailed in Commission Regulation (EC) No 501/2008, Commission Regulation (EC) No 3/2008, Commission guidelines AGRI/60787/2007of 1st July 2008 71, as well as Commission guidelines AGRI/64545-2007-rev4 of 23 November 2009. 68F The overall framework allowing to manage EU co-funded information and promotion (I&P) programmes on internal and third country markets, is described in section 1.1.3 and summarised below. It involves entities at Member State level and at EC level. At the Member State level, competent national authorities (or competent bodies) publish calls for proposals. Proposing organisations, such as a trade or interbranch organisations, design and submit a programme promoting agricultural product on internal or third country markets for pre-selection by the competent body This pre-selection is a mandatory step to be carried out at Member State level in order to check whether EU technical and administrative requirements are met. In the case of a multi-country programme – in other words, a programme put forward by more than one Member State (cf. evaluation question 4 for further details), several proposing organisations and several competent bodies may be involved. 71 Guidelines to be considered when assessing and managing part-financing programmes for promoting EU agricultural products. Final Report November 2011 Page 153 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The decision to select (or reject) a programme proposal is made by the EU Commission, namely DG AGRI Promotion Unit (D4). If the programme is selected to be co-funded by EU budget, the proposing organisation is responsible for managing the implementation of the co-funded programme. However, information and promotion activities of the cofunded programme are carried out by one (or several) implementing body (bodies). These last organisations selected by the proposing organisation shall have specific knowledge of the products and markets concerned as well as the resources necessary to ensure that the measures are implemented as effectively as possible taking into account the EU dimension of the programme concerned. During meetings, the management committee, which is composed of EU Member States Ministry representatives and EC promotion unit officials, issues opinions on the draft Commission decision. The overall management set-up works, but there are important issues in terms of selection and implementation procedures, which may impede the achievement of the objectives of the Regulation With respect to selection, Commission Regulations and accompanying guidelines define the basic roles and responsibilities: 72 69F The Member States are in charge of the calls for proposals, the pre-selection of programmes and the checking of the selection of the implementing bodies and The Commission is in charge of the final selection of the programmes. In practice, both MSs and the EC look into eligibility and quality requirements of the proposed programmes. The separation of roles in terms of eligibility examination seems to work well, even though communication between both levels remains necessary, for instance when the EC contacts MSs for questions of clarification on criteria that are under their scope. In terms of quality examination on the other hand, there appears to be a significant overlap in the repartition of roles between the Commission and the Member States. Most stakeholders interviewed in the framework contract evaluations and most interviewees in the sample of programmes of this evaluation believe to have a clear understanding of their roles in the management of co-funded I&P programmes. However, important difficulties appear: The selection process is regularly criticised from different sources. The framework contract evaluations highlighted several issues about the respective roles in the selection of proposals that can be summarized as follows (see table below): First, some Member States failed to fulfil their responsibility of pre-selection while this is a necessary step foreseen in the Commission Regulations. Secondly, there was widespread confusion among most competent bodies and proposing organisations about how selection was done at the EU level. Finally, some interviewees felt that the two-step selection process was somehow duplicative. 72 The following were summarised by the European Court of Auditors Report Final Report November 2011 Page 154 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 48: Issues in terms of selection procedures underlined by the framework contract evaluations covering the period 2002-2007 Framework contract evaluations Milk sector evaluation (IM70F73) (2008) Wine sector evaluation (IM) Organic sector evaluation (IM) USA and Canada evaluation (TC71F74) (2006-2008) Wine sector evaluation (IM) Organic sector evaluation (IM) (2006-2007) Fruit and vegetables evaluation (IM) (2007) Issues There appeared to be limited pre-selection efforts in some Member States. Some stakeholders are not confident that the selection criteria are being objectively applied and suspect that other issues may be influencing selection decisions. Reasons behind the final decisions of approval or rejection of programmes by the EC were not known or understood by some Member States. It was highlighted that the approval process was a duplicate process: both the MS and the EC had to go through the programme details and made their own evaluation with insufficient coordination. There was a suggestion to transfer the entire selection competence to the Member State (only mentioned by the organic sector evaluation). Both stakeholders at national level (competent body and Ministry levels) and at EU level were involved in programme selection. Source: Synthesis of lessons learnt from the framework contract evaluations, ADE, 2011 From the above it appears that the two step selection process, done at the Member State level (pre-selection) and at the EU level (selection), remains a critical issue. Most often, strong criticisms have been recorded in the sampled Member States, especially at the level of competent bodies (which are also paying agencies). For example, the competent body of a large beneficiary of the EU co-funding for I&P programmes felt a lack of interest by the Commission during Management Committee meetings for the efforts and choices made by the Member State, which resulted in a lack of discussion and a perceived unilateral selection by the Commission. There is also a long time between the submission of the programme proposals at Member State level and the start of activities after EC approval (7 months between submission to Member States and EC final decision; plus the time spent beforehand working on the proposal and the delay before effective beginning of the programme which are variable). On the Commission side, it appears that a significant effort in programme quality examination has to be done again or deepened compared to what was done at MS level. Issues related to the two step selection are further discussed under evaluation question 9 which deals with the administrative requirements linked to programme selection and implementation. 73 EU Internal market. 74 Third country markets. Final Report November 2011 Page 155 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC As an example, in the specific case of one multi-country programme (i.e. a programme put forward by more than one Member State), the pre-selection involving two Member States was difficult. Sometimes, both countries had not the same interpretation of the Commission Regulations and guidelines regarding the programme design. For instance, one Member State agreed about the programme proposal prepared by both proposing organisations while the other wanted to reject part of the content of the programme proposal. The Commission had to be contacted to help solving this issue and allow pursuing the pre-selection process. To avoid such problems some proposing organisations at Member State level suggested cancelling the mandatory pre-selection at Member State levels and directly sending the multi-country programme proposals for EC selection. In terms of implementation: Under the terms of shared management, the Member State is also responsible for monitoring the proper implementation of the programmes, payments to beneficiaries and related checks. A monitoring group, presided over by the Member State concerned and including in practice a representative of the Commission, meets regularly to monitor the progress of the various programmes. According to lessons learnt from the Switzerland and Norway evaluations, management was considered a bureaucratic burden. In addition, communication between proposing organisations and the Commission was rare. Except in the UK and in Poland, all the proposing organisations and competent bodies interviewed in the sampled programmes considered the programme management procedures in place heavy and time consuming. These criticisms mainly concern the lack of flexibility to modify an ongoing I&P programme – e.g. with respect to planning of activities, contract modifications, and programme schedule – but also the monitoring and evaluation requirements (see evaluation questions 8 and 9 for further details). However most interviewees underlined that these issues are not specific to the management of co-funded I&P programmes but concern EU affairs in general. Whatsoever this issue is not new and appears to remain unsolved for several years. There were also specific issues in the case of a multi-country programme (i.e. a programme put forward by more than one Member State), as listed below: A lack of coordination between the Member State Ministries, especially when the proposing organisations have to obtain approval both on messages to be conveyed and promotion material; A lot of administrative documents to exchange between involved Member States, mainly at the level of Ministries/competent bodies. In that respect, some involved interviewees proposed to give responsibility of the management to the leading Member States, which submitted a multi-country programme; Different interpretations between Member States in the implementation procedures and documentation requirements. In other specific cases, at Member State level only, roles have been misunderstood and not distributed in an optimal way. In the Netherlands, there were complaints about the implementation of one co-funded programme where the proposing organisation carried out a large part of the activities while normally devoted to the implementing body. In addition, a specific issue appeared in Germany: the abolition of the Central Marketing Agency (CMA) led to considerable redistribution of tasks and the various levels are still finding their feet. Final Report November 2011 Page 156 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC To what extent has the overall framework for managing promotion programmes, from the Commission, through Member States competent authorities, proposing organisations and implementing organisations, been appropriate to achieve the global objectives of the Regulation? Evaluative Judgement The overall management set-up, from the European Commission down to the implementing organisation, is clearly defined in the Commission Regulations and further guidelines. The roles of each stakeholder in the overall management set-up are described in details. However, issues are met at two levels: 1) the two step selection process of programme proposals and 2) the implementation procedures. While the overall management structure is not questioned, both these issues impede the achievement of objectives of the Regulation. In the first case, the two step selection process, at Member State level (pre-selection) and at EC level (selection) is questioned by stakeholders at each end, due to the important time spent on the procedure (over 6 months between application and final selection decision; which is considered very long in the context of promotion) and extent degree of duplication in selection roles. As a result of this two step process, a lot of time is spent without being certain to be co-funded by the EU budget. In the second case, implementation procedures from the European Commission down to the implementing organisations – including programme modifications, monitoring and evaluation, are heavy and time consuming. As far as multi-country programmes – programmes put forward by more than one Member State – are concerned, more time is needed to fulfil administrative and technical requirements (e.g. agreement on messages conveyed in the Member States implementing the programme) and lack of coordination between involved proposing organisations and competent bodies may slow down the implementation of the co-funded programme. 4.3.2 Evaluation question 7 To what extent have the guidelines defining the strategy for information and promotion programmes on the internal market provided useful guidance for the design and implementation of promotion programmes? Rationale and approach The question aims at verifying the extent to which guidelines are accurate, comprehensive, and clear with a view to designing and implementing EU co-funded programmes and measures. Specific attention is paid to the usefulness and the application of the guidelines by Member States (especially proposing organisations in charge of the design and the implementation of the programmes). This evaluation question is linked to the EQ 8 dealing with monitoring and evaluation reporting, as well as to the EQ 9 dealing with administrative requirements (selection and implementation procedures). Final Report November 2011 Page 157 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 49: EQ 7 Judgment criteria and indicators J.7.1 The guidelines have facilitated the design and implementation of the programmes I.7.1.1 Programming documents refer explicitly to the guidelines I.7.1.2 Stakeholders involved in programme design and implementation consider that the guidelines have helped identify adequately the key channels, target groups, messages to be conveyed and objectives I.7.1.3 Stakeholders involved in programme design and implementation consider that the guidelines include all the information needed to facilitate implementation of the programmes J.7.2 The guidelines have been applied I.7.2.1 Results of a market analysis of countries targeted by co-funded programmes are included in programming documents I.7.2.2 The channels, messages to be conveyed, target groups and objectives pursued by the programmes are those mentioned in the guidelines I.7.2.3 The rationale underlying the choice of channels, the messages and the target groups is explicitly mentioned and detailed in programming documents Source: ADE Detailed response JC 7.1 Although suggestions for their improvement are made, the guidelines have facilitated the design and the implementation of the programmes Regulatory and operational guidelines A set of guidelines is available with a view to providing guidance to design and implement co-funded programmes. Based on the nature of their content, they may be divided in two categories: Regulatory guidelines and operational guidelines. Regulatory guidelines are provided by the Council and the Commission. They represent the legal basis allowing the co-funding of promotion and information actions for agricultural products and are provided in: Article 5 (1) of Council Regulation (EC) No 3/2008: In this regulation, general indications are given in particular about (a) objectives and targets to be reached; (b) one or more themes to be covered by the selected measures; (c) the types of measures to be implemented; (d) the duration of programmes; (e) indicative distribution, by market and type of measure envisaged, of amounts available for the EU’s financial contribution to co-funded programmes. Annex I (B) of Commission Regulation (EC) No 501/2008: In this regulation, general indications are provided on co-funded programmes to be designed and carried out on the internal market. These guidelines propose messages, target groups and channels to be at the core of co-funded programmes for a list of 15 different themes and products. However, according to Council Regulation (EC) No 3/2008 (article 5(2)), the EC may Final Report November 2011 Page 158 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC also adopt specific guidelines defining the strategy to be followed in proposals for I&P programmes in third country markets. Operational guidelines are available at Commission level. Based on regulatory guidelines, they provide details about the procedures to design and implement co-funded programmes. They also clarify issues raised at Member State level. These guidelines are provided in the form of three main documents: Document AGRI/60787/2007-rev.5 “Guidelines to be considered when assessing and managing part-financing programmes for promoting Community agricultural products”, of 1st July 2008: This document completes regulations with details and explanations about the co-funded I&P programmes; their objectives; definitions of key words (programme, representative trade organisation, measures); submission of programmes at national level; implementing bodies; submission of programmes to the Commission; conclusion of contracts and provision of securities; financial contributions; advanced and interim payments; documents to be submitted; the monitoring group; checks; reductions, recovery and penalties; as well as some final remarks in addition to the legal requirements. Document Ref. ARES(2009)347563-27/11/2009 “Application form for promotion programmes part financed by the EU”: This document provides Member States with the template to be used for submission of the I&P programme, including three compulsory annexes (signed statement from proposing organisation(s) to honour financial commitments for the duration of the programme, signed statement from the proposing organisation(s) certifying that the programme is not receiving any other financial assistance from the EU, programme identification sheet). An explanatory note on the application form is also available in that document. Document AGRI-64545-2007-rev.4 “Positions and interpretations in connection with implementation of promotion and information programmes”, of 23rd November 2009: This document gathers clarifications by the EC to the attention of Member States on 28 topics related to the implementation of co-funded I&P programmes. Satisfaction and usefulness of the guidelines to design co-funded I&P programmes With very few exceptions, the programme proposals for promoting agricultural products on the internal market explicitly refer to the guidelines, mainly to the regulatory ones but also to the operational ones. The application form is used by proposing organisations to design and submit the programme proposals. There are no specific guidelines for third country markets and proposing organisations tend to design programme proposals for third country markets on the basis of operational guidelines available for the internal market. Some Member States have deplored this absence of guidelines for third countries. In a limited number of cases, competent bodies and POs had the impression that unwritten rules apply at the level of proposing organisations and competent bodies in the absence of these guidelines, which renders programme designing and pre-selection difficult. Regarding the programme design, most of the framework contract evaluations underlined that regulatory and operational guidelines provided by the EC were helpful, in particular in facilitating the choice of objectives and assessing the relevance of the Final Report November 2011 Page 159 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC programme proposals. Guidelines were regarded as a useful tool to frame the proposals. They include a set of general indications and clear orientations, taking into consideration the context of the markets targeted. The usefulness of the regulatory guidelines to design programme proposals has been confirmed by Ministries/ Competent bodies which answered the specific part concerning the regulatory guidelines in the online questionnaire 75. While 96% of the respondents (26 out of 27 respondents to that question) have declared that they were familiar with the guidelines, 66% (23 out of 35 respondents to that question) declared that these guidelines were useful to design I&P programmes 76. In addition, 73% of the respondents (19 out of 26) felt that the guidelines were useful to identify key channels while 92% (24 out of 26 respondents) declared that these guidelines helped to identify adequate target groups. Finally, 73% of the respondents considered the guidelines easy to use (see figure below). 73F 74F Overall, stakeholders at the level of proposing organisation and implementing bodies consider that the regulatory and operational guidelines include all the information required to design programme proposals and have facilitated the choice of messages to convey, target groups to reach and channels to be used. Even though limited, there were criticisms about these guidelines. Those having judged the guidelines useful to a very limited extent or not at all useful have considered that programmes proposals are first designed thanks to experience and knowledge of the markets targeted and then adapted to comply with the guidelines. The evaluator also highlights that the guidelines themselves appear to be outdated on several aspects, including in terms of channels that do not include the recent development of social networks for marketing (e.g. use of telephone information lines to promote fruit & vegetables, milk products, organic..., no reference to new channels such as on-line social networks...). However, it must be underlined by the evaluator, that over the past years, the guidelines had been applied in a flexible manner by the EC (see above mentioned results of the survey), allowing for instance the inclusion additional intermediary target groups if relevant. Teachers and journalists had been allowed as intermediary target groups in the fruit and vegetables sector. Currently, a multi-country programme in the milk sector included medical professionals which are not explicitly mentioned in the target group. This flexible handling of the guidelines is also referred to in the framework contract evaluation of the milk sector (2008), where a number of stakeholders underlined the need to clarifying the regulatory nature of the guidelines: Are guidelines indicative or prescriptive? 75 23% of the respondents to the questionnaire (8 out of 35 respondents) did not answer the question about the familiarity with the guidelines and 26% of the respondents (9 out of 35 respondents) did not answer all the questions about the usefulness and the easiness-to-use of the guidelines. These people have declared these questions not applicable in the framework of their task of managing EU co-funded programmes. 76 It is assumed that the guidelines have been useful when respondents have graded their satisfaction against the proposed statement with “to a large extent” and “to some extent”. Final Report November 2011 Page 160 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Moreover, there were specific suggestions to improve the guidelines for designing programme proposals, from the framework contract evaluations, the online survey, as well as from recent interviews carried out through the sample of programmes: The fruit and vegetables sector evaluation (2007) suggested that the regulatory guidelines should be extended to provide a common format for programme design including minimum requirements, an explanation and description of the programme strategy, provision for an intervention logic, and appropriate tools for measuring the results and impacts of the programmes, taking account of the types of measures implemented. These recommendations have largely been taken into account in the revised guidelines available to applicants on DG Agri website. Some stakeholders interviewed during the organic sector evaluation (2007) voiced demands for improving the methodological guidance for designing programmes and evaluating effects, impacts and efficiency of actions and programmes. The evaluation aspects are discussed in details under evaluation question 9. As far as bananas from outermost regions are concerned, objectives of the guidelines under the theme “Information on the graphic symbol for outermost regions” 77 cited in Annex I (B) of the Commission Regulation (EC) No 501/2008 need to be completed with objectives related to consumption increase. While the symbol for outermost regions is known, the aim of promoting this symbol alone is no longer felt sufficient; a further step would be now to increase consumption of agricultural products – including bananas – having adopted this symbol as market orientation. Some POs and competent bodies in the EU-10 Member States would like to have guidelines with more information or examples on how to prepare programme proposals – in general and in the specific case of multi-country programmes. To assist competent bodies and proposing organisations in the management of multicountry programmes and other programmes to promote agricultural products on third country markets, there is a clear request for defining specific guidelines for these types of actions. In some Member States and in some agricultural sectors, there is a strong demand for more emphasis in the guidelines about the use of messages related to health benefits. According to the framework contract evaluations, health benefits for consumers were not claimed explicitly. Even if difficulties in obtaining scientific supporting evidence were met, many programmes (e.g. in the fruit and vegetables sector or in the milk sector) claimed benefits relating to food safety, nutrition, quality and ‘naturalness’. However, opportunities to communicate about health aspects were missed in one specific case due the long time needed to approve proposed health messages at each level (promotion of olive oil). 75F 77 “Make producers, distributors and consumers aware of the symbol for outermost regions, and make producers using the symbol for outermost regions” Final Report November 2011 Page 161 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Figure 17: Results of the online questionnaire to Ministries/ Competent bodies in the EU-27 Member States about the usefulness and the easiness to use of the guidelines Useful to design I&P programmes 42% Useful to identify adequate key channels 42% Useful to identify adequate target groups 46% 31% 46% Useful to implement I&P programmes 19% 4% 4% 58% 38% 0% To a large extent 12% 35% 25% 50% To some extent To a very limited extent 4% 8% 46% 27% The guidelines are easy to use 8% 23% 75% 4% 4% 100% Not at all Source: Online survey, ADE, 2011 Contradictory opinions on the usefulness of the guidelines to implement co-funded I&P programmes, with a specific request for clarifications on multi-country programmes Information collected about the usefulness of guidelines for programme implementation was contradictory according to different sources. In the framework contract evaluations, the guidelines were reported as rather unclear or not useful for implementation. This opinion was motivated by the fact that the procedures outlined in the guidelines were considered too rigid in terms of budget and calendar modifications (as underlined in the wine sector evaluation). In addition, some POs needed clarifications about how to re-use and duplicate the I&P material with a view to improving effectiveness and efficiency (as underlined in the organic sector evaluation), although article 18.2 (1071/2005) (and currently article 23.2 of Commission Regulation 501/2008 mentions that “the material produced... may be used subsequently provided that prior written authorisation is obtained from the Commission, the proposing organisation concerned and the MS...”. On contrary, according to the results of the online survey filled by Ministries and Competent bodies, 86% of the respondents (22 out of 26 respondents to that question) have declared that the guidelines were useful to implement I&P programmes. More specifically, 27% of them (7 out of 26 respondents to that question) are satisfied to a large extent. With few exceptions, implementing organisations interviewed in Member States were satisfied with the guidelines for implementing I&P programmes. Most of them agree that the guidelines provide the basic information necessary to implement the programmes and that they have facilitated this implementation. As far as multi-country programmes are concerned, guidelines give rise to different interpretations among Member States, making it difficult for everyone to agree on Final Report November 2011 Page 162 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC what could or could not be done by the programme (e.g. illustrating the message of origin with the names of the products). There were few suggestions to improve the guidelines for implementing programme proposals: To provide more details on the management of controls and payments, especially when it comes to invoicing the EC; To provide more details in order to manage I&P programmes in the same way in different EU Member States, particularly with respect to the rules for the proposing organisations. JC 7.2 Globally, the guidelines have been well applied, according to the provisions made in Annex I (B) of Commission Regulation (EC) No 501/2008, sometimes with minor variations Most often a detailed market analysis was carried out before designing the programme proposal In nearly all the programmes selected for in-depth assessment, results of a market analysis of countries targeted by the I&P measures have been used to design the programme proposals. However, the quality of these market analysis differs from one programme proposal to another. In the majority of cases, detailed market analyses were carried out. For example, the Information on the Polish meat promotion programme proposal (SRWRP) studied in detail the demand for meat in Eastern Europe and Asian markets before designing its I&P measures. Results of this market study were completed by lessons learnt from a previous promotional programme implemented between 2007 and 2009 (e.g. for the choice of markets to be targeted: Russia, Ukraine, China, Singapore, Thailand) and existing trading partnerships (Poland, Russia and Ukraine). In the few cases where a weak analysis was undertaken, programme proposals have been designed based on results of previous I&P programmes (e.g. Information on the 5 am Tag promotion programme in Germany) or on a short description of the markets targeted (e.g. The Spanish Information on the OIAOE promotion programme and the French-Italian programme promoting apples on third country markets). In one specific case (the Greek Information on the Seko Dramas promotion programme), market analysis was reported to be not necessary as the knowledge of the markets targeted (Russia and Ukraine) was satisfactory and data on these markets was already available. There has been a match between channels, messages, target groups and objectives of the sampled programmes on one hand and those mentioned in Annex I (B) of Commission Regulation (EC) No 501/2008 on the other hand According to the framework contract evaluations, guidelines are generally well applied. Although some variations were evident, most of the sampled programmes pursued objectives, targeted groups, conveyed messages and used channels as mentioned in the EC guidelines. In the same way, the sampled programmes selected for in-depth assessment during this evaluation have used, sometimes with limited variations, the channels, Final Report November 2011 Page 163 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC messages, target groups and objectives mentioned in Annex I (B) of Commission Regulation (EC) No 501/2008. For example, in the CNIEL & NZO multi-country I&P programme (French part), the target group is broader than the one mentioned in the guidelines: it includes medical professionals, which are not included in the guidelines. In one specific case, one channel was implemented while it was not mentioned in the Annex (I) B of Commission Regulation (EC) No 501/2008. Indeed, the French Information on the UGPBAN, Caraïbes melonniers & Anafruit promotion programme has carried out TV-spots while this channel is not part of the channels mentioned in the Information on the graphic symbol for outermost regions theme, under which the programme has been co-financed. This case suggests that guidelines are more indicative than restrictive as long as the actions are coherent and coordinated in an I&P strategy. That being said, it could be interesting to add this specific channel under the symbol for outermost regions theme with a view to avoiding possible misunderstanding in the future. With few exceptions, explanations about the rationale of the programme is mentioned in the programming documents Generally programming documents of the sampled programmes selected for in-depth assessment included a clear and detailed strategy explaining how the programmes intended to reach their objectives. Most often, the rationale underlying the choice of channels, the messages and the target groups are detailed in the light of lessons learnt from past (cofunded) I&P programmes and/or from results of market analyses, and/or of past experiences of proposing and implementing bodies. For example, the British Milk Marketing Forum programme (UK-MMF) justified the choice of channels and the way these channels will appeal to the target group as follows: The use of popular celebrities is justified by their impact on teenage girls and young mothers, who are the target groups. In addition, the channels selected for the beginning of the I&P activities (visuals with celebrities that are popular among the target group) are used to build an emotional tie with milk and the later activities increasingly aim to promote more rational messages that will influence consumer patterns. There are some exceptions where the rationale underpinning the choice of channels, the messages and the target groups, is not detailed or is not mentioned. Even though objectives, messages and target groups are those mentioned in Annex (I) B of Commission Regulation (EC) No 501/2008, programming documents of the Spanish Information on the OIAOE promotion programme (ES-OIA) did not explain why they were chosen and how they were interrelated. Final Report November 2011 Page 164 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC To what extent have the guidelines defining the strategy for information and promotion programmes on the internal market provided useful guidance for the design and implementation of promotion programmes? Evaluative Judgement A set of guidelines to design and implement co-funded programmes on the internal market is available from Council and Commission Regulations. Regulatory guidelines provide the legal framework and operational guidelines provide details about the procedures to design and implement co-funded programmes to EU Member States. There are no specific regulatory guidelines for designing co-funded I&P programmes on third country markets. Consequently proposing organisations tend to design these programmes on the basis of guidelines for the internal market, which may not be fully adapted for this purpose and needs further reflection. Globally regulatory and operational guidelines are considered useful to design I&P programme proposals especially on the internal market. They provide a frame, that includes the information required to design programme proposals and have facilitated the choice of messages to convey, target groups to reach and channels to be used even if some of the information is outdated in particular in annexes of Commission Regulation (CE) 501/2008 (e.g. in the choices of communication channels). However, as confirmed by the framework contract evaluations and the online survey, the Commission handled the respect of the guidelines with flexibility and thus some old-fashioned elements did not pose a problem. There is thus a need for updating Annex I (B) of Commission Regulation (EC) 501/2008. This is confirmed by competent bodies or proposing organisations that have made some suggestions of updating Annex I (B) with a view to taking into consideration the evolving context in the promotion of agricultural products and themes. Currently, guidelines (especially operational) are considered useful for implementing cofunded programmes by competent bodies. They were much more criticized under the framework contract evaluations, but in relation with procedures. Mainly, clarifications are needed regarding the management of multi-country programmes, where Member States had different interpretations of regulatory guidelines (Annex I (B) of Commission Regulation (501/2008)). Generally, principles laid down in regulatory and operational guidelines have been well followed. Most often, programming documents have included a market analysis. Even if some of these analyses are weak, the majority of them detailed sufficiently the markets targeted. Based on results of market analyses, on past I&P experiences of proposing and implementing bodies, as well as on past co-funded programmes, programming documents have explicitly explained the rationale underpinning the choice of channels, the messages and the target groups. These channels, messages and target groups are in line with those of Annex I (B) of Commission Regulation (EC) No 501/2008, with minor variations sometimes. Final Report November 2011 Page 165 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 4.3.3 Evaluation Question 8 To what extent have the annual evaluation reports enabled effective monitoring and evaluation of the promotion programmes? Rationale and approach The question examines to what extent the quarterly monitoring reports and annual evaluation reports have enabled effective monitoring and evaluation of the promotion programmes. It more specifically aims at verifying the extent to which the structure, the guidance and the information required in the reports have facilitated a) effective monitoring of co-funded programmes, including generation of quantitative information on e.g. human and financial resources, as well as delivered outputs and the use made of the promotion material; b) effective evaluation of co-funded programmes, including analysis of results and impacts, taking into account the global and specific objectives of the promotion measures/programmes, as well as the use of lessons learned to adapt promotion measures with a view to improving the attainment of the objectives of the on-going programmes or the design of further programmes. Programmes implemented on both the internal and third country markets fall within the scope of this question. The answer to the question focuses on both the technical structures of the annual evaluation reports and their required content, as well as PO and competent body opinions. Table 50: J.8.1 I.8.1.1 I.8.1.2 I.8.1.3 I.8.1.4 I.8.1.5 J.8.2 I.8.2.1 I.8.2.2 I.8.2.3 I.8.2.4 I.8.2.5 EQ 8 Judgment criteria and indicators The quarterly monitoring reports have facilitated completion of effective monitoring of the implemented promotion programmes The structure of the quarterly monitoring reports is adapted to presenting the monitoring results of the co-funded programmes The quarterly monitoring reports include appropriate monitoring results (i.e. the consumption of human and financial resources, delivered outputs and the use made of promotional material) Stakeholders consider that the structure of the quarterly monitoring reports and the guidance for completing them in terms of monitoring results are clear and explicit Stakeholders have suggestions for improving the monitoring of and reporting on co-funded programmes The quarterly monitoring reports are used in the routine management of programmes The annual evaluation reports have facilitated completion of an effective evaluation of the implemented promotion programmes The structure of the annual evaluation reports is adapted to presentation of the evaluation results of the co-funded programme The annual evaluation reports include appropriate evaluation results, including on coverage of the audience/target groups and how far they were reached; results and impacts in terms of improvement of the knowledge/ perceptions of target groups on agricultural products and related themes, (new) market penetration, increased consumption or increased sales The evaluation results from the co-funded programmes are used to adapt the on-going cofunded programmes or to design further co-funded programmes Stakeholders consider that the structure of the annual evaluation reports and the guidance to complete them in terms of evaluation results are clear and explicit Stakeholders have suggestions for improving the evaluation of and reporting on co-funded programmes Source: ADE. Final Report November 2011 Page 166 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Detailed response There are three different reports: - the quarterly monitoring reports; - the annual reports; - the retrospective evaluation carried out at the end of the programme. A model of quarterly monitoring reports (QMR) is available in the in the annexes of the model contract to applicants on the website of DG Agri promotion unit. It also includes a model of annual reports. The annual report is an aggregate of the QMR. It is almost identical to the quarterly monitoring reports with a summary financial statement and a brief description of measures with a short paragraph about programme implementation. This should include any relevant observation and mention any difficulty encountered in implementing the programme and any amendment or cancellation of activities and the reason. 3% of the programme budget can be earmarked for the annual evaluation reports. An additional important aspect is mentioned in Commission Regulation 501/2008, article 19.1, about payment of balance. It states that the annual report should comprise a summary of the work carried out and an evaluation of the results obtained, as can be ascertained at the date of the report. This latter request is not included in the model of annual reports included in the model contract. A reference to the retrospective evaluation is given in the annex of the model of application form available on the Website of DG Agri. It is structured around five criteria78 similar to the framework contract evaluations, namely coherence, efficiency and effectiveness, impact and complementarity. J.8.1 The quarterly monitoring reports have facilitated completion of effective monitoring of the promotion programmes Although the structure of the quarterly monitoring reports is overall adapted for presenting the monitoring results, the quarterly frequency for monitoring purposes is disputed Monitoring reports are understood as quarterly monitoring reports (QMR) and annual reports (AR), the latter being basically aggregates of the quarterly ones and are dealt with in J8.2. 78 - The key questions to be considered are represented by the following five criteria: coherence between the implemented actions and the objectives of the Regulation; actions and information channels used and their cost-effectiveness; coverage and content of the programmes; impact and effectiveness of the actions; and Complementarity between Member States' programmes and programmes submitted by professional organisations. Final Report November 2011 Page 167 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC A model for both quarterly and annual reports is available in the annexes of the model contract to applicants on the website of DG Agri promotion unit. The two models are very similar; the annual report is aggregating financial elements of the year. The structure of the QMR annexed to the contract is basically followed in practice. The provided template is overall adapted for monitoring implementation progress even if some key information is missing, such as the indication of multi-country or multiproduct programmes. Although some POs found that the reports were easy to complete, opinions on their usefulness for monitoring were divided. Some mentioned that there is little added value beyond the fact of giving rights to payment. In addition, most proposing organisations and some competent bodies found QMR unnecessarily frequent. Quarterly monitoring reports include first monitoring results but there are serious doubts on their value for evaluation purposes The quarterly monitoring reports include in most cases input and output indicators, enabling a close monitoring of implementation progress, especially when the target values are included. Little other relevant information is contained in these reports though. Most interviewees use a basic version of the quarterly monitoring reports consisting of an enumeration of activities conducted, quantification of outputs and listing of expenditures and resources used. Only in a few cases the report analyses progress in greater depth. There is general agreement on the limited added value of the exercise (redundancy with financial reporting, administrative burden) Although the provided structure and guidance is clear and relatively explicit, administrative burden in the quarterly reporting exercise is criticised. The structure and guidance of the quarterly monitoring reports is clear and explicit. The requested information in the template is self-explanatory. Interviewees considered the quarterly reporting exercise a relatively easy and straightforward task, with a large degree of repetition from one report to another though, as well as repetition with the information contained in the programme itself. Experience gained in the past helps a lot for the next reports. Lack of space for forecasts in the templates and of a completed example was mentioned. Some interviewees were not aware of any guidance. In the survey, more than 70% of the respondents considered the provided guidance and requirements clear enough to fulfil the monitoring; only one responded negatively, while an important share (25%) did not express an opinion. More flexibility was asked (regarding possibilities to adapt promotion activities), as well as an alleviated administrative burden. In another part of the survey, interviewees suggested to adapt the requirements and guidance of the monitoring to the conditions of the market and to observe the principle of efficient use of funds. Final Report November 2011 Page 168 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC It is suggested to reduce the frequency of monitoring reports from quarterly to biannual or annual. Available QMR essentially list activities carried out during the quarter. They rarely compare actual achievements with targets and usually do not address deviations when they occur. Data is useful for completing the financial report though. Suggestions were made to reduce the degree of detail of the reports (especially in France); “smart” improvements were also suggested in the operational framework (e.g. electronic invoices), as well as provisions for a feedback by the national and EC authorities. . It is interesting to observe that only 15% of the interviewees in the survey had suggestions to improve the monitoring requirements and guidance (mainly on report templates) whilst the vast majority had no opinion. Use of quarterly monitoring reports in the management of programmes is mixed As they are, QMR can only be used for ex post assessments for management. However they provide a useful content grid, especially in case activities are continuously monitored rather than compiled ad-hoc every three months. Interviewees’ opinion is quite mixed. Italian and Greek POs & competent bodies seem to use information collected by the quarterly monitoring reports for management purposes, while others (France, Germany, the Netherlands, and UK) consider the QMR not adapted to daily management. J.8.2 The annual evaluation reports have facilitated completion of an effective evaluation of the implemented promotion programmes The annual evaluation report is not well understood and its structure is not entirely fit for purpose. Annual evaluation reports are in some cases understood as annual reports (aggregated QMRs) and in some cases as genuine evaluations. In practice the two interpretations of the report are either erroneously or deliberately merged. The Annual Report Template (Annex VIII of the Contract Template) is not well adapted for its main purpose, monitoring being the dominating element compared to evaluation. At its best it can provide a basis for evaluative analysis if a target/achievement comparison is provided and if inputs/outputs/targets ratios are calculated to provide information on cost efficiency. It does not include the request of the regulation of an evaluation of the results obtained, as can be ascertained at the date of the report (501/2008 art.19.1 mentioned above). Annex III of the Contract Template does include under its item 5 “evaluation of the results of measures”, the possibility of including the cost of evaluation of results of the programme measures, notably upon expiry of each annual phase of the programme. The approach proposed in the “Application form for promotion programmes partfinanced by the EU” with the definition of evaluation questions and criteria to be Final Report November 2011 Page 169 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC projected on the programme structure can be useful but is not explicit enough regarding evaluation. POs and competent bodies were aware of a standard structure as available in an annex to the grant contract and found it suitable for evaluation purposes. The content of available annual evaluation reports is limited in terms of “evaluation results” especially after the first year of implementation Due to their recent implementation, only few annual evaluation reports were found in the reviewed programmes (EL SEKO Annual Report and Study on Effectiveness, PDO programme of FR-IT-PT (Discover The Origin), Evaluation Presentation of FRNL milk program and the Final Report of the German Programme “5 am Tag”). The annual evaluations conducted in different programmes were very diverse. Approaches ranged from qualitative statements and basic impact assessments to more advanced methods including counterfactuals and estimates of the net effects of programmes in a multi-factor context. Both proposing organisations and competent bodies found that the annual reports enable no more than the assessment of efficiency and effectiveness of expenditures and reveal market trends but remain essentially monitoring reports. The results of annual evaluations are not used to adapt or to design co-funded programmes? With respect to the use made of evaluation results, POs either provided no reply or stated that they come too late or require too big administrative burden for programme adaptation. Instead, own appreciation of implementing bodies or experience from former programmes is used for that end. Hence annual evaluation reports are preferably used to adapt single activities (e.g. stopping a marketing activity if the awareness level is not affected) rather than major re-designs. They are also rarely used for follow ups. Is the provided structure and guidance clear and explicit? POs and competent bodies found the report structure adequate even if not well adapted to certain aspects like efficiency and effectiveness (an opinion not shared by the evaluators). Considering guidance most interviewees found it clear and explicit nevertheless there was a strong request for methodological standards, more precise guidance documents and uniformity. A few blamed the structure and guidance either because they are considered disproportionate for their programmes or because they prefer their own evaluation tools. Approximately half of the respondents in the on-line survey found the requirements and guidance enough clear. Roughly one fifth of the respondents had the opposite opinion. These reactions offer an interesting comparison to the assessment of the clarity of the guidance of the monitoring reports, where almost no negative responses were recorded. Final Report November 2011 Page 170 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Figure 18: Survey response on clarity of evaluation requirements (Source: Online survey, ADE, 2011) Are there suggestions for improving the evaluation of and reporting? POs addressed standardisation e.g. by introducing best practices in evaluation methodologies, a standardised structure of the reports or specific tools, e.g. problemsresponses list and post-test evaluation. In line with that suggestion was the request of flexible evaluation budgets, and the facilitated adaptation of the programme based on evaluation findings. However as a priority “things should remain simple”. Last but not least, the evaluation process should be understood as a dialogue rather than an examination. Finally, the survey response on suggestions to improve the evaluation requirements shows again a low level of participation. Only 15% of the interviewees had suggestions, while the vast majority either said they do not have one, or did not provide any answer. Final Report November 2011 Page 171 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC To what extent have the annual evaluation reports enabled effective monitoring and evaluation of the promotion programmes? Evaluative judgement Although the structure of the quarterly monitoring reports is overall fit for purpose, they are considered too frequent. In practice, they enumerate activities with input and output indicators and listing of expenditures. Even though they enable some monitoring, their value is limited for evaluation. They are mainly useful to receive payments by Member States. Although the provided structure and guidance is clear and relatively explicit, the administrative burden due to frequency (quarterly) is criticised. The annual reports (for which 3% of the promotion budget can be earmarked) look more like “annual progress reports”. The present format is not much more than an aggregate of the quarterly monitoring reports, not well adapted for evaluation purposes. The main request of Reg 501/2008 of an evaluation of the results obtained in the annual report is not included in the model of reports annexed to the contract but it is mentioned as a possibility in Annex III 5 a under expenditure on implementing measures. Currently, there is little evaluation of results in the submitted reports. In addition, there are obstacles in putting evaluation findings into practice, mainly due to the administrative burden of programme re-design. Since 2008, the new requirement of retrospective evaluations at the end of the programme – for which guidance is provided in the application forms – may be considered as useful evaluation. Overall the annual reports have facilitated the monitoring of promotion programmes but only partially contributed to effective evaluation. The format provided in the contracts does not include the explicit request of evaluation of the results obtained. Annual reports are currently rather focused on monitoring than evaluation. The final evaluations, for which 5% of the budget can be earmarked, should provide relevant information on programme impact and achievements79. 79 The final evaluations of the studied promotion programmes were not yet available, since these programme were still in operational phase Final Report November 2011 Page 172 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 4.3.4 Evaluation question 9 To what extent are the administrative requirements – selection procedure and implementation requirements – adequate to ensure that information and promotion programmes are selected and implemented in a transparent manner and based on relevant criteria? Is there any room for simplification of administrative requirements without compromising the necessary monitoring and evaluation requirements? Definition and comprehension of key terms Table 51: J.9.1 I.9.1.1 I.9.1.2 I.9.1.3 J.9.2 I.9.2.1 I.9.2.2 I.9.2.3 J.9.3 I.9.3.1 I.9.3.2 I.9.3.3 I.9.3.4 Final Report EQ 9 Judgement criteria and indicators Programme proposals are based on clear, sufficient and satisfactory selection procedure requirements Criteria to select the programme proposals and guidance related to the selection procedure are available at EC and Member State levels Stakeholders consider that the criteria for selecting programme proposals and the guidance on the selection procedure are clear and sufficient for selecting information and promotion programmes in an appropriate and efficient way Stakeholders have suggestions for improving selection criteria for programme proposals and guidance relating to the selection procedure Co-funded programmes are implemented on the basis of clear, sufficient and satisfactory implementation requirements Implementation criteria and guidance for implementing the co-funded programmes are available at EC and Member State levels Stakeholders consider that implementation criteria and guidance to implement the co-funded programmes are clear and satisfactory to ensure appropriate and efficient implementation of the co-funded programmes Stakeholders have suggestions for improving implementation criteria and guidance on implementing the co-funded programmes There is room to simplify selection and implementation requirements while retaining sufficient and relevant information to conduct proper monitoring and evaluation of the co-funded programmes There are no examples of overlaps among the selection requirements in the selection procedure There are no examples of overlaps among the implementation requirements in the implementation of co-funded programmes Stakeholders have suggestions for simplifying selection and implementation procedures keeping in mind monitoring and evaluation needs Selection and implementation requirements for I&P programmes are similar to those of other promotional measures financed under the CAP November 2011 Page 173 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Detailed response J.9.1 Programme proposals are based on clear, sufficient and satisfactory selection procedure requirements Sources for Criteria The Regulations are the primary sources of criteria for selecting proposals. These have been widely available at all levels and most POs have used them as the main source of guidance in preparing proposals (see also EQ 7). Some additional documents have been produced that are often referred to as providing additional guidance on how proposals will be selected. AGRI/60787/200780 describes what proposals should cover and therefore provides a basis for checking eligibility. AGRI/63454/2007 81 describes questions that should be addressed in evaluation and therefore provides background information on what issues are considered important. AGRI/64046/2008 82 is reported to describe the criteria used in selecting programmes. The first two documents are available on DG Agri website for promotion of EU farm products. The third document, which was referenced in the EC response to the Court of Auditors report about selection procedures, is a grid of selection criteria, internal to the DG Agri, D4. 76F 7F 78F Evaluation Grids at MS level Member States check eligibility (compliance with the national and Community Regulations, representativeness of the proposing organisation, technical and financial effectiveness of the proposing organisation, and technical and financial capacity of the implementing body) and quality of the proposals. Some Member States use evaluation grids for assessing the quality of proposals selected and submit these with the programme applications. These standard grids have four headings, as described in the table below. This grid is applied equally to programmes on the internal market and in third countries. The Member State scores are reported in the standard project information sheets compiled by the Commission. 80 ‘Guidelines to be considered when assessing and managing part-financing programmes for promoting Community agricultural products’, available on the DG AGRI website for promotion of EU farm products 81 This describes the EQ of the previous framework contract evaluations and has been integrated in the current applications forms as an Explanatory Note. This is also available on the DG AGRI website for promotion of EU farm products 82 Referred to in the EC response to the Court of Auditors Report. Final Report November 2011 Page 174 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 52: Standard Selection Grid by Member States Criteria General Interest Relevance to market Relevance to target group Quality and Effectiveness EU Dimension Cost Effectiveness Total Maximum Score 30 20 10 40 10 20 100 Source: Programme application documents submitted by Member States. In some cases, the quality and effectiveness criteria is split into: coherence with objectives (10), scope (i.e. duration and targeting (10), quality of messages (10), methods of measuring impact (5) and quality of presentation (5)). The availability to proposing organisations of the selection grid used by Member State competent bodies is mixed. In France and Spain, the evaluation grid is provided to proposing organisations as part of the annual call for proposals. In Portugal, the grid is available on the website of the competent body, but proposing organisations also considered that it was sufficient to follow the guidelines in the regulation and assume that selection would be based on these guidelines. In the UK, the grid is not available and there was some concern that, if the grid were made available, applications would be driven by scoring maximum points, which might undermine the creativity and focus of proposals. Italy has adopted a different grid that includes no criteria for ‘quality of presentation’, but does include one for the experience and competence of the proposing and implementing organisations, which is treated as an eligibility criteria in other Programme Identification Sheets. Several Italian programmes proposed linking the selection criteria to a quantitative assessment of the representativeness of the proposing organisation 83. 79F The most experienced competent bodies work closely with proposing organisations. There is evidence that some competent bodies choose to send low quality proposals to the Commission, because it is politically easier to pass responsibility for rejection to a more distant body. In the UK, there had been strong continuity of staff in the competent body, which was helpful to proposing organisations. In the Netherlands, the competent body provides good advice to help proposing organisations to maximise their chances of success. In Germany, there was a substantial drop in proposals in the few years immediately after one major competent bodies was closed down. In Poland, there was considerable exchange of experience and mutual support amongst the various proposing organisations, which led to a common standard and approach amongst applications. In practice, some Member States do a pre-selection, which already starts when working with proposing organisation to design a proposal. But there is also evidence that some 83 Document AGRI-64545-2007 confirms that there is no firm arrangement for assessing the representativeness of proposing bodies and that this is left to Member States, to be verified by the Commission. Final Report November 2011 Page 175 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC competent bodies fail to fulfil their responsibility of pre-selection while this is a necessary step foreseen in the Commission Regulations (as mentioned under § 1.1.3). Commission Selection As already mentioned in the descriptive chapter under § 1.1.3, the Commission makes the final selection of proposals. The guidelines imply that the Commission first checks eligibility (respect of submission deadline, competitive tendering of the implementing body, signed commitment for co-financing by MS and proposing organisation, coherent set of actions, no link with private brands except for designations under Community Regulation, eligible products & countries, compliance with the guidelines) & quality. It then gives priority to multi-country programmes and those complementary to the national/private initiatives. The Commission appoints two internal readers – and occasionally a third external one – to assess each proposal against the relevant criteria and relies heavily on the judgement of these readers, who provide a degree of consistency across programmes. In many cases, the first attention of readers is to check for omissions. In practice, those programmes that have been well prepared, without omissions, also tend to score well on quality criteria. A standard information sheet is compiled by the Commission presenting the scoring grid prepared by the Member States and adds an assessment by the Commission on four criteria: quality (including coherence and the quality of actions); European dimension; impact assessment; and cost effectiveness. For quality, EU dimension and cost effectiveness, the Commission assesses each application as “good”, “average” or “poor”. For impact assessment, the Commission generally provides a yes/no assessment. Information on the Commission assessment was available for 12 of the 15 sampled programmes and suggested that there was some consistency between the Commission assessment and the Member States score 84, suggesting that criticism of the double selection system may be overstated. However, this evidence comes only for successful programmes and the dissatisfaction reported by POs concerns the reasons for rejection. 81F The information sheets compiled by the Commission are sent to competent bodies. The Commission compiles ‘selection books’ at the end of each year with about 10 to 20 pages for each of the programmes accepted or rejected. In addition to the eligibility and quality criteria, the books also summarise the opinions of the independent readers and include an assessment explaining the Commission’s decision (see Table below), plus an overall score, generally out of 9. The selection books are published on CIRCA and made available to the competent bodies, which do have access to the readers’ assessments. 84 The average scores for those programmes that the Commission assessed as good and as average, respectively, was 31.7 and 23.0 for quality, 8.5 and 7.7 for EU dimension and 16.6 and 12.7 for cost effectiveness. Final Report November 2011 Page 176 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 53: Most Common Comments in EC Assessments on 2009 Programmes Source: ADE based on EC selection books 2009 In addition to the above, there is a much more detailed scoring sheet available (AGRI/64046/2008) in the Commission that defines different scoring criteria with a 1000 points total, based on representativeness (from a multi-country perspective and in terms of national representativeness), duration, number of targeted countries, quality of actions, coherence & strategy, budget, financing, objectives, evaluation methods and general parameters. Experience in the Commission suggests that a scoring system of this complexity is not practical and that a simpler system is required. This would be in line with normal commercial bid assessment practices, where the number of criteria is normally limited to between 5 and 10. According to the table above, unclear activities, unclear budget, no reference to guiding documents and inconsistent actions appear as weaknesses, as well as the lack of strategy for third country markets. Despite the practices adopted to ensure consistency, it is impossible to avoid some subjectivity in the selection process. For the cost effectiveness criteria, the Commission use comparative tables of pricing which enable them to compare unit costs and to identify extreme cases of expensive outputs; and to compare the coverage of the programmes with their total costs (this is based on the detailed costs for three years of actions). Analysis of unit costs may be useful to identify extreme cases. But in practice, market conditions vary greatly amongst Member States and even more in third countries and products and over time. The table below presents the number and budget of programmes approved and rejected from July 2006 to July 2010 (results until end of 2010 are provided by the note below the table). The table and note show that 257 programmes out of 447 pre-selected by Member States were rejected by the Commission (57%), accounting for 56% of the planned expenditures85. 85 This last figure does not take account of 2010 third country expenditure as this data was not available Final Report November 2011 Page 177 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 54: Programmes approved and rejected from July 2006 to end of 2010 Decision Internal market C(2006) 3079 7/07/2006 C(2006) 6458 12/12/2006 C(2007) 3299 10/07/2007 C(2008) 3738 22/07/2008 C(2009) 5583 23/07/2009 C(2009)10784 18/01/2010 C(2010) 4312 29/06/2010 Total Third countries C(2006)5987 11/12/2006 C(2007) 5911 7/12/2007 C(2008) 8233 12/12/2008 C(2009) 9404 30/11/2009 Total Total Internal market + third countries Approved programmes Proposals received Number Co-financing Rejected programmes CoNumber financing 48 49,942,266 € 33 26,659,129 € 41 61,087,592 € 32 51,617,128 € 3 2,546,414 € 36 48,566,109 € 193 240,328,638 € 79 1 56 72 48 16 55 327 31 1 23 31 16 13 19 134 27,660,899 € 2,700,000 € 38,857,435 € 45,699,154 € 27,855,353 € 17,931,027 € 30,299,761 € 191,003,629 € 28 25 30 20 103 10 18 11 10 49 9,079,319 € 19,591,571 € 17,817,101 € 11,105,132 € 57,593,123 € 18 7 19 10 54 20,969,668 € 10,598,650 € 29,895,002 € 16,262,011 € 77,725,329 € 430 183 248,596,753 € 247 318,053,968 € Source: EC, November 2010. Report from the Commission to the European Parliament and the Council on the application of Council Regulation (EC) No 3/2008; SEC(2010) 1434 final Note: The evaluators received information from the EC informing that in 2010, for third countries, there was a total of 17 projects submitted of which 7 were accepted. This high rate of rejection by the Commission in the context of budget availability may generate frustrations and misunderstandings and may indicate a use of the resources which is not optimal. 83F84F All but one (in Greece) of the proposing organisations said that they did not understand the basis for selection by the Commission and speculated about how this takes place. Several complained that programmes could be judged eligible and still be rejected, suggesting that there is confusion over whether selection is competitive. The Dutch, German and Greek competent bodies thought that the Commission decisions produced results that were very different to those of the Member States. The Spanish competent body argued that their priorities (i.e. response to market crisis) were different to those of the Commission (i.e. European integration). Three programmes (in the UK and France) speculated that the Commission needed flexibility to be able to spread funds more fairly across MSs and across products, although the Commission states that no explicit attempt is made to ration funds by country or product. The UK programme suggested limiting on the number of proposals per Member State, to encourage effective pre-selection. Many POs suggested that selection should be done at only one level. In most cases, it was not considered important which level took responsibility, although some (e.g. in Greece) suggested the Commission whilst others (Spain) suggested the Member States. Final Report November 2011 Page 178 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC J.9.2 Co-funded programmes are implemented on the basis of clear, sufficient and satisfactory implementation requirements Availability of implementation requirements. The rules for implementing programmes are provided as part of the call for proposals and in the form of contracts between the competent body and the proposing organisation. Further guidelines are provided in AGRI/60787/2007. The contract defines procedures covering the following issues, amongst others: changes to the budget; information requirements; payment; control of expenses; budget format; expenses allowed; advances; quarterly calendar of activities and quarterly and annual reporting. There were no reports of problems in accessing the implementation criteria and guidelines. Furthermore, the EC promotion unit has noted the possibility to organise ad hoc technical meetings on given subjects. Several programmes reported that competent bodies are able to provide guidance on implementation as and when requested. One programme (in Greece) reported that there is a lot of shared experience amongst proposing and implementing organisations of different programmes which helps to ensure that implementation goes smoothly. Flexibility. There were a variety of comments about the desirability of introducing greater flexibility in the budget transfers to allow work schedules to be adjusted more easily in the fast moving world of marketing. The rules in force require budget transfers of greater than 10% to be approved by competent bodies. Furthermore, budget transfers can only be made if savings have been made in undertaking work more cheaply than expected and it is not, therefore, possible to shift funds out of more expensive actions without a contract revision (see AGRI-64545-2007). Four programmes (in Greece, Spain and Italy) reported some problems with obtaining approval for budget transfers of greater than 10% and suggested, in particular, that it is too restrictive to require proposing organisations to submit a request to approve budget transfers to competent bodies 15 days before a new trimester. These programmes suggested simplifying the rules to present work schedules for each semester. The 2007 Fruit and Vegetables evaluation also suggested more flexibility, especially in the adaptation of the work schedule. In contrast, one programme (in the UK) said that, whilst it was important to be able to adapt work programmes quickly, the current rules of budget transfers worked well and that they had had no problems in gaining efficient approval on budget transfers from the UK competent body. Detail Required. All the programmes in France, the one in the Netherlands and Germany, reported that the level of detail required in monitoring reports was too high and that this reduced the efficiency of the programme. The Dutch programme pointed out that the proposing organisation has as much interest in the success of the programme as the Commission and can therefore be trusted to monitor its activities. In Germany, the bureaucratic requirements and level of detail required in the selection process are considered too high (e.g. price of a square metre in a fair that will take place in 3 years) and hamper the flexibility of the implementation of programmes. This problem of requiring excessive detail required was also reported in the framework contract evaluations (wine sector 2007, milk evaluation 2008). Final Report November 2011 Page 179 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC In two cases, there were complaints about lack of clarity in the Regulations and guidelines: For multi-country programmes, the guidelines leave room for different interpretations among Member States about the eligibility of activities (e.g. including a message of origin with the names of the products). There are no guidelines on the way to prepare quarterly invoices and hence to obtain EC reimbursement. In fact, validity dates for invoices (from implementing agencies, for example) are not specified. For example, it is not clear whether the date of validity refers to the date of services being provided or of the invoice from the implementing agency. Multi-Country Programmes. Two programmes (one in Italy and one in France) reported that, for multi-country programmes (i.e. put forward by more than one MS), obtaining approval for messages from different MS competent bodies often involved significant delays. This problem was also reported in the 2007 Fruit and Vegetables evaluation. The Italian programme suggested that there should be greater coordination between competent bodies at an early stage in multi-countryprogrammes, before the start of implementation. The French programme suggested that a single country should be allowed to lead on the implementation of multi-country programmes. A number of detailed suggestions were made. A competent authority suggested that rules concerning the use of social media sites were unclear and that the Commission had sometimes been opposed to their use. One programme (in the UK) suggested clearer guidance on the choice provided to implementing organisations over whether they charge and hourly rate or a flat fee. Providing worked examples could be useful. Two programmes (in the UK and in France) suggested that the rules regarding exchange rates were not satisfactory, although no specific suggestions were made to improve this situation. J.9.3 There is room to simplify selection and implementation requirements while retaining sufficient and relevant information to conduct proper monitoring and evaluation of the co-funded programmes Simplification. The evaluator has identified three essential elements for which there is room for simplification. In terms of selection, there is a perceived duplication of roles of MSs and the Commission. Indeed, both MSs and the EC perform selection on eligibility and quality, with a significant overlap in terms of quality examination. A different distribution of roles could ensure an easier and smoother selection process (details in the closing chapter). The level of detail required in terms of projected unit costs of actions (for the whole programme duration) in programme proposals is excessive, especially for the later years. This also has an impact on the implementation of programmes, limiting flexibility for changes as the programmes are underway. A proper cost-effectiveness assessment could be performed based on the detailed costs for the first year only. Indicative amounts could be used for the second and third years. Final Report November 2011 Page 180 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The level of detail and frequency of reporting during the programme is high and has limited added value for the quality of the programme. A revised reporting system could improve the process. In addition, the following specific suggestions were made at Member State level (mainly competent bodies and some proposing organisations): detail required for later years of the programme (year 2 and 3) should be reduced (e.g. cost-output ratios) (France and Spain); mandatory inclusion of market statistics and SWOT analysis (Portugal); some differences in criteria for internal market and third country programmes (Portugal); distinction between information programmes and promotion programmes (Portugal); clearer prioritisation of criteria, with more emphasis on quality and less emphasis on multi-country (Netherlands) or on the level of detail or presentation (France); reducing the selection weights assigned for work programmes in later years of the programme, to reflect the fact that work programmes would need to change during the programme (Spain); a two staged process dealing first with relevance and then effectiveness (UK); one programme (in Portugal) suggested that the Commission delegate to the MSs the ability to agree changes in contracts. POs and competent bodies also repeated their suggestions that there was some duplication in monitoring and evaluation reports, as discussed in EQ 8; a competent body (in the UK) mentioned that the criteria ‘Quality of Presentation’ in the scoring grid is redundant, since it is addressed in all the other criteria. The competent body wondered whether this criteria is used as a way of assessing the experience of the proposing and implementing organisations, which is considered as an eligibility criteria; Two programmes (in France and Portugal) reported that the need for all visual material to be approved by the MS (including all MSs, for multi-country programmes) was particularly limiting; One programme (in Portugal) suggested that it should not be necessary to present original receipts for all expenditure and the French competent body also suggesting simplifying the control of invoices; The UK competent body suggested that the ability to propose follow-up programmes helps to streamline the process as proposing organisations are already familiar with the procedures and the practices for reporting have already been established; From the framework evaluations, the 2008 USA-Canada evaluation suggested that it could be simpler and more efficient to shortlist a small number of implementing organisations, to simplify the tendering process and to ensure that organisations are experienced in following the regulations. The 2007 fruit and vegetables evaluation considered the data required (e.g. the accurate cost-output ratio of media actions over 3 years) for evaluating the programmes at the moment of sending them for approval to be too detailed). Final Report November 2011 Page 181 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Similarity of Requirements with other CAP Promotional Measures. Few of the respondents in competent bodies or proposing organisations were familiar with detailed implementation requirements of other CAP programmes involving promotional activities. In Italy, the competent body considered the implementation requirements to be a model of clarity, which they sometimes apply to other promotional activities funded under other regulations. However, in both France and Spain, the competent bodies consider that the selection procedures for the I&P programmes are more complicated than for other CAP measures. The CMO for Fruit and Vegetables is based on producer organizations (PO) that form the focal point of the CMO. POs submit operational programmes for a 3-5 years period. These programmes contain several measures, including i.e. investment, quality, environment and promotion of products (art. 103c Reg 1234/2007). Operational programmes are approved as a whole with all their measures including potential promotion. Managing these operational programmes is reported to be easier and more flexible from involved stakeholders (competent bodies in France). Expenses are paid once they have taken place based on presentation of invoices and work schedules are produced each semester, rather than each trimester. In addition, collective trademarks may be used. However, the competent bodies note that the average annual budget dedicated to promotion much smaller, around €65,000 in France. Competent bodies involved in the wine CMO (France, Spain) do also consider the management of promotion less time consuming with a single selection procedure; simpler and less constraining guidelines (“the message shall contain a reference to the intrinsic qualities of the product and must comply with the applicable rules in third countries to which they are intended”), halfyearly reports, possibilities of modifying activities and cost with only a notification and justification to the Administration To what extent are the administrative requirements – selection procedure and implementation requirements – adequate to ensure that information and promotion programmes are selected and implemented in a transparent manner and based on relevant criteria? Is there any room for simplification of administrative requirements without compromising the necessary monitoring and evaluation requirements? Evaluative Judgement There is a two-step selection process, at the level of Member States (pre-selection) and at the level of the Commission (selection). There are a number of supporting documents about the design of programmes and for selection guiding applicants and Member States, primarily the Regulations but also operational guidelines available on the website of DG Agri. Member States check eligibility and quality of the proposals. For this latter purpose, most Member States have adopted a grid-based selection system that reflects the guidelines in the Regulations. Many competent bodies provide assistance to proposing organisations and the experience of competent bodies has an important impact on the quality of Final Report November 2011 Page 182 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC proposals. During the period under review, some Member States did a careful preselection including support to applicants to respond to the Commission requirements while others fail their responsibility of pre-selection and just handover most proposals received leaving the responsibility of selection to a more distant body. There is some duplication because both MS and the EC assess eligibility and quality of the proposals. Selection at the level of the Commission is based on eligibility check followed by a quality assessment based on four main criteria: quality, European dimension, impact assessment and cost-effectiveness. Two internal readers (and sometimes a third external one) are appointed to assess the proposals. Their assessment is based on a more detailed scoring system based on a much larger set of sub-criteria (described in the detailed answer). Out of the requirements to fulfil in the selection process, one of the most criticised ones is the need to detail unit costs of activities in the final years of programme implementation. Such costs (pitch rental in €/m² for a fair in Hong Kong in 2013) are difficult to estimate and could be subject to changes if the programme is adapted underway. As a matter of fact, there has been a high rate of rejection at the Commission level (57% from July 2006 to end of 2010, not decreasing over time), not due to budgetary constraints because the total annual budget has often not been fully allocated. In order to reduce the confusion and delays associated with the two-phased selection process, many interviewees suggested placing a clearer leading selection role either in MSs or in the Commission. Despite the above-mentioned difficulties, the Commission endeavours to make use of standardised selection criteria and objective procedures. Main weaknesses are unclear or inconsistent activities, and lack of details in actions proposed and unclear budget and insufficient strategy especially for third country programmes. Regarding implementation requirements, these are available to proposing organisations and competent bodies through guidance provided by the EC Regulations, guidelines and working documents. There were mixed opinions on the efficiency of implementation requirements. Some proposing organisations considered them too heavy (too much detail is required) and time consuming (due to details and frequency). Comparisons were made with other CAP measures (Fruit and vegetables and wine under the CMO which require only half-year reporting and rural development programmes which require only annual reporting) which require less frequent reporting. The procedures do not allow sufficiently flexible response to the fast moving world of marketing. Multi-country programmes were particularly constrained by the implementation requirements which have to be duplicated in each MS. Suggestions (which are further detailed in the closing chapter) to simplify the administrative requirements consist in a revision of the distribution of roles between the EC and competent bodies in the selection process, a reduction of the reporting frequency and a reduction of the required level of detail in unit costs of programmes (for later years) in programme proposals. Final Report November 2011 Page 183 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 4.4 Replies to evaluation questions of theme 3 – Coherence and complementarities with other CAP and national and private promotion initiatives 4.4.1 Evaluation question 10 To what extent have the information and promotion programmes had complementary or synergetic effects on national or private promotion initiatives? Rationale and approach The question aims at verifying the extent to which complementarities and synergies between EU co-funded programmes and national or private campaigns on agricultural products and related themes have existed, with a view to boosting the product image in consumers’ eyes and increasing demand for these products. Specific attention was paid to the way EU co-funded programmes impact on the implementation of national strategies for agricultural products. The question is covered by two judgement criteria (Table below). Table 55: J.10.1 I.10.1.1 I.10.1.2 J.10.2 I.10.2.1 I.10.2.2 I.10.2.3 I.10.2.4 I.10.2.5 I.10.2.6 EQ 10 Judgment criteria and indicators Synergies between EU co-funded programmes and national/private sector promotion initiatives have been promoted prior to the approval of programme proposals Regulations/calls for proposals/selection procedures/programming documents refer explicitly to the promotion of complementarities and synergies between EU co-funded programmes and national/private sector promotion initiatives at either Member State or EU level There are no constraints or difficulties in developing complementarities/synergies between EU co-funded programmes and national/private sector promotion initiatives for agricultural products Potential complementarities and synergies between EU co-funded programmes and national/private sector promotion initiatives have been exploited The Member State or professional organisation has an information and promotion strategy for agricultural products at a general level and/or by agricultural product and/or country Programming documents identify potential complementarities at Member State and EU levels The EU-co-funded programmes support and reinforce the national promotion strategy for agricultural products at general level and/or by agricultural product Stakeholders are able to identify complementarities and synergies between EU co-funded programmes and national/private sector promotion initiatives Stakeholders consider that EU co-funded programmes are implemented in such a way as to maximise complementarities/synergies with national/ private promotion initiatives There are no examples of overlaps at either Member State level or EU level Source: ADE. Final Report November 2011 Page 184 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The data collected for this question tend to corroborate the evidence found in other parts of the study and no major contradictions were found. The main limitation in the data is the lack of objective evidence for synergies and complementarities as opposed to reported perceptions. Detailed Response J.10.1 Synergies between EU co-funded programmes and national/private sector promotion initiatives have been addressed prior to the approval of programme proposals There is explicit reference to complementarities and synergies between EU cofunded programmes and national/private sector promotion initiatives at either Member State or EU level in the Regulations/calls for proposals/selection procedures Commission Regulation 501/2008 refers explicitly to the promotion of complementarities between EU co-funded programmes and national/private sector promotion initiatives in relation to programmes aimed at third countries (recital 11 and Article 9(5)). The Member States must ensure that the pre-selected co-funded programmes are consistent with and complement national and regional programmes. The regulation also states that “preferential criteria must be laid down for selecting programmes so as to optimise their impact” (Recital 11). Council regulation 3/2008 also explicitly refers to complementarities stating that “such a policy usefully supplements and reinforces schemes run by Member States... Such action, by helping to open up new markets in third countries, is also likely to have a multiplier effect on national and private initiatives” (Recital 4). The “Guidelines to be considered when assessing and managing part-financing programmes for promoting Community agricultural products”86 state that “the complementarity of programmes funded by Member States, private sector programmes and programmes that are part-financed by the European Union should be clarified in order to increase synergies between the various programmes and to ensure that they reinforce one another effectively”. Complementarities/synergies exist with national programmes There are no public national programmes on the internal market or in third countries initiated by the competent bodies in four of the 23 EU Member States that responded, namely Denmark, Germany, Latvia, and Poland, thus the question of complementarities and synergies is irrelevant here. In seven EU Member States out of the 23 that responded, namely France, Ireland, Italy, the Netherlands, Portugal, Spain and the UK, there are national programmes and complementarities and synergies are reported. Some of these national programmes concern products (e.g. the promotion of fruit and vegetables, olive oil, quality meat, milk, PDO, PGI and organic products); others concern national health campaigns (e.g. the national health and 86 Directorate D. Direct support, market measures, promotion, D.4. Promotion of agricultural products, Brussels, 1 July 2008, MO/cm D(2008)-AGRI/60787/2007-rev. 5 Final Report November 2011 Page 185 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC nutrition strategy of the Ministry in the Netherlands) or regional and sectoral initiatives and participation in trade fairs. There are also national tax credits for certain promotional activities which can be considered complementary to the EU co-funding. Italy provides an interesting example of an attempt to coordinate EU, regional, provincial and municipal activities with the Communication Annual Plan carried out by Ministry of Agriculture. This plan provides a guideline for the institutional communication between EU, central and peripheral levels to design and implement promotional campaigns. In this way, the Ministry of Agriculture wants to reduce redundancies and overlaps in institutional promotional initiatives. There is however no specific budget for these coordination efforts. Complementarities/synergies with private initiatives are observed in around half of the MS responding to the online survey but little concrete information is given There are few explicit references made to complementarities and synergies between EU co-funded programmes and private initiatives in the survey. In some Member States (9MS out of the 23 respondents - Belgium, Denmark, Finland, Germany, Greece, Latvia, Luxemburg, Malta, Portugal) the competent bodies see few or no complementarities, in others (10MS out of the 23 that responded - Sweden, Austria, Cyprus, Spain, Italy, the Netherlands, France, the Czech Republic, Poland, the UK) complementarities are observed or assumed but there is little concrete information available, e.g. the Swedish competent body mentioned that “there might be complementarities if an EU co-funded programme runs at the same time as a private initiative”. In France, there are complementarities reported with actions led by private inter-trade organisations. No apparent constraints or difficulties in developing complementarities/synergies between EU co-funded programmes and national promotional initiatives are noted The vast majority of the POs and competent bodies interviewed in the Member States did not identify constraints to developing complementarities/synergies between EU co-funded programmes and national promotion initiatives for agricultural products. Constraints or difficulties in developing complementarities/synergies between EU co-funded programmes and national or private promotional initiatives are due to restrictions on the mention the origin or the (collective) brand More constraints were reported in developing complementarities and synergies between EU co-funded programmes and private initiatives, mainly on account of the prohibition of referring to brands in the EU co-funded programmes. E.g., there may be a certain confusion when producers participate in the name of their brand in the same fairs as the I&P programmes where their products are present in a generic way. In addition, producers often do not see the interest of generic campaigns and are not prepared to invest in them. Indeed, generic promotion of fruit and vegetables for example will benefit to all products available to the consumer, from all EU Member States but also from third countries, which is somehow contradictory with a competitive approach in use on free trade markets (specific aspects of generic promotion are further developed in EQ 11. Final Report November 2011 Page 186 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 85F J.10.2 Potential complementarities and synergies between EU co-funded programmes and national/private sector promotion initiatives have been exploited From the online questionnaire, 13 EU Member States out of the 23 that responded declare having a strategy at general policy level. These are Austria, Cyprus, the Czech Republic, Finland, France, Greece, Ireland, Italy, Luxembourg, Malta, Slovakia, Spain and the UK. 10 EU Member States namely Belgium, Bulgaria, Denmark, Estonia, Germany, Latvia, the Netherlands, Poland, Portugal and Sweden either do not have a national I&P strategy or have not responded. (Source: Online survey, ADE, 2011). From framework contract evaluations, the organic evaluation in 2006 underlined the complementarities between the Austria’s national strategy and EU co-funded programmes. The report also states that most countries in the case studies had or were elaborating global strategies for the organic sector. There was however no systematic approach in terms of synergy. (Source: organic evaluation report, p 78, 85) 86F87F From the online questionnaire, 8 of the 23 respondent Member States declare having a strategy at product level. These are Austria, Estonia, France, Greece, Malta, Poland, Slovakia and Spain. The majority of Member States stated that such a strategy did not exist in their case. (Source: Online survey, ADE, 2011). The table below recapitulates these lists of Member States. Table 56: List of Member States that declare having a national promotion strategy at general or at product levels. Promotion Strategy at General level AT CY CZ FI FR EL IE IT LU MT SK ES UK Promotion Strategy at Product level AT EE FR EL MT PL SK ES Source: ADE based on online questionnaire According to the milk evaluation report, only Belgium and Finland had a specific national or regional strategy for milk and milk products. In Germany, there was no milk strategy but there was a centralised follow-up for all agricultural products and institutions are mandated to develop responses to market evolutions. In the UK and Ireland, milk strategies exist but at the initiative of the milk marketing boards. (Source: milk evaluation report) Final Report November 2011 Page 187 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC EU-co-funded programmes provide support and reinforcement of the national promotion strategy for agricultural products at general level and/or by agricultural product Out of the 13 Member States that have national I&P strategies, 12 confirm that the EU cofunded programmes support and reinforce the national promotion strategy for agricultural products. These include building on research (Ireland), coordination of promotional initiatives targeting the internal and foreign markets (Italy), smaller inter-trade organisations (with limited financial resources) benefiting from the EU co-funded programmes which are seen to fill a gap left by national instruments (France), EU co-funded programmes complementing national health campaigns (the Netherlands), ensuring continuity in campaigns through successive EU cofunding (Portugal), supporting promotional activities and promoting products which would not otherwise be promoted (Estonia, Belgium). Complementarities and synergies between EU co-funded programmes and national/private sector promotion initiatives are identified by POs and competent bodies All the stakeholders interviewed both at the level of proposing and competent bodies are aware of national and European funding of the specific sector to which they belong. Almost two third of respondents to the questionnaire considered that complementarities and synergies between EU co-funded programmes and national/private sector promotion initiatives could be identified. Mentioned complementarities and synergies include: POs using different funding sources including private and EU sources for different campaigns (IT-INT, ES-OIA, FR-CNI) Market analysis leading to better programmes through a better knowledge of the sector (ES-OIA) EU co-funded campaigns supporting national campaigns for certain products, e.g. for olive oil (ES-OIA) or for milk (DE, IE, NL), whereby according to the milk evaluation report, most synergies were coincidental EU co-funded programmes complementing the campaigns carried out by the PDO Regulatory Boards and collaborating with PDO Regulatory Boards in industry fairs and other campaigns EU co-funded programmes producing multiplier effects by informing on the quality of European products (CZ) Same publicity material being used by both public and private campaigns (PL) From the framework contract evaluations, according to the organic evaluation report, stakeholders considered collaboration with the private sector to be hampered by their differences of views on commercial strategy and the fragmentation of organic shops. But the report mentions that stakeholders had exploited synergies with some private initiatives (e.g. in a Danish and Austrian retailing). Most stakeholders interviewed considered that cofinanced programmes created a leverage effect on the amounts of national funds allocated and on supporting national or private activities. (Source: organic evaluation report) According to the fruit and vegetables report, despite the rather limited references to synergies in programming documents, most stakeholders were able to identify potential Final Report November 2011 Page 188 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC synergies between co-financed programmes and MS initiatives and/or private initiatives. Concrete examples of such synergies were found in terms of goals (healthy food, food quality etc.) and messages conveyed (e.g. health message “5 a day”), but also in terms of target groups (e.g. schools) and channels (e.g. common web-portal). By far the most controversial and sensitive were the potential synergies with the private sector (for which only very few examples could be found). (Source: Fr & Veg sector evaluation) Perception of POs and competent bodies regarding the implementation of EU cofunded programmes in such a manner as to maximise complementarities/synergies with national/ private promotion initiatives The opinions on whether the EU co-funded programmes are implemented in such a way as to maximise complementarities/synergies with national/ private promotion initiatives diverge. A number of stakeholders do not consider the implementation optimal to maximising complementarities and synergies but rather limited or ad hoc. The fruit and vegetables evaluation finds that the development of synergies is actually quite difficult. Regarding the private sector, developing synergies with the private sector (e.g. re-use of logos and slogans, trademarks, etc.) is reported to be difficult under the current Regulation. (Source: Fr & Veg sector evaluation) However, the EU programmes can act as ‘incubators’ for future synergies and some seem to have achieved synergies already (e.g. IT-INT, IT-AIA, IT-UIS, IT-UIV, FR-UGP). Some POs and competent bodies consider that the synergies are not a result of the implementation of the EU programmes but of the similarity in terms of markets, targets and messages between EU and national/private initiatives. The EU programmes are considered to be consistent with the national programmes in that they are pursuing similar targets. Overlaps between EU co-funded programmes and national/private initiatives are not reported as an issue According to the questionnaire, overlaps between programmes do not appear to be an issue in many of the Member States except maybe in the context of individual events. Only a few Member States (Austria, Estonia, Ireland, Slovakia, the UK) felt that there was any risk of overlap. Reasons given were the similarity in target groups or activities. In Italy, the Ministry of Agriculture is working on a monitoring database to control and avoid overlap. This database will also contain a calendar in which all promotional initiatives implemented at European, national and regional level will be reported. In Spain, there was one example of (avoided) overlap in the promotion of olive oil. According to the proposing organisation (IAOE), this overlap was due to the fact that the national programme was approved before the EU co-funded programme. The solution was to concentrate the national campaign on the October-January period and the EU programme after February and to have different focuses of messages (consumption in the national programme; image and feelings in the EU one). In their view, both messages are complementary. The general view seems to be that information and promotion campaigns tend to reinforcing each other rather than overlapping, particularly when specific adaptations are made. Final Report November 2011 Page 189 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC To what extent have the information and promotion programmes had complementary or synergetic effects on national or private promotion initiatives? Evaluative Judgement Council Regulation (EC) N° 3/2008 and Commission Regulation (EC) No 501/2008 do both explicitly refer to complementarities between EU co-funded programmes and national/private sector promotion initiatives both on the internal market and in third countries. This explicit reference is included in guidelines to applicants and in the selection criteria but is not further elaborated (no further information is provided neither in the preselection documents from MS nor in the identification form of the EC or in the programme proposals). Around half of the Member States that responded to the online survey (15/23) declared having an information and promotion strategy for agricultural products at a general level and/or by agricultural product (or for some products). The vast majority of the stakeholders interviewed in the Member States could see no constraints to developing complementarities/synergies between EU co-funded programmes and national promotion for agricultural products. Almost all Member States that have national promotion strategies provide elements of reinforcement including in terms of research, coordination between internal and foreign markets, filling the gap of national funding, complementing health campaigns. However, although it seems that the EU co-funded programmes support and reinforce national promotion strategies, it was thought that the synergies were often coincidental. Concrete examples of such synergies were in terms of promoting similar goals (e.g. healthy food, food quality etc.), conveying the same messages (e.g. ‘5 a day’) and in reinforcing promotion on a target group (e.g. schools) and reusing channels (e.g. common web-portal). Two major constraints were identified in developing complementarities/synergies with private promotion initiatives, namely the restrictions on the mention of origin and the branding issue. Indeed, generic promotion benefits to all producers (one can say competitors) of the concerned products, be they European or even from third countries, which is somehow contradictory with investment from the concerned private sector. In some cases the re-use of materials (re-use of logos, slogans...) developed in common within the framework of a programme by the private sector having participated turned out difficult. Overlaps between EU co-funded programmes and national/private initiatives are not reported as an issue. Final Report November 2011 Page 190 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 4.4.2 Evaluation question 11 To what extent have the information and promotion programmes created EU added value? Rationale and approach The question aims at assessing the value added of EU intervention in information and promotion of agricultural products as compared to the value that would have resulted from potential programmes at regional and national levels by both public authorities and the private sector. The question is covered by three judgement criteria (table below) under which financial and non-financial dimensions have been analysed: Table 57: J.11.1 I.11.1.1 I.11.1.2 I.11.1.3 I.11.1.4 J.11.2 I.11.2.1 I.11.2.2 J.11.3 I.11.3.1 I.11.3.2 EQ 11 Judgment criteria and indicators EU financial contributions enabled information and promotion programmes to be implemented that would not otherwise have been undertaken or that would have been noticeably reduced Stakeholders state that information and promotion programmes would have been different without co-funding in terms of messages conveyed, channels used, target groups or objectives pursued Stakeholders state that without co-funding, programmes would have had a smaller budget or would have not been designed and implemented Stakeholders state that programmes that were rejected have not been implemented or have been implemented in a noticeably different way. Complementary programmes have been carried out by Member States, regional authorities, third countries, or the private sector at the same time as or following the co-fundedprogrammes Information and promotion programmes led to increased exchange of experience and know-how on information and promotion of agricultural products at a reasonable cost Stakeholders state that transnational meetings and conferences have been organised to exchange experiences on information and promotion programmes (e.g. in the framework of multi-country or multi-product programmes) Stakeholders consider that the costs of coordinating transnational programmes are reasonable compared to the national ones The image and the knowledge of EU agricultural products is improved The promotion of the intrinsic values of agricultural products through co-funded programmes have improved the image and the knowledge of these products (i.e. the EU origin) Stakeholders state that the co-funded programmes have led to the improvement of the image and the knowledge of promoted agricultural products Final Report November 2011 Page 191 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Detailed Response J.11.1 EU financial contributions enabled information and promotion programmes to be implemented that would not otherwise have been undertaken or that would have been noticeably reduced EU funding has an important leverage effect enabling generic, multi-country and multi-product programmes to take place Generic versus brand-oriented Most interviewed POs and competent bodies of the sampled programmes consider that without co-funding, programmes would either not have been implemented at all or only small parts of them. They would have been of a much smaller scale and probably only financed by the private sector (PO), in which case they would have been brand-oriented. Not surprisingly, the private sector prefers investing in its own specific products. On the other hand, messages, channels, target groups and objectives would have been the same without co-financing, with few exceptions reported (messages more linked to health benefits for milk in NL and not multi-product nor multi-country in IT, PT, FR). In the online questionnaire, around 28% of the respondents stated that promotion programmes not focusing on brands would exist without EU co-funding. 40% stated that they would not or only to a very limited extent (see figure below). Figure 19: Response from the online survey about EU added value Source: Online survey, ADE, 2011 Final Report November 2011 Page 192 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC According to the current guidelines of State Aid in the agricultural sector, national (publicly cofounded) programmes are submitted to the same restrictions as for EU co-funding under Council Regulation N° 3/200887. Indeed, the reported national promotion programmes do not focus on brands, e.g. Ireland, Spain, Estonia and Poland. In Ireland, Bord Bia is state-funded and the National Dairy Council is funded by levies from the trade. Promotion programmes in Denmark are mainly financed by the organisations thanks to received para-fiscal charges or mandatory contributions but also voluntary contributions. In Sweden, organisations can give general information about for example milk, meat or fruit. 8F In other countries, the promotional activities tend to be brand-focused but only with private funds, e.g. UK organisations like to promote brand-orientated products with the exception of an industrial body sponsored by the government (Agriculture & Horticulture Development Board) to promote generic produce from certain agricultural sectors. In Luxembourg, investing in generic promotion is considered costly just for ‘anonymous products’. Whether or not such generic programmes are carried out depends on the availability of financial resources in Italy. In the Czech Republic only the informational campaigns realised by National Health Authority are not brand-oriented. In Cyprus the companies would not undertake generic programmes if they were not co-financed as they do not always see the direct benefit. Source: Online survey, ADE, 2011 In cases where generic campaigns would have gone ahead without EU co-funding (FRUGP, FR-CNI, NL-CNI), their budget would have been sensibly reduced. In particular, the transnational aspects of the programmes would have been restricted as the proposing organisations would have had neither the budget nor the know-how to implement such campaigns with private funds (IT-UIS). The EU co-funding brought in a networking dimension which is an important added value (DE-TAG). The added value of generic promotion campaigns is seen in the possibility to provide the public with general information or raise awareness on products that are beneficial for health (in some countries like Germany this is a condition for the I&P campaign). In this sense, the I&P campaigns benefit to producers with regard to the notoriety of their product, in particular in cases where funds are lacking to carry out campaigns. The generic campaigns also improve the image of the EU as a producer of quality products. 87 Aid for advertising agricultural products may be authorised if the advertising campaign is centred on quality products - recognised EU designations (such as registered designation of origin, PDO and PGI) or for national or regional quality labels. Advertising campaigns must not be dedicated directly to the products of one or more particular company or companies. References to product origin are authorised if they correspond exactly to those references registered at EU level as recognised EU designations, or if they are subsidiary in the message for national or regional quality labels. Direct aid may not exceed 50 %. If the sector covers at least 50 % of the costs, the gross intensity of aid may reach 100 %. Generic advertising (without reference to product origin) is eligible for aid at 100 %. Lastly, if the annual budget of the advertising campaign exceeds EUR 5 million, it must be notified separately. Final Report November 2011 Page 193 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Single product versus multi-product promotion According to the online questionnaire, 12 out of the 23 MS claimed that promotion programmes focusing on one single product would occur to a large extent or to some extent without EU co-funding. 9 Member States said they would not occur or only to a very limited extent and 4 did not reply. In some countries (Spain, Poland) specific products (e.g. olive oil, meat, etc.) are promoted individually. In Italy, this is the case for sectors in crisis. In the Netherlands, the single products promotion campaigns are much smaller than multi-product campaigns and in Sweden they are rare unless an organisation deals with only one product, for instance honey. In the Czech Republic, the only single product programmes are the informational campaigns realised by the National Health Authority. In Finland, single products are only promoted as brands. Instead, promotion programmes usually target specific aspects of a particular food group, such as nutritional value of dairy produce, seasonality of fruit and vegetables, or standards of quality assured produce (IE). National labels (available to every producer in every selling point) benefit from financial support in Luxembourg. In Latvia, the financial resources are so limited that it is difficult to implement promotion actions without EU co-funding. Source: Online survey, ADE, 2011 Multiproduct programmes only for relevant combinations According to the online questionnaire, seven Member States have national promotion programmes focusing on several products at the same time (AU, DE, DK, ES, IE, LU, and MT) while eight do to a limited extent and eight not at all. National programmes dealing with more than one product mostly focus on products with common features like the Mediterranean Diet products. The Irish dairy promotion programme is sector-specific and includes different products such as milk, yoghurt and cheese. This is also the case in the DK. In Luxembourg, some labels covering several products benefit from financial support. In Italy, promotion is limited to a few producers’ initiatives to improve the image of their products. In Finland, multi-product promotion only happens when brands are allowed. Source: Online survey, ADE, 2011 Almost no multi-country programmes without EU co-funding According to the online questionnaire, only two Member States felt that multi-country programmes would occur without EU co-funding (IE, UK). 22 thought that they would not or only to a very limited extent. IE and UK further explained that these programmes would have existed but to a lesser extent, and with lower budgets (IE). Stakeholders in the UK felt that multi-country programmes rely on public funding as they have little commercial interest. Multi-country programmes do however provide a significant leverage effect, as the share of funding by the proposing organisation (20%) is divided among all participants. Final Report November 2011 Page 194 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC EU co-funding expands size and coverage From the framework contract evaluations, the third country reports also state that the programmes would not have reached or targeted those countries without the EU support. Five out of nine organisations involved in the India, China and S-E Asia evaluations stated that they would never have targeted those countries without the EU scheme. This trigger effect occurs mainly for small organisations which do not benefit from specific resources (levies, member fees, etc.) or government funding. The framework contract evaluation on fruit and vegetables points out that EU co-funding contributes significantly to enlarging and reinforcing campaigns at national, regional or product level that, in the absence of co-funding, would have been much smaller and different in terms of (1) messages (e.g. oriented to trademarks, brands and specific origins), (2) target groups and (3) information channels used (e.g. no TV-spots due to their high implementation costs). According to the evaluation, the added value of EU co-funding lies in the capacity to carry out large-scale programmes addressing a wide audience with expected high impact. It also contributes to generic campaigns and is essential for multicountry programmes that otherwise would simply not exist, as well as for some singlecountry programmes (e.g. Spain). Rejected Programmes According to interviews with POs some of the rejected programmes at EC level tend not to be implemented (DE-TAG, PL-SRW, UK-MMF, ES-OIA) unless they are improved and re-submitted. In other cases, reduced versions of the programmes are implemented (NL-CNI, FR, and FR-INT). Both cases exist in Portugal. Complementary programmes are carried out by Member States, regional authorities, third countries, or the private sector Private follow-ups to the programmes are likely in Greece (EL-EAS, EL-SEK). In Italy, a number of campaigns at national and regional level are running simultaneously but it is unclear as to whether they are coordinated and complementary between each other. The same is true for Spain. According to one French interviewee, the PO has a series of programmes taking place at the same time according to an approach which is not opportunist but strategic: the EU cofunded programmes are following the French strategy and not the other way round. The PO’s other programmes target different groups or promote different products. In Germany, Poland, Portugal and the UK, no complementary programmes were carried out. In the UK, the overall rise in confidence in dairy products boosted marketing activities. J.11.2 Information and promotion programmes led to increased exchange of experience and know-how on information and promotion of agricultural products at a reasonable cost Transnational meetings and conferences Meetings have been organised among PO in order to prepare multi-country programmes. These transnational meetings are done to elaborate a common programme and agree on messages, channels, etc. Exchange of experience takes place but is not the main part of the Final Report November 2011 Page 195 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC meetings (FR- and NL-CNI, Portuguese partner of the “Discover the Origin” programme between France, Italy and Portugal). Specific actions to promote the exchange of experiences were not organized between countries (IT-UIS). Nevertheless, a minority of POs state that the transnational meetings and conferences are an appropriate system to exchange experiences and information among the partners involved in I&P programmes (IT-UIS). Costs for coordinating transnational programmes are reasonable compared to the national ones The cost for coordinating transnational programmes is necessarily higher than for national ones (meetings of PO, translation...). These costs are considered high by some of the proposing organisations and the coordination was not always efficient due to misunderstandings between organisations involved and the effort needed to reach common views. In the view of some POs and competent bodies, the costs of coordinating multicountry programmes are so high that they would not be implemented if the EU did not actively encourage them. The example of France shows that even with the 2% extra for overheads, this does not leave much for the organisation of transnational meetings, translation etc. with overheads varying approximately between €10,000 to around 150,000€. However, feedback from a proposing organisation insists on the fact that the leverage effect of having several of sources of proposing organisations funding can more than compensate these costs. J.11.3 The image and the knowledge of EU agricultural products is improved The information and promotion programmes were (according to some of the framework contract evaluations and interviewed competent bodies) thought to improve the image of EU products and contribute to the development of positive perceptions (e.g. high quality, safety, taste, etc). The Greek, Italian, French, Portuguese and Polish Competent Bodies are of the opinion that promoting products with the EU-label adds to their attractiveness by improving their image in terms of quality and safety as well as authenticity. This is particularly true for products produced under quality schemes (PDO, PGI, TSG, and organic products) underpinned by specific production methods, nutritional and health aspects, labelling, animal welfare and environmental benefits. European certification is seen by the consumers as an umbrella brand that influences positively their purchasing behaviour. This is not the case in the UK where the unpopularity of the EU would risk harming the product in the opinion of the POs. POs and competent bodies also consider that multi-country programmes improve the image and knowledge of the targeted products efficiently because they reach larger target groups. This is confirmed by post-test actions, about the impact of the programmes and estimated target audiences. From the framework contract evaluations, promoting the intrinsic value of the products also has a positive effect overall on consumer behaviour. According to the fruit and Final Report November 2011 Page 196 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC vegetables evaluation report, information and promotion measures have contributed to improving the image of fruit and vegetables. (Source: Fr & Veg sector evaluation) In the opinion of an interviewed PO, the restriction to mention the brand and origin in the co-funded programmes weakens the programme impact on the improvement of the image/knowledge. For example, consumers in certain target markets are very sensitive to the origin of the products (New-York and Italy). Other stakeholders consider that the restriction to mention the brand and the origin in the EU co-funded generic campaigns also benefits to third country competitors of the EU. There is a lack of clear evidence as to the direct economic impact (e.g. sales and exports) of the generic campaigns, working on the image and awareness. However, the available data (e.g. export data for European apples or meat) suggests that the sales have risen as a result of some campaigns. Proposing organisations in some programmes (e.g. PT-BIV and PTFEN, FR-UGP, ES-OIA, UK-MMF) are confident that the campaigns have raised the image and knowledge of the products and thus their sales. The fruit and vegetables evaluation suggests that in terms of image, there are indications from closed programmes and surveys that information and promotion campaigns impacted positively on the image of fresh fruit and vegetables among young consumers, as well as among teachers and doctors as primary target groups. (Source: Fr & Veg sector evaluation) To what extent have the information and promotion programmes created EU added value? Evaluative Judgement EU funding has an important leverage effect enabling generic, multi-country and multiproduct programmes to take place. Without EU co-funding, programmes would be of a much smaller scale and funded mainly by the private sector (PO); in such a case they would be brand-oriented and not generic. Some programmes would not take place at all especially in third countries, where budgets needed are not affordable otherwise (and a reduction of size would make the programme unrealistic). The vast majority of Member States felt that multi-country programmes would not occur or only to a very limited extent without EU funding. Although competent bodies see a value added in these programmes, they are not always a priority at national level in a number of countries and their interest is sometimes questioned given their cost and the complexity. This criticism should not conceal the indisputable EU added value that such multi-country approaches potentially have by nature. On the contrary, reported difficulties to put them in place also reflect the fact that Member States still have slightly different rules even in an integrated EU market and different interpretations of the regulatory guidelines (Annex 1 of Commission Regulation (EC) No 501/2008). Despite the rather low occurrence of synergies being mentioned between national and EU-supported campaigns in programme documents and evaluation reports, most Final Report November 2011 Page 197 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC interviewees were able to identify existing synergies between co-financed programmes and Member State and/or private initiatives. Transnational meetings that were organised by proposing organisations to elaborate multicountry programmes serve mainly to coordinate actions and messages, but also contribute to the exchange of experience. The previous evaluation reports and assessments of the ongoing programmes lead to the conclusion that I&P programmes improve the image of EU products and contribute to the development of positive connotations (e.g. high quality, safety, taste, etc). The specific contribution of multi-country programmes on the image is also recognised by stakeholders. Promoting the intrinsic value of the products also has a positive effect overall on consumer behaviour. Overall there is a lack of clear evidence as to the direct economic impact (e.g. exports and sales) of generic campaigns, working on the image and awareness. However, available data for some specific campaigns suggest a link with increased sales or exports or consumption. 4.4.3 Evaluation Question 12 To what extent have information and promotion measures applied under Council Regulation No 3/2008 been coherent with other CAP promotion measures, notably with: the promotion measures applicable in the wine sector ; the promotion measures applicable in the fruit and vegetable sector ; the promotion measures applicable within the rural development policy ; specific support under Article 68 of Council Regulation (EC) No 73/2009? Rationale and approach The question aims at verifying to what extent different aspects of the information and promotion programmes under Council Regulation No 3/2008 such as their objectives, messages conveyed, groups targeted and channels used have or not contradicted, overlapped or complemented these of other CAP measures. In this question, particular attention is paid to the existence of clear and appropriately defined demarcation lines. The question is covered by three judgement criteria (table hereafter). The response to this question is based on available documentary information and interviews in the EC and in Member States involved in programmes submitted since 2008 and sometimes in other CAP measures. The main documentary sources are: the regulations quoted above, the programming documents, documentation about other CAP measures from DG Agri, Rural Development Programmes (RDPs) from several regions and extracts from Mid-Term Evaluations (MTEs). It should be noted that little information is available at EC level about specific promotion activities carried out under the operational programmes of producer organisations in the fruit and vegetable sector. Final Report November 2011 Page 198 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Table 58: EQ 12 Judgment criteria and indicators J.12.1 The demarcation lines between the different types of information and promotion measures for agricultural products are clear and appropriately defined I.12.1.1 The EU promotion policy framework for agricultural products includes a Programming section about the different other promotion measures for agricultural products (co-) documents and financed under the CAP, their complementarity and coordination evaluation reports I.12.1.2 Promotion activities are co-funded by only one CAP scheme/one specific Programming fund (i.e. the same activity is not financed twice by the EU CAP documents and instruments/specific funds) and EU funds used to promote agricultural products evaluation reports are articulated I.12.1.3 Stakeholders know the different types of information and promotion Interviews, measures (co-) financed under the CAP to promote agricultural products questionnaire I.12.1.4 Stakeholders can easily differentiate between all types of promotion Interviews, measures (co-) financed under the CAP to promote agricultural products questionnaire J.12.2 EU co-financed programmes and promotion measures implemented at the initiative of the Commission avoid contradictions with other CAP promotion measures I.12.2.1 There are no contradictions in terms of objectives, messages conveyed, Programming groups targeted and channels used in programming documents of EU co-financed documents and programmes, promotion measures implemented on the initiative of the evaluation reports Commission and other CAP promotion measures I.12.2.2 Stakeholders do not identify contradictions, in terms of objectives, messages conveyed, groups targeted and channels used, between EU co-financed Interviews, programmes, promotion measures implemented at the initiative of the Commission questionnaire and other CAP promotion measures J.12.3 Potential complementarities and synergies between EU co-funded programmes, promotion measures implemented at the initiative of the Commission and other CAP promotion measures have been promoted and exploited I.12.3.1 Regulations/calls for proposals/programming documents of EU co-funded Related documents programmes and promotion measures initiated by the Commission explicitly refer and evaluation to other CAP promotion measures for agricultural products reports Programming I.12.3.2 EU co-financed programming documents have identified potential documents and complementarities and synergies with other CAP promotion measures evaluation reports Programming I.12.3.3 Promotion measures initiated by the Commission have identified potential documents and complementarities and synergies with other CAP promotion measures evaluation reports I.12.3.4 Stakeholders are able to identify complementarities and synergies between Interviews, EU co-financed programmes and other CAP promotion measures questionnaire I.12.3.5 Stakeholders are able to identify complementarities and synergies between Interviews, promotion measures initiated by the Commission and other CAP promotion questionnaire measures I.12.3.6 Stakeholders consider that EU co-funded programmes are implemented in Interviews, such a way as to maximise the identified complementarities/synergies questionnaire Final Report November 2011 Page 199 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Detailed answer J.12.1 Lines of demarcation between different CAP measures are clear and appropriately defined The EU I&P scheme for agricultural products includes a section mentioning the different other promotion measures for agricultural products (co-) financed under the CAP and vice-versa, but complementarities and coordination are not explicitly mentioned. Drawing boundaries between what is funded under which CAP measure is among the goal of cross-references in the regulations, further competent bodies’ initiatives to establish accurate distinctions between promotion measures, as well as controls under the responsibility of the ministries to avoid double financing. Cross-references in the Regulations In the horizontal regulation for promotion (3/2008) there are several references to other CAP promotional measures under the single CMO and Rural development (and viceversa) 88 to avoid overlaps or duplication of financing, with the exception of the implementing regulation for fruit and vegetables that does not provide such crossreferences (Commission Regulation No 543/2011 laying down detailed rules for the application of Council Regulation (EC) No 1234/2007 in respect of the fruit and vegetables and processed fruit and vegetables sectors) . 89F Some competent bodies introduce further lines of demarcation or specific measures to avoid double financing. Competent bodies’ initiatives to establish further demarcation lines In some cases demarcation lines are also further specified by the competent bodies between certain types of programmes, specific funds or specific selection criteria. For instance in the wine sector, in Italy, the Ministry of Agriculture did not select any technical proposals for wine under Regulation (EC) No 3/2008 because similar initiatives had been implemented under the reformed wine CMO through Regulation (EC) No 555/2008, with the only exception of multi-product programmes where wine was included among a basket of goods under Council Regulation N° 3/2008 were. 88 Council Regulation (EC) No 3/2008 refers to CMOs and to Council Regulation (EC) No 1234/2007; Commission Regulation (EC) No 501/2008 refers to Council Regulation (EC) No 1698/2005 in Recital 14 as follows “to avoid any risk of duplication of financing, I&P measures receiving support under Council Regulation (EC) 1698/2005 should be excluded from support under Council Regulation 3/2008; Regulation (EC) 555/2008, laying down detailed rules for implementing Council Regulation (EC) No 479/2008 on the common organisation of the market in wine makes repeated references to Regulation (EC) No 3/2008. Final Report November 2011 Page 200 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Some Rural Development Programmes introduce further demarcation lines between their measure 133 and Regulation (EC) No 3/2008. For instance, in France promotion of quality schemes under measure 133 of the Rural Development Programmes is managed by French regions and may only concern products from the concerned region. Competent bodies’ specific measures to avoid double financing Some competent bodies seem to be particularly concerned by double funding and want to avoid financing twice the same programme. In Portugal, this is a very sensitive subject for public entities and they have implemented a stamp system with the National Authority logo for all the bills to prevent these situations. In Poland, the Agricultural Market Agency administers two different promotion instruments and checks this issue. Finally, some MSs strongly encourage the use of one fund against another. For instance, in Greece, proposing organisations are invited to focus mainly on Council Regulation N° 3/2008 for promotion policy compared to others CAP measures. Some lines of demarcation exist implicitly between CAP promotion measures mainly due to differences in the types of beneficiaries or markets targeted. This suggests that better distinguishing between target beneficiaries at different levels of the value chain could reinforce complementarities between CAP promotion measures. Below, major differences or similarities at the level of the beneficiaries, the markets covered and the eligible activities are discussed by CAP measure. Fruit and vegetable sector Although there are some similarities for promotion measures in the fruit and vegetables sector between the support provided by the single CMO (Council Regulations (EC) No 1234/2007) on one hand and (EC) No 3/2008 on the other hand, there are differences especially in terms of beneficiaries, use of collective trademarks and selection and implementation of activities (cf. table 5, chapter 1) as shown in the table hereafter. Table 59: Item Beneficiaries Differences between Funds concerned with promotion of fruit and vegetables CMO Fruit and vegetables sector Information and promotion measures Professional Trade/ Branch organisations representing the sector Producer organisations Promotion of trademarks Promotion of the collective trademark of the Only generic promotion is allowed producer organisation is allowed Internal market or Third country markets; Markets however the measures may not be outsourced Internal market or Third country markets targeted by the Producer organisation outside the EU Place of Promotion measures are part of an operational Call for proposals: Trade or inter-trade promotion programme, which includes also support to organisations submit a programme to MS measures other activities (e.g. production, quality, etc.). Source: ADE 2011 based on the analysis of Regulation (EC) No 1234/2007, Regulation (EC) No 3/2008. The type of beneficiaries supported differ: Regulation (EC) No 3/2008 concerns trade (or inter-trade) organisations representing the sector in one or more Member States (or at Final Report November 2011 Page 201 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC EU level), while the single CMO fruit and vegetables allows funding of promotion for producer organisations and associations of producer organisations. Trade or inter-trade organisations are expected to be representative of the fruit and vegetables sector while producer groups are legal bodies formed by the farmer level of the value chain. Regarding the promotion of trademarks, only generic promotion is allowed under Regulation (EC) No 3/2008 while the CMO allows for the promotion of the collective trademark of the producer organisation. Note that the promotion measures under the CMO are part of an operational programme, which includes also support to other activities (e.g. production, quality, etc.) while the promotion programmes under the Regulation (EC) No 3/2008 are only concerned with information and promotion measures. Wine sector In the frame of the reform of the wine CMO of 2008, measures for the promotion of wine on third country markets were integrated into the eligible measures, instead of market interventions; the objective being to strengthen the competitiveness of the sector (see descriptive chapter (§1.1.2 Wine)). The differences between support to the promotion measures applicable in the wine sector under the reformed CMO (Regulation (EC) No 1234/2007) and under Council Regulation (EC) 3/2008 mainly concern the type of beneficiaries, the markets targeted, the possibility to mention brands, as well as duration and the average budgetary allocation (cf. table 9 of chapter 1). Table 60: Item Beneficiaries Differences between Funds concerned with promotion of wine CMO Wine sector Producer organisations Private companies Inter-branch organisation Professional organisation Public entities Information and promotion measures Trade or inter-trade organisations Inform consumers about the variety, EU quality and production conditions of EU wines and about responsible drinking pattern Internal market or Third country Markets targeted Third country markets markets Promotion measures shall not be brandOrigin of the product may be oriented or encourage the consumption of indicated as part of an information or a product on grounds of its specific origin. promotion operation in the case of Mention of However, the origin may be indicated in wine with a geographical indication. origin or brands the case of designations conferred under Indication of individual or collective Community rules. The origin may also be brands is possible as long as no State mentioned if it is secondary to the main aid is involved message of the programme. Place of Call for proposals: beneficiaries apply Call for proposals: Trade or inter-trade promotion to the promotion measure inside the organisations submit a programme to MS National Support Programme measures Source: ADE 2011 based on the analysis of Regulation (EC) No 1234/2007 and Regulation (EC) No 3/2008. Purpose of the measure Final Report Improve competitiveness wines in third countries of November 2011 Page 202 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Concerning the type of beneficiaries, inter-branch organisations are eligible under both regulations, but the CMO allows other types of beneficiaries such as producer organisations, private companies, professional organisations and public entities. The purpose of the measure under the CMO is to improve competitiveness of EU wines in third countries while the objective of Regulation (EC) 3/2008 is to inform consumers about the variety, quality and production conditions of EU wines and about responsible drinking pattern on the internal market and in third country markets. Under the CMO, promotion measures are part of a National Support Programme. One major difference is that the origin of the product may be indicated as well as individual or collective brands under the CMO while under Regulation (EC) 3/2008, promotion measures shall not be brand-oriented or encourage the consumption of a product on grounds of its specific origin. Rural Development Policy The support to EU or national food quality schemes under the Rural Development Policy shows differences with Council Regulation (EC) 3/2008, especially with respect to beneficiaries. Indeed, Regulation (EC) No 1698/2005 allows producers groups’ promotion for agricultural products produced under EU or national food quality schemes on the internal market. The promotion policy on the other hand allows promotion measures at the initiative of trade or inter-trade organisations representing the sector for PDO, PGI, TSG or organic products on the internal market or third country markets. There are small differences regarding the type of products (including national quality schemes for RDP) and the markets targeted (limited to the EU market for RDP), (cf. table 10 of chapter 1). However the same restrictions exist in both regulations about the mention of the origin or brands. Table 61: Differences between Rural Development Policy and Regulation (EC) No 3/2008 Item Rural Development Policy Information and promotion measures Agricultural products produced under PDO, PGI, TSG or organic products EU or National food quality schemes Producer groups. Professional and/or Type of inter-professional organisations Trade or inter-trade organisations representing one or more sectors representing the sector beneficiaries cannot qualify as ‘producer groups’. Internal Market or Third country Markets targeted Internal market markets Any reference to the origin of The origin of a product may be products shall be secondary to the indicated provided the mention of the central message of a campaign. origin is subordinate to the main However, the origin of a product Mention of origin or message. brands may be indicated as part of an Promotion of commercial brands is information or promotion operation not eligible for support. in the case of a designation under EU rules. Source: ADE 2011, based on the analysis of Regulation (EC) No 1698/2005 and Regulation (EC) No 3/2008 Type of products Final Report November 2011 Page 203 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Article 68 of Council Regulation (EC) No 73/2009 As developed in chapter 1, Article 68 of Council Regulation (EC) No 73/2009 barely touches upon information and promotion of agricultural products. Indeed, most Member States limited the use of measures within Article 68 to the case of specific types of farming or activities with enhanced environmental benefits as well as improvement of the quality of agricultural products. Only one Member State – Sweden – activated the “Improving the marketing of agricultural products” measure within Article 68 that is broader than just promotion (it concerns more the quality and marketing of agricultural products (including infrastructure) than just promotion). Awareness regarding the demarcation lines Competent bodies As described above, the competent bodies, through their initiatives to articulate promotion funds, are pretty much aware of the demarcation lines between these Funds and can easily differentiate them. Proposing organisations Proposing organisations in seven of the nine visited Member States were aware of the different types of information and promotion measures under the CAP, at least for their sector. In two of the nine visited Member States they declared not being aware of promotion measures under the CAP other than the I&P scheme. In one Member State, the Ministry of Agriculture organizes working sessions with the main producers’ organisations at least once a year, to improve their knowledge on different funding opportunities. The implementing bodies are not very concerned with these demarcation lines although some of them, as explained above, seem to provide insights on the subject to proposing organisations. According to interviewees, awareness levels regarding the demarcation lines depend mostly on their experience in the sector. J.12.2 Contradictions have been avoided A comparison of objectives, messages conveyed, groups targeted and channels used, between EU co-financed programmes under Council Regulation 3/2008 and other CAP promotion measures show that there are no contradictions and that other CAP promotion measures are less restrictive (see table below). Indicative figures of annual budgets show the importance of some of these measures. While the objectives of the promotion policy is to increase knowledge, enhance image and support consumption and demand of EU agricultural products, the objectives of other CAP measures address explicitly competitiveness. Of course, these objectives are clearly linked. Final Report November 2011 Page 204 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Messages conveyed are explicitly defined for the promotion policy, and are to a large extent similar to those defined for RDP. For wine CMO measures, messages seem quite complementary with the promotion scheme, and no messages have been defined for the F&V CMO. The promotion policy defines specific target groups which is not the case for other CAP measures. Channels used are clearly defined by promotion policy, as well as for the wine CMO and RDP but not for the F&V CMO. Table 62: Objectives, messages conveyed, groups targeted and channels used of EU co-financed programmes and other CAP promotion measures Item Objectives Information and promotion measures Increase knowledge, enhance image and open up new markets in order to support demand and consumption of EU agricultural products Improve competitiveness of the EU wine sector by reducing market interventions and moving towards pro-competitive measures Improve competitiveness and market orientation of the EU F&V sector, reduce income fluctuations and promote consumption CMO Wine sector CMO Fruit and vegetables sector Rural Development Policy Improve competitiveness of agricultural products by improving and promoting their quality Item Information and promotion measures CMO Wine sector Messages conveyed Wine : EU legislation strictly regulates production, quality indications, labelling and marketing, so guaranteeing for consumers the quality and traceability of the wine on offer, the attraction of being able to choose from a very wide selection of EU wines of different origins, information on EU wine cultivation and its links with regional and local conditions, customs and tastes Fruit and vegetables: to promote a ‘five-a-day’-type approach, the products are natural and fresh, quality (safety, nutritional value and taste, production methods, environmental protection, link with the product’s origin), enjoyment, balanced diet, variety and seasonal nature of the supply of fresh products; information on their tastes and uses, traceability, accessibility and ease of preparation: many fresh fruit and vegetables require no cooking Highlighting the advantages of EU products, especially in terms of quality, food safety or environmental friendliness CMO Fruit and vegetables sector Not defined Rural Development Policy Activities shall draw attention to the specific features or advantages of the products concerned, notably the quality, specific production methods, high animal welfare standards and respect for the environment linked to the food quality scheme concerned, and may include the dissemination of scientific and technical knowledge about those products. Item Information and promotion measures CMO Wine sector CMO Fruit and Final Report Groups targeted Wine: Distributors, consumers, excluding young people and adolescents, opinion leaders: journalists, gastronomic experts, educational establishments in the hotel and catering sector Fruit and vegetables: households, school canteens, children in educational establishments, mass caterers, doctors and nutritionists Not defined Not defined November 2011 Page 205 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC vegetables sector Rural Development Policy Induce consumers to buy the agricultural products or foodstuffs covered by food quality schemes included in the rural development programme under Article 32 of that Regulation Item Information and promotion measures CMO Wine sector CMO Fruit and vegetables sector Channels used Wine : Information and public relations measures, training for distributors and caterers, contacts with the specialised press, other channels (Internet site, leaflets and brochures) to guide consumers in their choice and to develop ideas for consumption at family events and festive occasions, fairs and exhibitions: stands presenting products of several Member States. Fruit and vegetables: Electronic channels, telephone information lines, PR contacts with the media and advertising, contacts with doctors and nutritionists, educational measures targeting children and adolescents in educational establishments by involving teachers and school canteen managers, point-of-sale information actions to consumers, other channels (leaflets and brochures with information on the products and recipes, children’s games, etc.), visual media (cinema, specialised TV channels), radio spots, participation in trade fairs Public relations, promotion or advertisement measures, participation at events, fairs or exhibitions of international importance; information campaigns, in particular on the EU systems covering designations of origin, geographical indications and organic production; studies of new markets, necessary for the expansion of market outlets; studies to evaluate the results of the information and promotion measures. Not defined Activities shall include, in particular, the organisation of, and/or participation in, fairs and exhibitions, public relations and promotion exercises and advertising via the different channels of communication or at the points of sale. Source: ADE 2011, based on the analysis of Regulation (EC) No 1234/2007, Regulation (EC) N°1698/2005 and Regulation (EC) No 3/2008 Rural Development Policy The main difference lies in the possibility to mention brands (individual or collective trademarks) and the origin under the CMO for wine and F&V but not for the co-funded I&P actions. This possibility is considered as very effective. This fact reveals imbalance issues rather than contradictions (see EQ 10 on complementarities with national and private initiatives for more details on the branding and origin issues). Some POs and competent bodies underline that it is much more difficult to get selected under the promotion policy (3/2008) than under the wine CMO, the latter being also much easier to manage and less time consuming. This opinion is shared by stakeholders from the fruit and vegetables CMO who consider both regulations complementary, promotion under the operational programmes being much easier to manage though. Consequently, promotion activities under CMO and Rural Development support are considered more appealing (less competition for budget, easier selection procedures, etc.) by beneficiaries than the EU co-financed programmes. Final Report November 2011 Page 206 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC In Portugal, the wine CMO promotion programmes clearly appear to be easier and more attractive, essentially from the following main differences with the EU co-financed programmes: The list of third countries includes less countries than EU co-financed regulations but adds some interesting countries (like Angola) Wine CMO promotion programmes require a report every semester and a final report containing an assessment of the costs and programme benefits Messages and target groups are not listed in the CMO regulations contrary to the EU co-financed programmes It is possible to promote brands in the CMO regulations, which is not permitted in EU co-financed programmes Table 63: Regulation €m Indicative figures of annual EU budget for each of these measures shown in the descriptive chapter 1.2 3/2008 1234/2007 – Fruit & Veg 1234/2007 – School fruit 1234/2007 – Wine 1698/2005 RDP 50 30-40 90 150-250 29 The table above illustrates the relative imbalance and financial importance of other CAP measures compared to the horizontal promotion under Council Regulation 3/2008. J.12.3 Potential complementarities and synergies have been promoted and exploited Potential complementarities and synergies between EU co-funded programmes and CAP promotional measures have not been reported in the programming documents for information and promotion programmes. There are explicit references to other CAP promotional measures in the regulations for EU co-funded programmes (and vice-versa), but these cross-reverences have not given rise to specific actions to promote complementarities and synergies. Some POs highlight that there is no systematic check concerning complementarities and synergies between the different promotion measures, but that this check is done ad-hoc and implicitly. By remaining close to the provisions laid down in regulations, it can be assumed that at least overlaps are avoided while synergies and complementarities are implicitly implemented at the strategic level through the selection of proposals by the competent bodies. According to competent bodies in MSs largely involved in promotion under the wine CMO, part of the procedures, guidelines and formats from the promotion policy were used to develop those for the wine sector. Final Report November 2011 Page 207 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC To what extent have information and promotion measures applied under Council Regulation No 3/2008 been coherent with other CAP promotion measures? Evaluative Judgement Currently, promotion of EU agricultural products can be financed in a horizontal way through Council Regulation 3/2008, in a much more limited way for national or EU quality schemes through rural development programmes (RDPs) and in a sector-specific way for wine and fruits and vegetables especially since the recent reforms of these CMOs. Cross-references between promotion activities are included in the various regulations governing promotion activities financed under Council Regulation 3/2008 and under other CAP measures. These cross-references aim to avoid multiple financing. Demarcation lines are generally defined or exist implicitly due to differences in the types of beneficiaries or markets targeted between measures. In addition, some competent bodies endeavour to define more specific demarcation lines between the promotion policy and other CAP measures. However, the approach is more a separation procedure than a desire to articulate those different measures to ensure complementarity. Competent bodies and a large number of proposing organisations seem to be aware of the different information and promotion measures for agricultural products (in their sector) and their demarcation lines. There are no contradictions in terms of objectives, messages conveyed, groups targeted and channels used between EU co-financed programmes and other CAP promotion measures. Objectives of other CAP measures are more oriented towards increased competitiveness whereas the promotion policy aims to increase consumption and demand. Council Regulation 3/2008 defines specific target groups which is not the case for other CAP measures. Channels used are clearly defined by Council Regulation 3/2008, as well as for the wine CMO and RDP. Messages conveyed are explicitly defined for Council Regulation 3/2008, as well as for RDP with large similarities. Stakeholders consider promotion under other CAP measures as being easier to manage, to get selected, and also to implement, with less administrative burdens. The possibility of mentioning the origin and brands that exists under certain conditions under the CMO (individual or collective brands for wine, collective trademarks for fruit and vegetables) is considered very effective and makes an essential difference with Council Regulation 3/2008. As both promotion measures are supported by public funds this difference is not well understood. The important financial amounts dedicated to the promotion of wine under the CMO need to be underlined by comparison with the resources available through the promotion policy. These amounts are established by Member States which decide on the allocation to each of the 11 measures in their National Support Programme. Not all Member States allocate resources to wine promotion. Nevertheless, promotion is a major strategy of the sector to increase its competitiveness in the context of the reform abandoning market support measures. Few or no complementarities and synergies were reported or identified between measures by the stakeholders although some of them (competent bodies and proposing organizations) endeavour in some cases to ensure a certain degree of complementary. Although the promotion policy is coherent with other CAP measures, with exception of the branding issue, the current policy set-up with several promotion measures under the CAP without real articulation (only avoiding overlaps) and without explicit search for complementarities may not be very efficient. Final Report November 2011 Page 208 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 5. Closing chapter This closing chapter reviews the conclusions derived from the intervention logic of the promotion policy, the various evaluation questions and includes some major findings of the descriptive part when necessary. The conclusions and recommendations are presented by theme like the replies to the evaluation questions: Theme 1 Policy relevance and effectiveness Theme 2 Management of information and promotion programmes Theme 3 Coherence and complementarities with other CAP and national and private promotion initiatives 5.1 Conclusions 5.1.1 Theme 1- Policy relevance and effectiveness Conclusion 1: Increased consumption and demand of EU agricultural products appears to be the global objective of the EU promotion policy though it is not explicitly specified in Council Regulation (EC) No 3/2008. Also, the potential benefits for the producers and for the consumers are not explicitly mentioned. Based on the intervention logic The promotion policy seeks to supplement and reinforce activities developed by Member States by boosting product image in the eyes of consumers in particular regarding quality, nutritional value and safety of EU agricultural products and their methods of production. This is reflected in the specific objectives identified in Council Regulation (EC) No 3/2008 and in the annexes of Commission Regulation (EC) No 501/2008 which aim at increasing knowledge and enhancing the image of EU agricultural products among consumers. In addition, there is one particular objective for third country markets, namely to open up new markets. Council Regulation (EC) No 3/2008 and Commission Regulation (EC) No 501/2008 suggest that supporting demand and consumption is the global objective (particularly in annex I of Regulation (EC) No 501/2008) but are not fully explicit in that respect. It is obvious but not explicit that European producers should benefit from increased consumption, thanks to increased sales and exports. This goal is consistent with the objectives of the CAP. On the other hand, the interest of the consumers to buy and consume more is as such not sufficiently underpinned by health, diet or environmental concerns in the Regulation. Final Report November 2011 Page 209 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Conclusion 2: The list of products and themes that may be covered by promotion is large and responds to a high diversity of issues at stake in terms of market and consumption trends. The list of products is a good basis to achieve the objectives of the regulation as long as it is used as a tool to define more structured strategies or to prioritise programmes, which is currently not the case. Based on EQ1 The list of themes and products covers a large number of agricultural and food products at the first processing stage (e.g. it includes fresh and processed products, single products (milk) /group of products (fruit and vegetables; organic products)/themes, “standard” (olive oil) versus quality schemes (PDO, PGI, TSG, organic products)). It includes most of the important products in terms of EU export values. According to the statistical approach undertaken in this study, which combines consumption trends (growing/stable or declining market) and trade trends (export and import trends at EU level and on the world market), there are very diverse reasons that justify promotion of the various products included in the list. The list of products for the internal market and third country markets are largely similar, with however slight differences not always well justified. For instance, among the products eligible for the third country promotion only, and thus excluded on the internal market, are food preparations based on beef, veal and pig meat, products processed from cereals and rice and spirit drinks with a geographical indication. Hence, some competent bodies considered useful to merge the two lists. The answer to the question whether or not this list is relevant compared to the objectives laid down in the Regulation (namely improved knowledge, enhanced image and opening up new markets in order to support consumption of EU agricultural products) is not straightforward. Indeed, the large scope of the lists encompasses main EU agricultural products and they do altogether contribute to boosting the image and knowledge of EU products. The broad coverage of the list is appreciated because it allows flexibility, but as such it is unfocussed and cannot guarantee alone the achievement of the objective of the regulation as long as structured strategies are missing. That is why it should be used as a tool to define more structured strategies or to prioritize actions in programmes. Conclusion 3: The list of eligible countries and geographical regions covers most parts of the world. According to an indicative scoring based on GDP, trade and consumption trends, some countries appear overall more relevant than others. Yet the list is to be regarded as a broad basket of opportunities in which targets may be pinpointed by Member States. Based on EQ2 The list of third countries and regions covers large parts of the world except some of the poorest countries (namely in sub-Saharan Africa). It includes most important trading partners of the EU in terms of export values. The indicative country scoring system developed in this study (based on macroeconomic indicators, such as EU export trends to Final Report November 2011 Page 210 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC third countries, third countries consumption and their GDP trends) shows important differences in terms of potential macroeconomic interest to set priorities among countries. However additional criteria (including qualitative ones) need to be added to the system indicatively developed. Regarding the geographical areas included in the list, this broad level of coverage does not enable precise assessment of their importance for promotion. Geographical areas are essentially useful because they leave a large degree of flexibility to Member States for action. Conclusion 4: The Commission initiatives complement actions at national level which primarily depend on initiatives of intertrade organisations and this is relevant to promote EU agricultural products, both on the internal market and in third country markets. They can also have a leverage effect on co-financed programmes if there are links with co-financed programmes. Based on EQ3 Commission initiatives may cover information campaigns especially on European quality schemes on the internal market and in third countries as well as participation in events, fairs and exhibitions with the specific aspect of high level trade visits on third country markets. The concept of Commission initiative is relevant in the context of the promotion policy which primarily depends on the initiatives of national trade and intertrade organisations, both on the internal market and in third countries. Indeed, the EC can launch initiatives of Community interest which are not undertaken by proposing organisations. Furthermore, the Commission may undertake campaigns at EU level which concern all 27 EU Member States. Commission initiatives have mostly concentrated on the organic sector and on PDO, PGI and TSG products and are in line with the main priorities of the Regulation. On the internal market, the Commission launched a promotion campaign on the organic sector (2005-2009) that was coherent and linked with the European Action Plan for organic food and farming. This campaign was useful for organisations which launched organic promotion programmes thereafter (possibility to reuse promotion material and messages). On third country markets (USA, Japan and China), the Commission launched the EAT campaign (European Art of Taste), an information campaign about European quality designations for food (PDO, PGI, TSG and organic products) in 2005-2007, The evaluative judgement on this campaign is restricted by the limited amount of information available on the campaign. Few of the programme documents referred to the campaign and views were mixed. Nevertheless, the Commission initiatives on both internal and third country markets are overall relevant to promote EU agricultural products. They can act as a lever for other promotion actions of EU agricultural products if coordination with cofinanced programmes is set up. Moreover, the Commission organises on average once a year so-called high level trade visits to third countries that include participation in international events, mainly trade fairs. Final Report November 2011 Page 211 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC These often involve a Commissioner accompanied by a delegation of representatives of the European agri-food sector. They aim to value a positive European image of agricultural products, facilitate further relations for national stakeholders and enhance business opportunities for exporters. The immediate feedback from these visits is positive, although the longer-term impact on business opportunities has not yet been assessed. Conclusion 5: Although the EU promotion policy framework has encouraged multi-country and multi-product programmes, their occurrence is limited. The number of proposals is hampered by the specific difficulties they face for design and implementation. Their advantages in terms of economies of scale, leverage effect, wider target group reach and cooperation make them especially relevant in achieving the objectives of the Regulation. Based on EQ4 There are three types of “multi-programmes”, namely multi-country programmes, multiproduct programmes and multi-target programmes. Genuine multi-country programmes are those proposed by several Member States. In addition there are also multi-target programmes, involving measures in more than one Member State or in more than one third country. The multi-product and multi-country programmes have advantages in terms of critical mass, reaching wider target groups, synergies and economies of scale, leverage effect and cooperation at European level, but also numerous disadvantages in terms of administrative burden, logistics, cultural and regulatory differences (including different interpretations of the guidelines) between countries and competition between producers. Multi-product programmes are rather marginal (8% of the programmes from 2001-2007 and only 4% of the programmes in 2008-2010). They are appropriate for some specific combinations of products (e.g. basket of products, combination of products as cheese and wine, Mediterranean diet, etc.). Due to complicated implementation (such as different marketing issues to address, different perceptions of consumers on products, different seasonality, dilution of the message, etc.), they are viewed as counter-productive by some proposing organisations, which submit multi-product programmes to improve their chances of receiving co-funding more than anything else. Through some combinations of products however, they can reinforce the EU image by articulating a common message on EU features. Multi-target programmes combine the advantages of penetrating new markets while ensuring economies of scales. The genuine multi-country programmes should provide common activities with large economies of scale, synergies and leverage effect. They contribute to favour the cooperation and exchange of experience. Promoting cooperation between Member States is not only relevant to the objectives of the promotion of agricultural products Regulation, but also more globally for EU cohesion. The difficulties in designing and implementing such programmes (identification of reliable partners, cultural and language hurdles, coordination, different rules and interpretation of guidelines in Member States...) are a reason of their current limited occurrence and require a specific support from the Commission. Final Report November 2011 Page 212 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Conclusion 6: The quality of proposals is improving and there is widespread exchange and adoption of good practices. Yet, the adoption of good practice on the measurement of efficiency could still be improved. Based on EQ5 and EQ9 89 Solid market analysis, identification of SMART objectives, clear targets, well adapted actions and good monitoring are widely appreciated, supported and adopted practices among proposing organisations. 90F The progressive adoption of good practices in Member States enables an improved quality of the proposals (reflected in the analysed approved programme proposals and confirmed by stakeholder interviews). This trend runs in parallel with the growing experience gained by applicants and stakeholders, but is not translated into higher success rates at the level of the EC selection, characterized by important fluctuations without clear trend. There is still room for improvement in programme proposals, illustrated e.g. by the unclear activities, the insufficient programme strategy (especially for third country programmes) the lack of details and justification for the choice of communication channels used in the programmes (applicants do not provide much justification of their choice of communication channels (or combination of several channels), which should be based on the best possible cost efficiency). Many programmes reported that it was difficult to isolate their impact on demand and sales. That seems acceptable for information programmes and programmes targeting a large audience, less for those having a limited target group. Overall, there is no ‘promotion recipe’ and good practices have their limits in a heterogeneous context such as this of information and promotion of agricultural products, which pleads for a case by case approach for the design of the Programmes, with renewed investment by the applicants at each submission. Finally, although most stakeholders believe that the quality of the proposals is improving, this is not evidenced by a higher success rate at EC level until the end of 2010. The improvement of communication and exchanges between both levels is therefore an important issue. 89 SMART objectives : specific, measurable, attainable, relevant, time-bound Final Report November 2011 Page 213 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 5.1.2 Theme 2 – Management of information and promotion programmes Conclusion 7: Although the overall framework set-up for managing promotion programmes is clearly defined, the two-step selection process and the timeconsuming implementation procedures - especially for multi-country programmes - are issues hampering the achievement of objectives of the Regulation Based on EQ6 and EQ9 The overall framework set-up for managing promotion programmes, from the European Commission down to the implementing bodies, is well defined in the Commission Regulations and guidelines. At the Member State level, competent national authorities (or competent bodies) publish calls for proposals. Proposing organisations, such as a trade or interbranch organisations, design and submit a programme promoting agricultural product on internal or third country markets for pre-selection by the competent body. This pre-selection is a mandatory step to be carried out at Member State level in order to check whether EU technical and administrative requirements are met. In the case of a multi-country programme – in other words, a programme put forward by more than one Member State (see evaluation question 4 for further details), several proposing organisations and several competent bodies may be involved. The decision to select (or reject) a programme proposal is made by the EU Commission. If the programme is selected to be co-funded by EU budget, the proposing organisation is responsible for managing the implementation of the co-funded programme. However, information and promotion activities of the co-funded programme are carried out by one (or several) implementing body (bodies). These last organisations selected by the proposing organisation shall have specific knowledge of the products and markets concerned as well as the resources necessary to ensure that the measures are implemented as effectively as possible taking into account the EU dimension of the programme concerned. During meetings, the management committee, which is composed of EU Member States Ministry representatives and EC promotion unit officials, issues opinions on the draft Commission decision. There are certain overlaps in the two-step selection process between the Member State level (pre-selection) and the EC level (selection). In practice, both the Member States and the EC examine eligibility and quality criteria and duplications exist (especially regarding quality criteria) and the procedure involves a significant delay (over 6 months between application and final selection, which is considered to be long in the context of promotion). Many stakeholders suggested placing the leading responsibility for selection more clearly with MSs or with the Commission. As things work today, the selection made at EC level is not always well understood by Member States due to the following elements: the high number of criteria taken into account, the difficult measurement of the fulfilment of these criteria, and the limited discussions on the selection results (although the selection Final Report November 2011 Page 214 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC details are made available to Member States), due to the fact that management committee meetings leave little room for in depth discussion. Furthermore, the selection requirements are criticised as well. Among them, one of the most criticised ones is the need to detail unit costs of activities throughout the whole programmes. Many Member States’ competent bodies provide assistance to proposing organisations and offer their experience to improve the quality of the proposals. Nevertheless, the high rate of rejection until end of 2010 suggests that the logic governing the selection made at Commission level is not enough known by the applicants and/or by the pre-selection team in Member States. This high rate of rejection is not well understood among the applicants, particularly in a context of EC budget availability, which is often the case in the I&P scheme. The problem may partly reside both in opportunistic behaviours at Member States level (where programmes would not be prepared or pre-selected seriously), too complex selection at EC level due to a very large number of criteria used and the difficult measurement of their fulfilment, or for the least in weak communication between the EC and the Member States. In all cases, the selection process of programmes, particularly the distribution of roles, may be improved and become more efficient. Main weaknesses are unclear or inconsistent activities, lack of details in actions proposed, unclear budget and insufficient strategy especially for third country programmes. The implementation procedures (reporting, management, etc.) are considered heavy and time consuming by applicants and competent bodies, especially regarding the programmes put forward by more than one Member State for which the difficult coordination between proposing organisations and competent bodies may also slow down the processes. The procedures are too rigid to allow responding with enough flexibility in the fast moving world of marketing. Especially, the level of details required in the proposals in terms of unit cost of actions over a period of three years in order to evaluate best cost/effectiveness congeals unnecessarily the activities over the further years of the programme Conclusion 8: The guidelines, only defined for the internal market, are useful to design programme proposals although their content is partially outdated. They are viewed as less useful to implement programmes. Generally, they have been well followed; their rationales are explicit and correspond to those of the Regulation. Based on EQ7 The guidelines provided by the European Commission to design, implement and manage co-funded information and promotion programmes on the internal market may be divided in two types: the regulatory guidelines, and the operational guidelines detailing the regulatory ones. There are no specific guidelines for designing and implementing co-funded information and promotion programmes on third country markets. Consequently, proposing organisations tend to design and implement these programmes on the basis of guidelines for the internal market, which may not be fully adapted to third countries. Final Report November 2011 Page 215 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Although some suggestions have been made by competent bodies and proposing organisations to improve the regulatory and operational guidelines 90, they are considered useful to design programme proposals on the internal market, and have facilitated the choice of messages to convey, target groups to reach and channels to be used even if some of the information is outdated (e.g. new communication channels such as on-line social networks are not yet mentioned). 92F In practice, there has always been a quite flexible interpretation of the guidelines by the EC and this issue is viewed positively by the stakeholders who use them as a useful common working tool more than binding instruction to design the programmes. Nevertheless guidelines are viewed as less useful to implement I&P programmes. Generally, guidelines have been well applied and most often, programming documents have included a market analysis as recommended, even if some of them are quite light. In selected programmes, the rationales underlying the choice of communication channels, the messages and the target groups have been explicitly mentioned in programming documents based on results of market analyses, on past experience of proposing and implementing bodies, as well as on past co-funded programmes. The choice of channels, messages and target groups are in line with those of Annex I (B) of Commission Regulation (EC) No 501/2008, or sometimes slight variations of them. Conclusion 9: Quarterly monitoring reports are considered too frequent and are mainly used to support payments. Even though they enable some monitoring their usefulness, their value and hence effectiveness are questioned. The annual evaluation reports, that are aggregates of quarterly reports, have facilitated monitoring, but only partially contributed to evaluation; their potential has not been fully exploited due to their heavy monitoring focus, weak guidance and methodological inappropriateness. Based on EQ8 Monitoring of I&P programmes rests on quarterly monitoring reports and annual reports. In addition, the concept of retrospective evaluation has been introduced since 2008 for all programmes (internal market and third countries). Although the structure of the quarterly monitoring reports is overall fit for purpose, they are considered too frequent. In practice, they enumerate activities with input and output indicators and listing of expenditures. Even though they enable some monitoring, their value is limited for evaluation. They are mainly useful to receive payments by Member States. Although the provided structure and guidance is clear and relatively explicit, the administrative burden due to frequency (quarterly) is criticised. The use of quarterly monitoring reports in programme management is mixed. 90 A systematic assessment of the relevance of the guideline contents (e.g., an overview of the situation of the sector, goals, target groups, main messages and main channels) has not been done because it goes beyond the scope of this study. Final Report November 2011 Page 216 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The annual reports look more like “annual progress reports”. The present format is not much more than an aggregate of the quarterly monitoring reports, not well adapted for evaluation purposes. The main request of Commission Regulation (EC) N° 501/2008 of an evaluation of the results obtained in the annual report is not included in the model annexed to the contract (Annex VIII B). It is however mentioned under Annex III C.5a91, that the budget may include costs for annual evaluation not exceeding 3% of the actual costs of implementing the measures. Currently, there is little evaluation of results in the submitted reports, although 3% of the programme budget can be earmarked for such evaluations. In addition, there are obstacles to putting evaluation findings into practice, mainly due to the administrative burden of programme re-design. In addition, there are obstacles of putting evaluation findings into practice the next year, mainly due to the administrative burden of programme re-design. Retrospective evaluations at the end of the programme (for which 5% of the budget can be earmarked) may compensate though by contributing to longer term improvements. Some guidance for these retrospective evaluations is provided in the application form. 5.1.3 Theme 3 – Coherence and complementarities with other CAP and national and private promotion initiatives Conclusion 10: No specific constraints to developing complementarities/synergies between EU co-funded programmes and national promotion initiatives have been identified, but there are two major issues for developing them with the private sector, namely the mention of the origin and the branding issue. Based on EQ10 and EQ11 Council Regulation (EC) N° 3/2008 and Commission Regulation (EC) No 501/2008 do both explicitly refer to complementarities between EU co-funded programmes and national/private sector promotion initiatives both on the internal market and in third countries. This explicit reference is included in guidelines to applicants and in the selection criteria but is not further elaborated (no further information is provided neither in the preselection documents from MS nor in the identification from the EC or in the programme proposals). Member States were asked in an online survey whether they had a national strategy or national programmes. Around half of the respondent Member States (15/23) declared having an information and promotion strategy for agricultural products at a general level or by agricultural product (or for some products). For those Member States with national strategies, complementarities and synergies with the EC I&P action have been identified although little concrete information is given. In any case, having a strategy facilitates in principle identification and implementation of complementarities and synergies. 91 Annex III of the model contracts concerns other expenditure of implementing measures. Final Report November 2011 Page 217 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The vast majority of the stakeholders interviewed in the Member States could see no constraints to developing complementarities/synergies between EU co-funded programmes and national promotion for agricultural products although they did not provide a lot of evidence for such synergies. Moreover, although it seems that the EU cofunded programmes support and reinforce national promotion strategies, it was thought that the synergies were often coincidental. Concrete examples of such synergies were in terms of promoting similar goals (e.g. healthy food, food quality etc.), conveying the same messages (e.g. ‘5 a day’) and in reinforcing promotion on a target group (e.g. schools) and reusing channels (e.g. common web-portal). Furthermore, overlap between programmes does not appear to be an issue in Member States. Two major constraints were identified in developing complementarities/synergies with private promotional initiatives, namely the restrictions on the mention of origin and the branding issue. Indeed, generic promotion benefits to all producers (one can say competitors) of the concerned products, be they European or even from third countries, which is somehow contradictory with the sense of business prevailing in the private sector. In some cases, the EU co-funded programmes are used to supplement national/private initiatives as an additional and very important source of funding, in particular for smaller inter-trade organisations or for generic campaigns that would otherwise not take place. Conclusion 11: There is evidence that without EU co-funding, generic, multicountry or multi-product promotion would only exist to a limited extent, which illustrates the added value of the EU I&P scheme. Even if there is a lack of clear evidence of the economic impact on exports and sales, the information and promotion programmes contribute to improving the image of European agricultural products Based on EQ11 EU funding has an important leverage effect on generic, multi-country and multiproduct programmes. Without EU co-funding, such programmes would have a much smaller scale and would be funded mainly by the private sector (PO). Due to this private funding, they would be brand-oriented and not generic. Some programmes would probably not take place at all, especially for third country promotion, where minimum critical mass is needed in terms of budget, not often affordable without EU support for smaller operators. Co-funded multi-country programmes also generate an important leverage effect as several parties participate in a programme, allowing a higher impact. These elements emerge from the previous evaluation reports and interviews with proposing organisations and competent bodies in MS. The transnational meetings that are organised by proposing organisations to elaborate multi-country programmes are mainly to coordinate the programmes and agree on their design (channels, messages, etc.). These meetings do however, as a side effect, contribute to the exchange of experience among organisations. Final Report November 2011 Page 218 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The I&P programmes are generally thought to improve the image of EU products and contribute to the development of positive connotations (e.g. high quality, safety, taste, etc). The specific contribution of multi-country programmes on the image is recognised by stakeholders. Promoting the intrinsic value of the products also has a positive effect overall on consumer behaviour. For some types of consumers, the more the product relies on a high quality image (e.g. wine, PDO, organic etc) the more useful the EU image is considered to be. However, there is a lack of clear evidence as to the direct economic impact (e.g. exports and sales) of generic campaigns, working on the image and awareness. Conclusion 12: I&P measures applied under Council Regulation No 3/2008 EU are coherent with other CAP measures in terms of objectives, messages conveyed, groups targeted and channels used except the possibility of mentioning brands under certain conditions under the CMO. Regulations and implementation focused on demarcation lines rather than searching for complementarities. The coexistence of an increasing number of promotion measures due to CMO reforms, with important financial allocations could limit the overall efficiency. Based on EQ12 Promotion of EU agricultural products can be financed in a horizontal way through Council Regulation 3/2008, in a much more limited way for national or EU quality schemes through RDPs and in a sector-specific way for wine and fruits and vegetables 92. 93F Cross references to promotion activities are included in the various regulations and try to avoid any possibility of duplication of financing. Demarcation lines are generally defined or exist implicitly due to differences in types of beneficiaries or markets targeted. In addition, some competent bodies define further specific demarcation lines and most interviewed stakeholders, with few exceptions, seem to be aware of the different information and promotion measures and their demarcation lines. There are no contradictions in terms of objectives, messages conveyed, groups targeted and channels used, between EU co-financed programmes and other CAP promotion measures. Objectives of other CAP measures are more oriented towards increased competitiveness whereas the promotion policy aims to increase consumption and demand of EU agricultural products. Stakeholders consider promotion under other CAP measures being easier to manage, first to get selected, but also easier to implement with less administrative burdens. They also wonder about the possibility of mentioning brands under certain conditions under the CMO that is considered very effective, but is not possible under the promotion policy. The important financial amounts dedicated to the promotion of the wine under the CMO need to be underlined by comparison with the resources available through the promotion 92 Several other CAP measures provide support for promotion activities, namely measure 133 of Rural Development Programmes (Council Regulation (EC) 1698/2005) for quality schemes promotion (€29m co-funding per year), the single CMO for wine and fruit and vegetables (Council Regulation (EC) 1234/2007) with respectively promotion on Third Countries for wine (approximately €150-250m per year); and for fruit & vegetables in the framework of operational programmes (approximately €30-40m co-funding per year) and the school fruit scheme (€90m co-funding per year). This has to be compared to around €50m co-funding per year from Council Regulation (EC) No 3/2008 Final Report November 2011 Page 219 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC policy. Nevertheless, promotion is a major strategy of the sector there to increase its competitiveness in the context of the reform abandoning market support measures. Few or no complementarities and synergies were reported or identified by the stakeholders although some of them try in some cases to ensure that the different promotion measures implemented are complementary. Although the promotion policy is coherent with other CAP measures, with exception of the branding issue, the current policy set-up with an increased number of promotion measures resulting from recent CMO reforms, with important financial allocations, could limit overall policy efficiency. Conclusion 13: Better communication between different stakeholders and decision levels in terms of Commission initiatives, the selection process, detailed implementation rules and strategies could improve policy design and implementation. Based on EQ3, EQ4, EQ6, EQ7, EQ9, EQ10, EQ11, EQ12 As it has been highlighted throughout the evaluation, misunderstandings are existing at different levels, and mostly between the two EC and MS levels. This is illustrated by different elements, such as the following: The potential leverage effects of EC initiatives on co-funded programmes may be hampered due to the limited information exchange on their content and coverage; Outcomes of high level trade visits could be further shared with a larger number of stakeholders after the visit ; Multi-country programmes encounter significant difficulties in design and implementation due to different interpretations between MSs of the EC requirements. These issues reduce the multi-country programmes’ efficiency ; There may be some misunderstanding and duplication in the two-step selection process ; Newcomer Member States express difficulties in designing programmes corresponding to EU requirements; Synergies between co-funded programmes and national initiatives are possible but are more coincidental, due to limited planning and cross-information; Relatively few complementarities among CAP promotion schemes are exploited. Based on this conclusion, the evaluators have formulated recommendations which encompass this issue, in recommendations 1, 2, 3, 4 and 10. Final Report November 2011 Page 220 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC 5.2 Recommendations Recommendation 1: Clarify the global objectives of the promotion policy (increased demand and consumption), and the potential benefit for producers and consumers. Objectives might differ between the internal market and third country markets Based on C1, C13 The regulation supporting the promotion policy does provide clear specific objectives. These aim to improve the knowledge and boost the image of EU agricultural products in the eyes of consumers, in particular as regards the quality, nutritional value, safety and production methods; and open up new markets in third countries. The regulation does not however specify the global objectives (increased consumption and so demand) intended beyond these specific objectives. This objective should be made explicit and be clarified in terms of intended benefits for consumers and producers, keeping in mind that objectives and target might differ between the internal market and third country markets. Establishing more explicitly the link between information/promotion and demand/consumption and benefits for producers and consumers on the internal market and producers for third country markets helps develop a more strategic approach with clearer priorities in link with the European strategy (see recommendation 2). Recommendation 2: Enlarge the list of products and themes and maintain a large geographical coverage, but associate it to a European promotion strategy that gives focus and priorities to ensure contribution to the global objective Based on C 2, C3 and R1 The analysis of the list of themes and products showed the large number of different issues at stake in terms of trade trends and consumption trends which justify promotion activities. Due to this fact, the broad focus of the list was considered a good basis to achieve the objectives of the regulation. It is thus recommended to further enlarge this list, within the limits authorized by the Treaty of the European Union and its Annex I, in order to increase flexibility of action. On the other hand, the European strategy (and national strategies) allowing to concentrate the allocation of funds on identified priorities should be defined. A similar conclusion was obtained about the list of third countries and geographical areas. Its wide coverage is relevant as it leaves flexibility to identify best combinations of countries and products, but this should be done thanks to appropriate prioritisation exercises in order to optimise allocation of funds. A multi-criterion system including macroeconomic indicators (GDP, consumption, and market trends) and qualitative criteria could be used in that respect. Final Report November 2011 Page 221 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Recommendation 3: Define a clear European promotion strategy for agricultural products and revise this strategy on a regular basis Based on C1, C2, C3, C4, C5, C7, C8, C10, C11, C12, C13 and R2 A consistent and comprehensive European strategy on information and promotion of agricultural products is missing. Promotion is done to support demand and consumption of EU agricultural products. Market trends should be included as a dimension of the strategy. This evaluation proposed an approach to position the different products/themes to be promoted according to their position on the world market (trends in export and importand share in EU exports)93, their classification in terms of internal market trends94 and the position of the EU products/themes on third country markets95. The European strategy should request an explicit positioning of information and promotion programmes with regard to these market trends. This positioning could be a basis to set priorities. Complementarities and synergies of the information and promotion measures with other interventions of the CAP, not only promotion measures, should be included in the strategy. The current criteria guiding the selection of sectors/themes/products (Council Regulation (EC) 3/2008 art. 3) should be integrated to the strategy and updated. It includes the European dimension (e.g. contribute to cohesion by promoting multi-country programmes); European production standards; specific circumstances (as already included e.g. crises). The strategy should encompass all promotion measures that currently exist under the CAP. It should address complementarities and synergies at each level (inside horizontal promotion namely between co-funded programmes and Commission initiatives and external coherence with other CAP measures and national/private initiatives based on principles of subsidiarity and additionality). 93 Four trends were identified, namely growing and significant, growing but small, declining and small, declining but significant for the EU. 94 Seven categories were identified, through the different combinations of consumption and/or trade trends of eligible agricultural products (positive, stable, negative). 95 Six categories were identified combining worldwide consumption and EU export trends (at extra-EU level). Final Report November 2011 Page 222 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Recommendation 4: Require Members States to define their national promotion strategies. These strategies should ensure coherence, complementarity and synergies with EU supported promotion Based on C1, C2, C3, C4, C5, C7, C9, C10, C11, C12, C13 and R1 Member States should be asked to describe their national promotion strategies for agricultural products. These strategies should clarify Member States priorities in terms of products/themes and their potential public support to promotion. They should also distinguish the promotion actions on the internal market and third countries. This strategy should show the approach of the MS towards promotion, its potential national/regional programmes and available information on private initiatives especially from intertrade organisations. This would be a first step to ensure consistency, complementarities and synergies with EU and private initiatives as stated in recital (4) of Council Regulation (EC) N°3/2008. The strategy has to include all other promotion-linked schemes supported by European cofunding (including Council Regulation (EC) No 3/2008, wine and fruit & vegetable CMO promotion, school fruit & milk schemes, Rural Development Plan measure 133). These strategies, encompassing promotion at Member State level should ensure internal and external coherence and the maximisation of synergies and complementarities of different promotion schemes. Such national strategies exist in other policies, not only in Rural Development Plans but also in e.g. the European school fruit scheme. It appears justified to require Member States to ensure coherence and to show how different sources of funding will complement each other in the most efficient way. Recommendation 5: Strengthen the EU support to the programmes put forward by more than one Member State, particularly relevant in achieving the objectives of the Regulation Based on C4, C5 and R2 An additional financial support should be given to the programmes put forward by more than one Member State. Indeed, these programmes favour the cooperation, exchange of experience, synergies and leverage effects. Promoting this cooperation between Member States, appear to be highly relevant regarding the global objectives promoting EU agricultural products in the framework of the CAP. The difficulties in terms of different administrative burden and logistics as well as in implementing such programmes (cultural, languages, coordination...) are understandable and justify thus a specific support from the Commission. This should be an increased share of overheads and implementing body fees. Regarding implementation of programmes, it could be left to one leader Member State to take charge of checking requirements such as reporting, instead of each MS being involved. Furthermore, a permanent contact with the EC level, through exchange platforms and Final Report November 2011 Page 223 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC workshops could be set up to encourage exchange of knowledge on overcoming difficulties encountered in such multi-country programmes (see R on workshops). Recommendation 6: During selection, define a clear distinction in roles for Member States and the Commission and improve communication between them. Based on C7 and R2 To, make the two step selection process more efficient, there should be a clear distinction between the role of Competent Bodies and the Commission. The framework would remain the same, with Competent Bodies emitting calls for proposals and Proposing Organisations submitting programme proposals. After their competitive selection, the Competent Bodies would then transmit proposals to the Commission which would make the final decision based on its own criteria. In practice, roles could be separately defined as follows: The Competent body would be responsible of the following: Eligibility: no proposal should be submitted to the Commission until the Competent Body has judged it to be eligible. The Commission will have an obligation to check the judgement of the Competent Body and the Competent Body must provide sufficient evidence to justify its judgement. Verifying market analysis, relevance to the MS strategy, duration, scale and targeting, coherence of actions to objectives and its European dimension. Again, the Competent Body must report the basis of its judgement so that the Commission can confirm its agreement. If the Commission disagrees with the judgement, it must provide written and explicit reasons for doing so. The Commission, on the other hand, would take care of the following: Assessing the methods proposed for impact assessment. Experience has shown that this has not yet been treated with sufficient seriousness by Proposing Organisations and Competent Bodies and leadership is required from the Commission to promote more realistic attention to impact assessment. Judging the EU dimension of proposals, since there is some evidence that Competent Bodies are not always consistent in their assessment of this criterion. In recognition of the fact that there are differences in national strategies and European strategies, the Commission should be responsible for assessing the extent to which a proposal meets the European strategy. Assessing the experience and representativeness of the Proposing Organisation. In the past, this has generally been treated as an eligibility criterion, although the Italian Competent Body has used it as a selection criterion. Because Competent Bodies often become too closely involved in Proposing Organisations, the judgement on this criterion should be made by the Commission. In common with most commercial bidding arrangements, the Proposing Organisation will need to present their experience in their proposal in sufficient detail to allow the Commission to reach a judgement on whether they are well suited to the assignment. Final Report November 2011 Page 224 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC The assessment of criteria should be summarised using a points system adapted from the system currently in use. Recommendation 7: Adopt more flexible implementation procedures and limit reporting requirements to alleviate the administrative burden Based on C7, C9 and R2 Management of programmes should be adapted to the quick changing world of marketing. This can be done by leaving more flexibility to programme managers once a budget has been approved. In the current situation, difficulties are encountered due to limits in transfers of expenditure between different items relating to the programme measures as indicated in the contract (proposing organisations must submit requests to the competent body to approve transfers 15 days before a new trimester for budget transfers greater than 10%). Reporting could be limited by replacing quarterly monitoring reports by only annual or biannual progress reports. It needs to be checked if this recommendation does not introduce additional cash flow constraints among implementing bodies allowing them to operate smoothly over the year. Annual reports should include a summary of the work carried out and an internal evaluation of the results obtained, as can be ascertained at the date of the report, as stated in the art. 19 of Commission Regulation 501/2008 but not mentioned in the model of contract. Evaluations of the programme activities should be performed and published at the end of the programme as introduced since 2008 with retrospective evaluations. Recommendation 8: Reorganise the guidelines into one document which would not be part of a Regulation, and which would encompass issues of EU priorities, guidance and operational requirements Based on C6 Guidelines should be maintained because an operational framework is necessary to guide co-funded promotion and its specificities (generic promotion, focus on EU image, etc.). The high number of guidelines (Regulatory: Article 5(1) of Council Regulation (EC) No 3/2008, Annex I(B) of Commission Regulation (EC) No 501/2008, and Operational: Document AGRI/60787/2007-rev.5, Document Ref. ARES(2009)347563-27/11/2009 and Document AGRI-64545-2007-rev.4) should be reduced and harmonised, as already started as from 2010 with the handbook summarising all operational documentation available (published at the website accessible for MS competent authorities).One guideline document should encompass a) EU priorities and guidance; and b) operational requirements. This document should not be included in a Regulation, as it must remain flexible and adjusted to the context of the time. Guidelines should also consider third countries (as it is not the case today), for which the content should be different (different priorities, lesser knowledge of target groups, market situation, etc.). Final Report November 2011 Page 225 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC Recommendation 9: Update and make more explicit the rules governing the reuse of I&P material for later (brand) promotion Based on C3 and R2 Although private brands should not be eligible to co-funded programmes, the use of I&P material or messages for brand promotion should be welcomed and encouraged, provided that all stakeholders have equal access to I&P material and that there are safeguards to ensure that the EU identity is not linked to inappropriate products. Such is the case for internationally accepted messages (e.g. private brands also use the ‘eat 5 servings of fruit & vegetables per day’ recommendation). In some cases, private brands are also interested in reusing messages or material of generic promotion (e.g. such an example was highlighted in the milk sector, with the idea of reusing the ‘milk moustache’ of a co-financed programme for private sector branding). Such initiatives – if properly conducted – can lead to significant leverage effects on agricultural products promotion. However the current situation leaves room for different interpretations which can lead to disputes on the subject. A pilot phase could be led on the internal market, as it is more difficult to monitor and police advertising standards in third countries. Recommendation 10: Support generic promotion by a European label Based on C3, C10, C11 and R2 European producers are facing the competition of products from third countries that do benefit as well from the EC-supported generic promotion on the internal market. A closer link to European producers which could be recognized by a European identification in the promotion actions, including visual and/or content elements, should improve this situation and ensure a European return on generic campaigns. Recommendation 11: Provide for effective communication mechanisms between all the stakeholders at the EC level and in the Member States concerned by the information and promotion of agricultural products. This could be achieved by systematically informing on EC initiatives and setting up a an EU Exchange Platform where field actors can share their experience and the Commission can explain their priorities and views, for mutual enrichment and improvement of procedures Based on C3, C13 and R2 As explained in conclusion 12, all parties (and particularly newcomers) would benefit from enhanced communication, in order to better agree on priorities, constraints and objectives of co-funded programmes and Commission initiatives. Information on EC initiatives could be made available in advance on the promotion page of the DG AGRI website and on the websites of Competent Bodies. Exchanges on Final Report November 2011 Page 226 Evaluation of promotion and information actions for agricultural products ADE - METIS - AGROTEC programmes could be ensured by an EU Exchange Platform on the Promotion and Information Actions for Agricultural Products. The approach of this Platform would complement the one of management committees. In no circumstance would the existing formal decision process and committee in place be affected by the EU Exchange Platform. This Platform would have a flexible setting to allow exchanges in different forms and would consist of e.g. workshops, web-based shared library and discussions, a helpdesk, field trips, etc. The key stakeholders would be the main actors and beneficiaries of the Platform, such as the proposing organisations, implementing bodies but also farmers and consumers organisations and of course the European Commission (possibly with an observer status) and competent authorities in Member States. The Commission and/or the Management Committee would decide about the important strategic lines of the Exchange Platform for the year, such as the agenda of workshops, the agreement on the last year’s activity report and the next year’s work-programme proposed by the secretariat, or the issuing of important documents. On a daily basis, the secretariat would organize thematic workshops, animate debates, answer to questions via the helpdesk, maintain the website and invite stakeholders or independent experts. The main objective of this Exchange Platform would be to maintain an active community of stakeholders interacting between them, as well as with the competent authorities in MSs and the European Commission: top-down and bottom-up exchanges of views, transfer of know-how and good practices, formulation of proposals and suggestions. It would also be a possibility to share results from high level trade visits with a large number of stakeholders. Apart from this interacting community, the main deliverables of this Platform would be: a website, workshops, technical documents, catalogues of good practices, lists of FAQs, etc. The Platform could make suggestions to the Commission (e.g. revision of lists of themes, products and countries) but on a strict informal basis and it would be up to the European Commission and/or the Management Committee to take final decisions or not according to the decision procedures in place. It also has to give the opportunity to managers of multi-country programmes to expose the specific constraints encountered, e.g. due to the different legislations in MS or to the differentiated interpretations of guidelines by the administrations. The Commission has to participate in forum and to contribute to clarify unresolved questions. Final Report November 2011 Page 227
