Evaluation of the EUROPA site (June 2007 - February 2008)

Source: http://europa.eu/survey/feb08/results/index_en.htm

As noted in an internet wiki, the evaluation survey is now available online.

I hope it will be also the case for Parliament and Council in a near future

In order to collect input for defining the Internet strategy of the European Commission for the coming years, an in depth evaluation of the EUROPA website was conducted in 2007 over a period of 7 months and consisted in answering the following questions:

To what extent does Europa satisfy the diverse citizens' needs in terms of information (relevance and effectiveness)?
To what extent the transaction services provided through Europa cover the needs of the different target publics (utility and effectiveness)?
To what extent does Europa allow for a dialogue with the citizens on European issues (effectiveness)?
To what extent does Europa provide a coherent, user-friendly set of information and communication tools (efficiency)?

The evaluation was entrusted to the company Ernst & Young after a restricted call for tender in the context of the framework contract on evaluation and evaluation-related services (contract no. BUDG 06/PO/01/Lot 003).

A large data collection was carried out consisting of the following elements:

A large survey, available in 23 languages has been launched on the site and more than 8000 visitors answered it.
11 Focus groups and interviews have been conducted with European citizens, EU professionals from the public and private sectors, young people and disabled persons.
Benchmarks screened 30 sites from the public and private sphere and aimed at identifying best practices and assess the position of EUROPA in the internet sphere.
Log files analysis on the basis of 72 parameters and 7 485 753 lines, each corresponding to a click on the EUROPA website, during one day.
Usability tests encompassed expert review (usability inspection aimed at finding usability problems in the design), expert walkthrough (to test navigation scenarios through the search engine and the classic hierarchical system.), automatic and manual test approach.

Data analyses led to the formulation of both strategic and pragmatic recommendations, which are summarised in the executive summary [82KB] of the evaluation report.

The complete results of the evaluation can be found in the full report [2.4 MB] and its annex [781KB].

In the interests of an ongoing process of improvement, we periodically launch user surveys on Europa. These provide us with valuable information about the nationality, preferred languages, centres of interest, age, sex, level of satisfaction, criticisms and suggestions of users. They thus help us to gear Europa to the public's requirements.