Evaluation research
Evaluation is a comprehensive and complex assessment of selected actions/projects or programme.
We can distinguish two types of evaluation:
- Ex-ante evaluation – carried out before the project’s start, in order to verify whether – considering the assumed goal and financing – the selected project in its current form will bring about the expected results.
- Mid-term/on-going evaluation focuses on the current state-of-the-art of the project, hence the research is carried out before the project’s completion. The outcomes of such evaluation might serve to correct the actions within the project, or better tailor the actions in order to obtain the expected results.
- Evaluation ex-post – focuses on already completed project. The main objective of this kind of evaluation is the assessment of whether the assumed project’s goal has been accomplished. In addition, ex-post evaluation offers an insight into project’s actual effects (material, personal, financial and other).
In evaluation research we use data triangulation, that is mixing different research methods in order to carry out the most accurate and precise analysis.
The evaluation approach adopted by our research teams is based on quantitative (CATI/CAWI) and qualitative techniques (ITI, FGI and others), primary and secondary data sources analysis (i.e. data obtained in the project, comparisons, estimates, other reports and publicly available data).
We also build unique indicators enabling the measurement of efficiency and effectiveness of an action or project.
Evaluation– main goals
Regardless of the kind of evaluation, its main objective is the comprehensive assessment of a planned or implemented project, programme or actions. Main evaluation criteria, which we take into consideration, are the following:
- Effectiveness – it allows us to evaluate whether the effects of a programme/project has been reached in accordance with the assumed objectives.
- Efficiency – allows us to assess the cost-effectiveness of the project. In other words, this criterion allows to estimate whether the cost-results ratio is satisfactory. However, the cost-effectiveness of an action is measured not only in terms of financial costs, but also in terms of material and personal investment. High efficiency means that the project’s/programme’s assumed goals were reached with relatively low investment.
- Utility – the goal of its project is the highest utility and usefulness. Hence, it is essential to answer the question whether the actions or implemented project have genuinely contributed to the accomplishment of the assumed goal and whether they brought about tangible benefits.
- Validity – it allows us to assess if, and to what extent, implemented actions/interventions genuinely addressed previously identified problems and challenges.
- Sustainability – implemented projects, or more specifically, their effects, can differ in terms of permanence. This criterion allows us to assess whether positive changes resulting from the project and its implementation, will hold in the long term.
These are the criteria, which we always apply to all the research projects implemented by our team. Only taking all of them into consideration enables a comprehensive and multidimensional project analysis. It might occur, for instance, that even though the project was effective in the long term, its cost-effectiveness was on the low side. Hence, we will call such project economically ineffective due to the disproportionate investment in the course of the project, even though the assumed goals have been accomplished. On the other hand, the project might be useful and valid, yet not fully effective, as not all the assumed objectives have been met. Thus, all the factors need to be accounted for, both internal, related directly or indirectly to the project, remaining within the control of the project’s managers, or external ones. The latter are all the factors which remain out of managers’ control, such as COVID-19 pandemic, or the Russian invasion on Ukraine.
Our researchers carefully analyse socio-economic environment and the current political and legal context. Hence the foundation for the evaluation research is 1) identifying the relevant factors 2) their analysis 3) final assessment of their actual impact on the successfulness of the whole undertaking. These steps are followed because it might occur that project’s outcomes would be more beneficial, if it were not for the external factors, impossible to eradicate.
Evaluation - examples
The first stage of carrying out a successful evaluation projects is its accurate and precise planning. Together with the client, we draft the research methodology and, most importantly, strategy, which is also based on the elaborated indicators. We establish the list of the research problems, usually in form of questions which are of the utmost importance for the client. At this stage we also design the research tools (scenarios, questionnaires).
Next, we start the fieldwork.
Once the data collection is completed, we commence the analysis and work on presenting its outcomes in the form chosen by a client:
- final report,
- presentation,
- brochure (flyer),
- catalogue/publication.
We collaborate with numerous institutions in Poland: Ministries, Marshal’s Offices, Job Offices, Business Support Organisations, Higher Education Institutions, to name but a few; as well as with individual clients.
What techniques do we use in evaluation research?
We use the following techniques in our evaluation projects:
CATI
Telephone interview
CAWI
Online survey
PAPI
Pen and paper survey
Face to face FGI
Group interviews
FGI on-line
Group interviews
Ethnography
Observation and description
IDI
Individual face to face in-depth interviews
ITI
Individual online in-depth interviews
CAPI
Computer-Assisted Personal Interviews
Dyads, tryads
Small group interviews
User tests
Product tests, consumer’s assessment
DR
Desk research, secondary data analysis
MOBI
Face-to-face survey on mobile devices
Eye tracking
Biometric research – tracking the eye movement
Face tracking
Biometric research – facial expression analysis
Mystery e-mailing
Mystery shopper – testing customer service via email
Mystery calling
Mystery shopper – testing customer service via phone
Mystery shopping
Mystery shopper – testing face-to-face customer service
Project design methods
Design thinking, iterative design
Motion tracking
Motion capture - XSENS, Vicon, Kinect
Auditorium questionnaire
A questionnaire filled in by a specific group being in the same room
EMG
Biometric research – analysis of micro expression of facial muscles
EEG
Biometric research – brain activity analysis
GSR
Biometric research – the galvanic skin response
Why choose our offer?
Since 1996, we have been carrying out evaluation research. Our team of experts is highly knowledgeable on sociology, economics, statistics, political science and other fields. For years we have used our knowledge and expertise to create our original research strategies.
Owing to the highest quality standards (PKJPA, ESOMAR), professionalism and acquired experience, we take great pride in high indicator of returning clients representing public and market institutions.
We are specialists in this field, which translates to our clients achieving their expected goals.
We will be happy to answer your questions and suggest research methods most suitable for each individual case. We kindly invite you to get familiar with our full offer.
Evaluation research - Price list
The evaluation research, which we carry out, is customised and tailored to each client’s needs. Therefore, we strongly recommend contacting us, in order to obtain a detailed price list.
FAQ
Evaluation is a comprehensive and complex assessment of a selected actions/project or programme, which can be carried before the start (in order to verify whether the assumed goals and approaches are correct), during the project (in order to monitor its progress and also to adapt/amend actions) or after its completion, in order to comprehensively assess its outcomes.
In evaluation research we use data triangulation, that is combining different techniques and methods in order to carry out the most valid and accurate analysis. The evaluation carried out by our researchers relies on quantitative (CATI/CAWI) and qualitative techniques (ITI, FGI etc.) and analysing primary and secondary data (i.e. data obtained in the project, comparisons, estimates, other reports and publicly available data).
It is worth carrying out evaluation research. It enables us to accurately assess the effectiveness of our actions (including the costs of execution) and it is a valuable source of information for creation/planning further actions and projects.
In evaluation research we should take into account the project’s effectiveness (whether its goals were achieved), efficiency (whether the costs were adequate for its effects), usefulness (whether it was genuinely useful and beneficial for a group/project), valid (whether it was developed in accordance with its goals) and sustainable (whether its effects sustained in the long term).
We support all the industries
