There has been a long discussion in Ukraine about how to measure and evaluate scientific research. How to understand if this is real science or a set of beautiful phrases? Funding, the reputation of individual scientists and entire institutions and teams working on projects depend on the answers to these questions.
scientific Irina Egorchenko in the article “How to evaluate Ukrainian science: four important things”, explains what changes await Ukraine.
Some participants in the discussion suggest focusing mainly on formal, in particular, quantitative scientometric indicators, and the requirements for publications for defending dissertations or the allocation of funding. Others are suggesting that experts should be allowed to decide and that publication requirements should be abandoned.
“But we understand that “peer review”, where there is a human factor, depends not only on the professionalism of experts, but also on subjective things, ranging from personal relationships to resistance to corruption. Therefore, without special measures to ensure independence and impartiality, an adequate assessment is impossible.”the author explains.
Recently, the International Assembly of Stakeholders, which included more than 350 universities and organizations from more than 40 countries, including Ukraine, prepared and adopted the Agreement on Reforming Research Evaluation.
The agreement proposes to abandon the inadequate use of journal impact factors and Hirsch indices, as well as other indicators that do not reflect the quality of a particular study, to evaluate research.
“At first glance, these ideas sound in unison with the demands of the opponents of Scopus, who want to return to a time when it was enough for a scientist to have: a) articles in local journals without independent peer review, and b) good relations with superiors. But this is only at first glance.d»– notes Yegorchenko.
The agreement offers focus primarily on peer review, which involves not just an assessment of the study by people who are considered experts in a particular field, but also certain procedures for selecting independent experts. Evaluate scientific research should not officials, and the scientists themselves.
Expert evaluation of scientific research is possible under several conditions:
- real provision of academic integrity;
- development of an independent expert assessment procedure avoiding conflicts of interest;
- different assessment procedures for “fast” and “slow” industries (“fast” are areas in which new knowledge changes quite quickly, new results appear frequently and are relevant for a relatively short period);
- training in the analysis of scientometric indicators – so that it is not just a comparison of quantitative data.
The implementation of the new principles will require training and financial resources, and not just Ukrainians.
How to distinguish a real scientific study from an empty piece of paper with beautiful phrases? Calculate the Hirsch index, the number of publications, or take the word of the conclusions of experts and officials? There have been heated discussions about this for a long time. But it seems that we have a chance to put an end to the disputes. Not so long ago, representatives of Ukraine signed an international agreement. What does this mean for Ukrainian science – read in the material Irina Egorchenko “How to evaluate Ukrainian science: four important things“.