In accordance with Order No. 41, the system for granting scientific organisation status in the field of medicine is undergoing significant changes: the Ministry of Health of the Republic of Kazakhstan has introduced a number of criteria that applicants must meet. Which aspects are key, and which indicators are taken into account in the assessment process?
What levels of assessment exist for medical organisations?
In one of our previous articles, we analysed Order No. 41, which amends the order of the Minister of Health of the Republic of Kazakhstan "On approval of the rules for granting and reviewing the status of a scientific organisation in the field of healthcare, as well as the rules for assessing the performance of scientific, scientific-technical and innovative activity." For more details on the updated procedure for granting status, follow this link.
In order to conduct as comprehensive an analysis as possible of the applicant organisation, the Ministry of Health of the Republic of Kazakhstan has established three levels of assessment. Each of them involves taking into account separate areas of the institution's activity.
Scientific potential – the basic conditions confirming that the organisation has the structural and personnel base necessary for scientific activity, namely scientific units with state registration, equipment for scientific research, personnel holding an academic degree (at least 10%), research personnel, and cooperation agreements.
Personnel and income – quantitative indicators linked to staff numbers and financial results:
| Indicator | Clinical profile | Non-clinical profile |
|---|---|---|
| Share of scientific staff in research groups | ≥ 10% of personnel | ≥ 10% of personnel |
| Income from science (as a share of annual budget) | ≥ 0.5% | ≥ 5% |
Performance – an assessment of the publication and research activity of staff and students, based on three formulas:
- Sa (personnel: publications, citations, patents)
- Sb (clinical innovation: high-tech medical services)
- Sc (students: publications, citations, patents and conference participation by students).
Criteria for granting status
Since the process of assessing a medical organisation involves passing through several stages, it is important to understand which aspects are the main ones and what minimum threshold must be overcome. That is why we shall look at the key criteria in more detail.
Scientific potential
Before moving on to performance, an organisation must confirm five blocks of potential – all of which are mandatory and must be documented when submitting an application.
- Scientific units – the presence of structural scientific units or research groups with state registration.
- Research equipment – the presence of equipment enabling scientific research to be carried out in line with the organisation's profile.
- Personnel with an academic degree – at least 10% of employees holding an academic degree (PhD or a doctorate in the relevant field) out of the total production personnel.
- Personnel indicators – the Hirsch index and publications in Web of Science or Scopus among the organisation's research personnel.
- Cooperation – valid cooperation agreements with universities and international healthcare organisations.
Particular attention should be paid to the requirements for personnel holding an academic degree. It is important that employees not only hold an academic degree, but that this degree is confirmed by a diploma and that they are on staff at their principal place of employment.
At the same time, the calculation is made on the basis of the entire production personnel, that is, employees of scientific units together with the teaching staff of the organisation, rather than solely on the basis of the number of staff in scientific units.
Minimum performance indicators
In order for a medical organisation to be able to submit an application for status, it must reach two threshold indicators in the field of publications and citations. For instance, in order to be eligible for approval by the Scientific and Technical Council under the Ministry of Health of the Republic of Kazakhstan, an organisation must have prepared:
- at least 5 publications in WoS/Scopus over 3 years, in which an employee of the organisation is the first or corresponding author
- at least 10 citations in WoS/Scopus over 3 years (in total across all of the organisation's publications)
It should also be emphasised that the role of the author is of fundamental importance: only publications in which an employee of the organisation is listed as the first author or corresponding author are counted. Furthermore, the affiliation stated in the publication must specifically indicate the organisation being assessed, otherwise the article is not counted.
Requirements for research personnel
A separate section of Order No. 41 concerns the qualitative characteristics of research personnel – that is, employees of scientific units and/or members of research groups and/or other employees holding an academic degree. The text of the document states that:
- at least 50% of research personnel must have publications – at least one article in WoS/Scopus over the last 3 years, with the journal having been indexed at the time of publication
- at least 40% must have a non-zero Hirsch index according to WoS or Scopus data (the value taken is from whichever database gives the higher index)
- at least 50% must have at least 3 years of research work experience out of their total length of service (confirmed by employment contracts and records of work in scientific units/research institutes; the experience is cumulative and does not need to be continuous)
Determining performance
Performance is indeed the most complex and most detailed element of the assessment, since it involves a comprehensive calculation of numerous factors according to three formulas – Sa, Sb and Sc. The first two of these are mandatory for all organisations, whereas Sb applies only to organisations of a clinical profile and to university hospitals.
Sa – performance of production personnel
This is the main formula, which is calculated for any organisation regardless of profile. It takes into account six indicators: the volume of funding for scientific activity, publication activity (articles, monographs, chapters), citations and the Hirsch index, patents, income from the commercialisation of developments, and participation in conferences.
Publications and citations carry the greatest weight in the final score, since together they account for half of the assessment, which underlines the priority given to publication activity over other forms of scientific activity.
Sb – clinical innovations
This formula is additional and applies only to organisations of a clinical profile or to university hospitals. It takes into account a single indicator – the number of high-tech medical services (HTMS) introduced – and is added to the institution's final score.
Sc – performance of students
This formula applies just as obligatorily as Sa, but it assesses the scientific contribution of the organisation's students, namely their publications, citations, patents and conference participation. Unlike Sa, the final sum here is divided by the average annual number of master's, postgraduate and doctoral students, which makes it possible to take into account the scale of the organisation's educational activity.
Distribution of indicators and their weighting coefficients
| Formula | Who is taken into account | Indicator | Weight (Q) | What it takes into account |
|---|---|---|---|---|
| Sa Sa = I1·Q1 + I2·Q2 + I3·Q3 + I4·Q4 + I5·Q5 + I6·Q6 |
Production personnel | I1 – funding | 0.10 | Volume of funding for scientific activity |
| I2 – publications | 0.25 | Articles in WoS/Scopus/Springer, monographs, chapters in a collective monograph, commentaries | ||
| I3 – citations + Hirsch index | 0.25 | Citations of publications and the h-index of employees | ||
| I4 – patents | 0.15 | Foreign/national patents, certificates of intellectual property | ||
| I5 – commercialisation | 0.15 | Income from commercialisation, implementation of developments, clinical trials | ||
| I6 – conferences | 0.10 | Publications in Conference Proceedings WoS/Scopus | ||
| Sb Sb = I7 |
Clinical profile only + university hospital | I7 – HTMS | — | High-tech medical services introduced |
| Sc Sc = (I8·Q8 + I9·Q9 + I10·Q10 + I11·Q11) / N |
Students | I8 – publications | 0.30 | Articles by students in scientific journals |
| I9 – citations | 0.30 | Citations of publications by students | ||
| I10 – patents | 0.20 | Patents by students | ||
| I11 – conferences | 0.20 | Presentations by students at conferences and publications in conference proceedings |
As can be seen, the procedure for assessing a medical organisation applying for scientific status has undergone significant changes: it has become more detailed and comprehensive, which, in turn, contributes to the formation of objective rating lists based on transparent and clear data. Medical institutions should therefore already be systematising their publication and research activity in order to prepare thoroughly for the assessment and to become more competitive.
If your medical organisation requires publications in Scopus or Web of Science in order to obtain the status, get in touch with Scientific Publications. Our specialists will help determine how many articles you need to prepare in order to meet all the established requirements, and will support you at every stage of the process. Would you like a free consultation? Fill in the form below, and our manager will contact you shortly. Together towards new scientific achievements!
