|
|
|
MONITORING OF INFORMATION AND EVALUATION SYSTEM (MIES)
|
A. INTRODUCTION
1.The Statistics (Monitoring
& Evaluation) Division in the Ministry of Health & Family Welfare is responsible
for monitoring and evaluation of the National Family Welfare Programmes in the country.
The information flows from the primary levels and is consolidated at the State level
on a monthly basis before the information is sent to the centre for the national
level consolidation. The system for capturing information on Family Welfare programmes
has evolved over the years based on the changing needs of the Ministry. Similarly,
for the National Health programmes like TB, Malaria, Leprosy etc, the respective
Divisions in the Ministry have evolved their data reporting system. The Department
of AYUSH and National AIDS Control Organisation also have their own reporting system.
|
2.The Government of
India launched the National Rural Health Mission (NRHM) in April, 2005 with an aim
to achieving the targets set by the Millennium Development Goals (MDGs) 4, 5 and
6 and making the health delivery system more responsive to the health care needs
of the people of India. The Reproductive and Child Health – Phase-II (RCH-II) is
a critical programme under the National Rural Health Mission (NRHM). The NRHM has
a pro-poor focus and aims at establishing bottom up planning and monitoring processes
and systems so as to enable increased people’s participation, decentralization of
health services and accountability of health delivery and care personnel.
|
3. The Government of
India is committed to raising public expenditure on health from the current 0.9%
of GDP to 2-3% of GDP and substantial inputs are being infused into Public Health
System so that adequate capacities are created in the health sector. The NRHM aims
to undertake architectural correction of the health system to enable it to effectively
handle increased expenditure allocations and promote policies that strengthen public
health management and service delivery in the country. It has, as its key components,
provision of a female health activist in each village (ASHA); a village health plan
prepared through a local team headed by the Health and Sanitation Committee of the
Panchayat; results and outcome based management and performance based funding; feedback
through regular monitoring and evaluation; strengthening of the rural hospital for
effective curative care and accountable to the community and integration of the
National Health and Family Welfare Programmes and funds for their optimal utilisation
in the delivery of primary healthcare.
|
4.With the launch of
the NRHM, there has been a concerted effort towards streamlining and convergence
of the various interventions for Health, Family Welfare, AYUSH and NACO. In this
context, the Statistics Division has integrated the key indicators for these interventions
in a common MIES format that would facilitate efficient monitoring of these programmes.
The State Governments also need to revise the primary registers for capturing of
the required information at the disaggregate level.
|
5. In the meanwhile,
we are continuing to receive the information on the National Family Welfare Programmes
in the pre-revised format. The reports/information is received monthly, quarterly
and annually as per requirement for monitoring the schemes/programmes. A monthly
performance Review is prepared on the basis of the reports for monitoring the monthly
progress of the programme. Besides, annual returns on socio-economic and demographic
particulars of Vasectomy, Tubectomy and IUD acceptors viz. (i) number of living
children (ii) age of wife of acceptors etc, are also collected and published in
the annual publication ‘Year Book’. In addition, the M& E Division also organises
the conduct of various surveys like National Health Family Survey (NFHS), District
Level Household Survey (DLHS) etc. This Division is also responsible for conducting
the activities of the Population Research Centres (PRCs), Regional Evaluation Teams
(RETs), NIHFW and IIPS, all of which are associated with the research activities
to support the statistical and demographic activities of the health sector.
|
B. Monitoring & Evaluation under RCH-II/NRHM
6. The Monitoring &
Evaluation Strategy of Ministry of Health and Family Welfare (MOHFW) for the National
Rural Health Mission (NRHM) and Reproductive and Child Health Programme II (RCH-II)
programme increasingly focuses on achieving output/outcome results and has clearly
articulated the set up of monitoring and evaluation system. The technical strategies
in NRHM/RCH II are designed to increase access and improve service quality for specific
evidence based interventions. In line with principles of RCH-II/NRHM, most of the
states have prepared their Programme Implementation Plans (PIPs) and have also worked
out in detail the logical framework wherein output/outcome indicators have been
spelt out. Like wise under NRHM, the district plans are to be evolved with the district
specific objectives/goals. Since the financial disbursement from GOI to the State
and from State to districts are all linked to the performance and achievement of
the proposed objective/goal, M & E Division is in the process of evolving an
effective MIES to track the progress of the various initiatives under RCH-II/NRHM.
|
7. Besides, the MIES
strategy under NRHM the emphasis is not only on monitoring the physical performance
but also to evaluate the quality of services and to conduct the management evaluation
assessment of institutional arrangements for delivering the services. The elements
of MIES are classified into three distinct components of programme inputs, monitoring
and tracking and quality assessment review and evaluation.
|
(a) Quality Assessment Mechanism (QA)
8.Assessing and continuous
improvement in the quality of RCH services is one of the thrust priorities of NRHM/RCH
II programme. The MOHFW intends to undertake a process of evolving a methodological
framework for accessing maternal health, child health and family planning services
being provided by the public health system in RCH II programme. Since, quality assessment
and improvement is in nascent stage, it was decided to adopt a simplistic approach
and confine to a few selected indicators of reproductive and child health programme
so that the health system is able to absorb and internalise QA activities as part
of the routine activities. Being a new concept, it has been decided to pilot QA
in some selected districts before up scaling at the national level. On the basis
of the pilot, the details of assessing and evaluating quality of services will be
worked out and appropriate parameters will be devised. However, as this activity
is going to be initially through external facilitation, the methodology of conducting
the study, details of number of health institutions to be covered, frequency of
visiting the institutions and undertaking the activity will be finalized after the
pre-testing exercise.
|
(b) Programme Management Evaluation
9. One of the initiatives
in RCH-II includes creating new management support structures at centre, state and
district levels. Under programme management, evaluative studies will be piloted
to assess the management capacity of the public health system. Subsequently, appropriate
tools will be designed for enhancement of management skills of public health personnel.
IIM, Ahemdabad was identified as the nodal institute for preparing the tools for
assessing the institutional arrangement for service delivery in the states. The
Institute conducted a pilot study in Gujarat and Rajasthan and submitted report/instruments
to the Ministry which were circulated to the States.
|
(c) Community Monitoring
10. The National Rural
Health Mission (NRHM) and the Reproductive and Child Health Programme (RCH-II) have
articulated the need for decentralization of health programmes and strongly advocated
community management of the health programmes. Keeping this in view, M & E Division
is in the process of developing a framework and tools for implementing Triangulation
of Data involving Community Monitoring, which is to be piloted initially before
up scaling at the national level. As this is a new concept in the public domain,
appropriate tools, methodology and frame work are to be prepared and tested. The
Development partners are providing technical assistance for this aspect.
|
(d) Validation of data through Triangulation Methodology
11. MIES under RCH-II/NRHM
also envisages the need for validation of data by triangulation to minimize the
potential of misreporting. To be effective for policy development and programme
management; triangulation data generated will allow for comparisons over time and
lateral comparisons between target groups simultaneously. At the same time, it will
enable increased participation by all stakeholders in managing and developing accountable
and responsive services and supports, participatory decision making based on data
reflecting enabling factors and implementation bottlenecks. Given these advantages
of the approach, there is a consensus within the M&E division of GOI to experiment
this method but an appropriate methodology of triangulation of data in reproductive
health is yet to be formalized. As a matter of fact two sources of information-
one from MIS and the other through surveys are often available. However, the third
component of community reporting in a formalized manner is a new concept that has
to be evolved. A methodology for community monitoring mechanism and later triangulation
process are going to be piloted. On the basis of the experience gained in the pilot
study, a practically feasible methodology for triangulation will be evolved and
introduced as part of the MIES.
|
(e) Evaluation Surveys
12.Besides having regular
Monitoring and Evaluation mechanism in house as well as through Population Research
Centres (PRCs) and Regional Evaluation Teams (RETs) in respect of ongoing interventions,
M&E Division also organizes large scale surveys namely National Family Health
Survey (NFHS) on the lines of Demographic and Health Surveys conducted in the other
countries, Districts Level Household Surveys (DLHS) Facility Survey to assess and
evaluate the outcome/impact of the programmes /interventions from time to time.
The surveys through data at district/state level covering the areas viz Family Planning,
Immunization, Maternal Health &care, Infrastructure facilities available at
various health facilities levels including trained /skilled manpower (medical and
paramedical) in the country. The Survey data also gives information by social groups
viz SC, ST, OBC, Others. In pursuance to the decisions of the National Commission
on Population, the Ministry is now actively considering to conduct an Annual Health
Survey so that the District Health Profile of each district could be prepared and
used as an input for policy initiatives. In the meanwhile, the DLHS would aim to
provide the baseline, midline and endline surveys for assessing the impact of the
health interventions on the community.
|
(f) Population Research Centre (PRCs)
13.The Ministry of Health
and Family Welfare established a network of 18 Population Research Centres (PRCs)
scattered in 17 major States. These PRCs are located in various Universities (12)
and other Institutions (6) of national repute and are under the administrative control
of M&E Division. The Centres are responsible for carrying out research on various
topics of population stabilization, demographic, socio-demographic surveys and communication
aspects of population and family welfare programme. The PRCs have been operated
as a Plan Scheme which has been continuing and it is being proposed to extend the
same in the subsequent Plans.
|
(g) Regional Evaluation Teams (RETs)
14.RETs are responsible
for monitoring and evaluating the programme implementation of Health and Family
Welfare services provided to the community in the country and to check the reliability
of information on Family Welfare Programmes. The seven RETs are located in the Regional
Offices of the Ministry of Health & Family Welfare. Each Evaluation team is
supposed to undertake tour of 20 days every month and cover 2 districts having 6
centres (2 rural family welfare centers and urban F.W. Centres in each districts)
selected randomly covering on an average of 700 acceptors of family planning including
RCH beneficiaries for field verification. Sample verification is done by the team
members contacting personally the selected acceptors of Family Welfare Services
who are selected from the registers maintained in the Health Centres. The RETs are
functioning under the guidance and supervision of M & E Division but their administrative
control rests with their concerned Regional Offices for Health and Family Welfare.
|
(h) Area Specific studies and concurrent evaluation
15.The M&E Division
provides technical inputs in formulating studies to conduct concurrent evaluation
of various Programmes implemented by the Ministry under NRHM as well as coordinates
the same with the field organization involved in fieldwork etc. Most of the studies
are allocated to the 18 Population Research Centres, International Institute for
Population Sciences, Mumbai, National Institute for Health & Family Welfare.
In addition the M&E Division is underatking a scheme for concurrent evaluation
of the NRHM by independent agencies that would be entrusted with the task of evaluating
the impact of the Mission in its various dimensions across various States.
|
C. Significance of MIES
16.It may be appreciated
that monitoring and evaluation is a key component of the NRHM as it aims to provide
critical indicators that would assist in identifying and developing mid course corrections
so that the goals of the NRHM and the Millennium Development Goals are achieved.
In particular the MIES framework under the NRHM would have the following advantages
once it is in position and fully operational:
• Addresses community needs and expectations
• Helps in preparation of Annual Action plan based on the community needs.
• Facilitates amalgamation of districts plans with state PIP’s
• Addresses unmet need for services and provides insight on the extent of met services
• Creates a system approach for monitoring and evaluation of RCH-II programme.
• Evolves a system of community monitoring • Flexible in approach and allows decentralized
planning
• Helps incorporation of state-specific indicators
• Allows for finalizing list of indicators upfront
• Adaptable to incorporate the next level of health revision-NRHM.
• Provides all requisite information to all stakeholders i.e. community, district,
State, Centre, donor partners and all other agencies.
• Increases accountability of programme managers in monitoring and strengthens feedback
mechanism
• Provides mechanisms for institutionalization managers
• Strengthens the hands of programme
|
D. WEAKNESSESS IN THE PRESENT M&E SETUP
17.The weaknesses in
the data reporting system in the States has repercussions in the consolidation of
health related information at the National level. The Ministry appointed a Task
Force on HMIS in March, 2006 and this has gone in depth into the weaknesses in the
system and made several remedial suggestions. Briefly, the weaknesses are listed
below:
(i) Legislative – Health being in the State List, leads to coordination problems
in implementing the health interventions in the states as also for monitoring of
information.
(ii) Administrative and Organisational – this emanates from having different Departments
for Health and Family Welfare in the States and also multiple reporting on various
issues from the primary health institutions.
(iii) Upward flow of information – There is an inherent bias in the upward reporting
of information which needs to be corrected by providing critical feedback down the
system to the primary interface. This would not only improve the accountability
of the information but also its ownership. PIP’s
(iv) There is an urgent need to leverage the advances in Information Technology
so that data can flow more quickly and be easily validated. A GIS based application
would be useful in mapping the resource availability with the needs.
(v) There is also a need to designate a nodal Health Information Officer at all
levels who would be mandated to ensure the flow of information in both directions.
(vi) Strengthening and streamlining the data reporting system by integration of
the parallel efforts by different agencies.
|
18.One of the new initiatives
under the NRHM is to have a well established M &E System at all the levels in
the health system starting from block level onwards. This was also recommended by
the HIMS Task Force. For a better M&E System, the following broad areas are
essential:-
(A)Identifying Nodal Information Officer at all levels
(B)Structuring the Information flows
(C)Infrastructural strengthening- IT, Networking, Manpower
(D)Information flows from the private sector
(E)Earmarking upto 3% of the State’s Budget in Information Technology interventions
|
PROPSED HMIS STRATEGY
|
(A) Identifying Nodal M&E Information Officer at all levels
19.1At the Central level,
it is proposed to integrate the data collection machinery in the various Programme
Divisions by establishing a National Bureau/Centre for Health Statistics (NBHS).
The proposed NBHS would essentially be a Resource Centre for the collection and
dissemination of all statistics related to Health & Family Welfare and also
coordinates the statistical activities for the Ministry. The NBHS may be headed
by the Statistical Adviser of the MOHFW in terms of the recommendations of the National
Statistical Commission. Thus the proposed NBHS would be responsible for integration
of all information that is being presently collected by the M&E Division, CBHI,
NICD (including IDSP), NACO, and AYUSH etc. It is also proposed to augment the manpower
and upgrade the infrastructure at the national level to meet the data requirement
of various stakeholders.
|
19.2At the State level,
this task could be performed by the State MIS Officer in the State Programme Management
Unit (PMU) where ever they exist. In States that are yet to establish the PMU, they
need to create such a position and the qualifications; eligibility and emoluments
would be the same as that of the State MIS Officer.
|
19.3Some States have
already appointed a District Data Assistant at the District PMU level. These could
be re-designated as the District M&E Officer. For States that do not have a
PMU in position, they need to create such a position and the qualification, eligibility
and emoluments of the District M&E Officer would be the same as that of the
existing District Data Assistant.
|
19.4Thus at the PHC
and CHC levels, an M&E Officer is to be identified or appointed to handle the
flow of information through the specified reporting forms for the various NRHM programmes.
|
(B) Structuring the Information flows
19.5The success of the
proposal is integrally dependent on the key stakeholders providing and making available
the NRHM related information with the Nodal M&E Officer (at any level). Thus
it will be necessary to ensure that the respective Programme Officers (RCH, RNTCP,
NVBDCP, NLEP, IDSP etc) simultaneously endorse a copy of the compiled data to the
Nodal M&E Officer at that level (State, District or Sub-District). The Nodal
M&E Officer will ensure that the analysis of this data is sent to the State/District
Mission Director and also fed back to the lower and parallel formations so that
they are aware of their status and how they are performing vis-à-vis their peers.
The Nodal M&E Officers will be encouraged to leverage the advances in Information
Technology in establishing an intelligent and responsive database.
|
(C) Infrastructural strengthening - IT, Networking, Manpower etc
at all levels
19.6Adequate support
for reinforcing the hardware and software support and manpower is to be given/established
at the District and State level statistical units/divisions in the Health &
Family Welfare Department. This will also require integration and merger of health
and family welfare statistical units in the States.
|
(D) Information from the private sector
19.7Presently, Health
Statistics are compiled in the Government Health Sector only as an offshoot of the
administrative data collection. In the last few years, the private sector has been
providing health facilities in a big way, not only in the urban areas but also in
the rural areas. The NRHM envisages involvement of the private sector in improving
the health care delivery systems through various interventions like NGO involvement,
PPP initiatives, community mobilisation etc. In the process, several private health
care facilities are also being accredited for providing services on a payment basis.
It is contributing significantly towards meeting the basic health care needs and
in providing other specialised medication and diagnostic services. However, there
is no systematic collection of information regarding these private health establishments,
as these are not required to be registered. The guidelines for accreditation by
GOI/State Govts in standardizing the quality and scope of services are being finalized.
In the process of accreditation, the following issues need to be considered while
framing the Guidelines for accreditation:
(1) The M&E Division
had evolved a format for capturing data on the NRHM/RCH-II interventions in consultation
with the various Programme Divisions and it includes information to be captured
from the private sector also. Presently there is no formal mechanism to capture
information from the private sector. To begin with, information from the accredited
institutions could be captured through statutory/mandatory returns.
(2) Thus while accrediting
the institutions, it may be ensured, as a part of the accreditation exercise, that
these institutes report data on the key parameters (indicated in the format) to
the NRHM Mission Director in the State/District as a mandatory exercise.
(3) (3) In addition,
the incentives being proposed for these institutions by the various Programme Divisions
should be invariably linked and be dependent on the institutions providing the data
on these key parameters.
(4) Appropriate forms
for data reporting by these Institutions could be especially designed to capture
both physical and financial performance.
|
19.8As brought out above,
most States have not evolved a holistic Monitoring and Evaluation strategy for the
health programmes. Some States have confined their M&E strategy to hiring Consultants
and procuring computers. Although Monitoring and Evaluation is IT intensive and
requires capital investment in a planned manner, only a few States have paid adequate
attention to this activity and have bundled it as an integral component in the implementation
of the health programmes. Moreover, at the time of approving the budget, it is usually
the M&E component that gets marginalised and so also does the data and information
flows. It is thus proposed that while approving the budget outlay for the State,
if they have undertaken a holistic approach towards M&E, keeping in view points
(A) to (D) above, the outlay for M&E may be preserved as per their PIP proposals
upto 3% of the total outlay, which will be in line with the IT Action Plan of the
GOI towards IT investments. This will ensure that the monitoring and evaluation
systems in the States are continuously kept in view as an integral part of the PIPs.
|
20.The Empowered Programme
Committee of the NRHM, in January, 2008, approved the above strategy for improving
and strengthening the Monitoring and Evaluation framework under NRHM.
|
|
|