CDTI Abia

Sustainability of the Abia State

CDTI ^project,

Nigeria.

June 2003

World Health Organization

African Programme for Onchocerciasis Control

Assessment of the

REPORT

VOLUME 2 - INSTRUMENTS

Ekanem Ikpi Braide (Team Leader) Charles Franzen

Yisa A. Saka Sunday Isiyaku Obinna Onwujekwa

RECU

I 5

^{SEP. 2003}

A?ocrDtt

lnstrument 1: national/ State level

NOTE:

.

This

instrument

is

for

national level,

which

coordinates programme imptementation in the

country. lt

could

also

be used (modified

slighfly if

necessary)

for

another level

of support

other than the main implementation level {districU LGA}. An example of

this

level is

the'State' in

Nigeria.

p,reviseasnecessary,andoverseeimplementationofCDTl

Policy.

This

level also provides

support

to

the

level below

it:

.

Providing targeted

training,

HSAM and

monitoring/ supervision.

- rzen, Obinna Onwujekwe and Sunday lsiyaku

Abbreviatio ns/ acronyms CDD

CDTI FLHF HSAM NGDO NOTF

community directed distributor

community directed treatment with lvermectin first line health facility

health education, sensitisation, advocacy, mobilisation

-

^i.e. activities that are aimed ^at getting all the key players to participate wholeheartedly in the ^programme

non-govemmental development organisation national onchocerciasis task force

t

.t

^{Cfrecf wnether ttrere is a} year plan

for

onchocerciasis

control,

appearing as part of an overall

written

^plan

for

the health service at

this

level.

has bec:ame integrated into the health service, management is accepting ownerchip of ^{the programme}

-

both good for sustainability.

Characteristics of

the indicator

^Sources

of information

a.

Onchocerciasis control should be integrated into the overall written year plan of the health service at this level. Note that this plan is usually not very detailed.

.

Examination ^of:

*

_Written plans: yearly, quarterly, monthly etc.

*

Minutes of planning meetings.

.

lnteruiew wilh senior heatth senrice staff at this level.

Describe the oresent situation:

There is an existing ^plan for onchocerciasis control activities ^and it is part of the overall annual plan of the department of public Health and Primary Health Carc and also of the State Ministry of Health services. There ^are also monthly ^plans for core CDTI aclivities ^at the project office.

All programmes submit ^plans, which are consolidated into an annual plan for the MoH. There is no direct evidence thai plinning is integrated, as this is not the system within the Ministry. All programmes submit plans, which are consolidated into an annual ^plan. The planning process is not integrated, as this is not the policy within the Ministry.

lf Olanninq and implementation of CDTI is not part of the overall vear ^plan:

.

Whv is this?

Not Applicable

.

Which steps are beinq taken to improve the situation?

Not Applicable

Sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

Analysis

.

When writing the

repft

you have to summarise:

*

The evidence about how well fhis indicator is being achieved.

*

Reasons for poor pefiormance (if any). sfeps being taken to improve it, and how f/,is is likely to affect sustainability.

.

Your overalljudgement: is this indicator for sustainability being achieved?

Fully I Highly I Moderately I Slighily I

^{Not at}

ail |

^{ruot appticable}

1.2

Gheck whether there is a year plan

containing

details

of

all

activities

needed

for

CDTI at

this

level.

This iMicator assesses whetherthe

poject

is functioning effectively, and whether management

access

ownerchip of the programme

-

^{both good for} sustainability.

Gharacteristics of the

indicator

Sources

of information

a. This more detailed plan should make provision for all key elements of onchocerciasis control: Mecitizan supply; targeted training _; ta rgeted HSAM; targeted monitoring/ supervision.

(Note that in the case of a country, which contains many ^projects, the NOTF plan will be less detaited).

The plan varies from yearto ^year, showing that it is targeted to the specific needs of each year.

Examination of written ^plans:

yearly, quarterly, monthly etc.

lnterview with senior health service and project staff at this level.

Sustainability evaluation instrument no.1

-

^{national level}

-

March 2003 version Find

Describe the plan for the oresent vear:

There is a written ^plan for CDTI activities at this level. The plan provides key elements of CDTI, is not a re-write of previous years plan and is targeted at specific activities. This plan exists within the overall Ministry of Health plan.

Though a plan exists with a bulk budget amount for all activities, it does not include activities that would need to be funded by the various partners. lt does not specify the costs to be bome by each partner neither does it indicates activities that would be funded post APOC. A supplementary ^plan for 2003 was produced just _before the evaluation team anived.

Describe the plan for the orevious vear:

There was a similar ^plan for 2002, which varied with respect to line item and bulk budget amount.

lf the olan is incomolete. or simplv a re-write of previous plans:

.

WhY is this?

Not Applicable

.

Which steps are beino taken to imorove the situation?

Not Applicable

Analysis

t When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

_{Reasons for} poor performance (if any), steps being taken to improve it, and how this is likely to affect sustainability.

.

Your overall judgement: _is this indicator for sustainability being achieved?

FullylHighlylModeratelylstighty

Check whether

all

partnens (govemme overall

planning

process.

indicator assesses the

gogrunme6

this is good for sustainabitity.

- ^if

^{each partner is about its role,}

Gharacteristics of

the indacator-

_Sources

of information

o.

^/-\1l patruets snout(I ContnOute tO the foutine planning of

a

projeci.

b.

Partnerc should be clear about their own roles, and those of the other parlners.

'

Examination of:

*

_Plans: yearly, quarterly, monthly etc.

*

_{Minutes of NOTF meetings.}

.

lnterview with senior staff at this level (p@ect, govemment, relevant UN agencies, NGDOs etc.)

The team was informed that the main partners

-

^{Ministry of Health} (oncho coordinator, Assistant

chief

^planning

officer NocP/NGDo

of the Ministry and _review meetings are also heid where state DPHIi'HC) and the tne

NGbo panl

fpioj"cfAdministrator)

aii

presented and adopted. There

parl"iprte

in were the ^ptanning.however no minutes of these meetings.

These plans are consolidated into the MoH ptans and sent to the state planning commission for approval. There was no evidence that participatory planning was done.

This is not the practice in the MOH

The Permanent secretary whg is strongly in.support of integration is planning to have a meeting with ^programme officers and supporting agencies to disluss intdjration.

When writing th

*

The evidence about how weil this indicator is being achieved.

- ::::**lJpoor

performance (if anv), steps uein[ taren to

i*pror"

it, and how this is likety to affecr

. ,

^Yorr

^or"t lljudg",

1.4

Check whether

specific

_n

funding

is withdrawn.

7l,s

assesses

ownerchip _{of it} and can mobilise the resources rf needs. management has begun to take Characteristics of the

Members of NOTF _h

programrne sustainability. _Thi.s planning should include: identifying resource gaps, strategies to cut expenditure, and strategies to find oepenoante sources of resorrrc,es.

There should be written evidence that such pranning has taken prace.

There should be evidence that the ptans are being iuccessrutiylmpemented.

Examination _{of tfre} written susiainability plans.

lnterviews with senior staff at this level (government, _NGDO, NOTF etc.).

sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

Describe the ^present situation

The state oncho coordinator has produced a plan for sustainability post

Apoc

that coverc the period of october 2003

-

January 2004. This is yet to be agreed and finaliseo witn irri: MoH and the NGDo. There is no ^ptan for the period beyond.

.

_Why is this?

The Permanent secretary explained that the MoH is waiting for LGA chairmen to be appointed before sustainability plans are developed.

The state oncho coordinator is also waiting for the planning meeting.

: Thereisaplantoho1dmeetingswffieyareappointed.

When writing the

reportffi

*

The evidence about how weil this indicator is being achieved.

*

_{Reasons for poor} performance (if any), steps bein[ taken to

i*proru

it, and how this is likely to affect

.

Your

overalljrdn"

Check whether staff at

this

level

ii

assesses whether programme is functioning of the

indicator

Staff members at this tev

level immediately berow them. Staff shoud nof supervise the FLHF.or community levels. ^,Spot checks' rnay however be done from time to time.

Staff members at this level should have empowered staff members at the level below them to supervise activities at their own level, as well as levels further down.

*

Supervisory checklists, plans and reports.

*

_{Visitor's books at}

_allthe

_levels

below this one.

*

Trip authorisations.

.

lnterviews with:

*

_{Staff at this level.}

1

Staff at levels below this one.

There are 5 SOCT members and each one is responsible for monitoring and supervision in 1 or 2 LGAs. A monthly workplan is prepared in which monitoring and supervision is inctuoeo ano canied out by the staff. Monitoring and supervisory checklist are used and reports written. Trrougn

wra;;pil;;;;;1;;i",

there were no reports of follow up activities.

sustainability evaluation instrument no.1

-

national level

-

^t

t"."nloog ,*oi*

Analysis

I When writing the report you have to summarise:

-

*

_{The evidence about how well} this indicator is being achieved.

*

_{Reasons for poor} performance (if any), steps being taken to improve

sustainability. it, and how this is likely to affect

'

Youroverall judgement: isthis

indicatorfors@

Fully I xishty lffoderaCi, Slightly I

^{Not at}

ail |

Not appticable

2.2

Check whether

monitoring/ supervision is

being planned anC

carriio out

in an efficiena and indicator assesses whether the

gogrartme

is

Characteristics of the

indicator

_Sources

of information

a.

one routine supervision visit per year

m@

(as a separate entity, or as a district).

b.

Supervision visits for cDTl should be integrated where appropriate with supervision of other programmes.

c.

Resources for supervision (human, transport etc.) should be efficiently used:

*

Using as few staff members as possible.

*

Planning trips to cut down on distance travelled.

*

Not spending unnecessarily many nights out etc.

d.

Supervision visits should be thorough, using a checklist.

.

Examination of:

*

Supervisory checklists, ^plans and reports.

*

_{Visitor's books at the level}

immediately below this one.

.

lnterviews with:

*

Staff at this tevel.

*

_{Staff at} the level. immecliately below this one.

The SOCT monitor and supervise distribution based on planned activities. This is mainly at the LGA levelwith spot checks to communities. There is a minimum health package in the State and sometimei all programme officers work together- For example recently in Obingwa (UNICEF-assisted LGA) alt programme officers visited as a team and carry out separate interventions.

Occasionally there are integrated monitoring and supervision, but this is usually tied to the Oncho programme schedule because of the project vehicle. Other programme officers oflen scheduled their monitoring

tips

to coincide Oncho activities.

Describe the situation the previous vear:

Same as above

Not Applicable

.

_Which steps are beinq taken to imorove the situation?

Not Applicable

When writing the report you nav-

to-uffiarise:

*

_{The evidence about how well} this indicator is being achieved.

*

_{Reasons for poor} performance (if any), steps being taken to improve it, and how this is likely to affect sustainability evaluation instrurnent no_1

-

^national

^level-

March 2003 version

.

Examine the trend

.

,Your

or"t"ll

2.3

Check

whetherthe*

indicated by the

monitoring

_systbm.

management

a.

As soon as proUn

supervision visits, or from coverage data (i.e. areas with row

-

:ovgrage).the appropriate manager should deal with them.

o.

uucn proDtems should usually be passed on to the appropriate managers at the next iever berow to dear with, with the necessary support _ thus empowering these persons.

c.

Successes should be noted and reported, and appropriate feedback given.

d.

There should be evidence

of

aclion taken based on recommendations _in the reports of previous _monitoring exercises.

. Examinationofttre@

]

l].ans: ^yearty, quarterly, monthly _etc.

*

Minutes of staff/ planning meetings.

*

_{Reports of} previous monitoring exercises.

*

Letters of commendation.

*

_Letters with information and feedback

.

lnterviews _with:

*

_{Staff at this level.}

*

_{Staff at} the levels below this one.

. _Whuj

_writin

] I[:r:9gl*

^{about how wefi this} indicator is being achieved.

. ffi:l*t ^tlJ;:or

performance (if anv), steps bein[ taren toimprove it, and how this is likety to affect Describe the presenGituation

The system in place for.solving problems _is the normal administrative channel. lf there is a problem at the first instance the

socr

in-charge or firat

r-cn ii

responsible for ensuring tne proolem is iesorveo. lf the problem is not

t?S'r'rTJlis

reported to the state oncho cooniinator _ano wneie'she cannot handte it, she passes

*

on rhe where successes are recoded, they are noted and the officer or team is commended. For exampte

in

2001

H'[YA':"ffif ,f,ff.Xfi.in

therapiutic coverase and the coordinatorwas nominateo to attend an

NorF/Apoc

.

_Whv is this?

Not Appticable

Not Appticabte

.

,Yorr

or"rattjrOg

sustainability evaluation instrument _no.1

-

^national

^level-

March 2003 version

3.1

Check whether

sufficien

stlstem at

this

level, in good time.

This indicator asses

becoming more integrated into the fov6mment ^sysfem.

The Mectizan su

i::r".**t}:p

,!e.t|r"-.11"t9* routinety.useo ror

t[e

supprv oi o:tn", drugs.

The system should be effecrive, uncomprieated and

eniciini.

This system should use dependable, sustainable resources for its operation. lt is.desirabte that the government should supply these

resou*r.

The system should supply sufficient Mectizan-for the needs or att fire projects concemed, ln good lime.

.

Examination _{of all} Mectizan ordering and slock control

documentation at this level.

.

lnterviews with staff at this level (govemment,

once the state is notified of the availability of Mectizan@ it is collected by the state from the NGDg office in owerri and kept in the oncho coordinator's

offid

from where the LeAs are quickry informed to come and collect their allocations' Though this is considered as the project system antt is seen

.. orp"norite

_tor quick delivery to the LGAs, it is not within the drug supply system of the MoH.

Same as above

The medicar store is not safe that is why the drugs are not stored there.

:.

None yet

Whenwritingtneffi

+

The evidence about how weil this indicator is being achieved.

-

ffj:ilt^tl#oor

performance (if any), steps oein! taten to improve ir, and how this is tikely to affecr

.

_{What fras Ue}

.

,Yorr

orarrlljuOgt

-

^national

^level-

March 2003 version Sustai nability evaluation

insttmntao.

1

4.1

Check whether staff members at

this

level are being used appropriately as trainers.

This indicator assesses whether the ffogramme is funclioning Characteristics of the

indicator

Sources

of information

a.

^{Staff at} this tevel should routinely only train staff at the level immediately below it.

b.

Staff membes at this level should have empowered the level

immediately below them to train lower levels.

Examination of training materials, plans/ programmes, reports:

*

At this level.

*

At the levels below this one.

lnterviews with:

*

_{Staff at} this level (the trainers).

*

_Staff at the very next level below (the trainees).

*

_Staff at the districU LGA level.

*

_{Staff at the} FLHF level.

The SOCT train the LOCTs on CDTI and they in tum the CDDs. The SOCT members are assigned topics during training. Training materials like reporting forms, flipcharts, posters and brochures were seen.

Describe the situation the vear before:

ln 2002 the situation was the same as above, but new LOCT members transfened from non-endemic areas were trained in addition to existing members.

lf staff members are not beinq used efficientlv as trainers:

.

_Whv is this?

Not Applicable

.

Which steps are beinq taken to improve the situation?

Not Applicable

Analysis

.

When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

_{Reasons for poor} performance (if any), steps being taken to improve it, and how this is likely to affect sustainability.

.

Examine the trend in the way in which staff are being used as trainers:

'

^{Your overall} judgement: _is this indicator for sustainability being achieved?

FullylHrghtvlModeratelylslignit

4.2

Check whether

training

is being planned and carried

out

in an

efficient

and integrated manner assesses whether

Wrymfire

is fmctioning

efficienily an

integrated manner.

Characteristics of the

indicator

_Sources

of information

a- There should be an objective need for each episode of training.

This means there should be evidence that slaff to be trained lack knowledge and skills to perform the job, and the training should then focus on this deficiency only. Repeat traininq of alreadv skilled

.

Examination of training materials, planV prograrnrnes, reports:

*

At this level.

*

At the levels below this one.

sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

b.

c.

staff should not happen.

lf circumstances permit training for CDTI should be integrated with othertraining, e.g. in in-service training programmes.

Resources for training (human, transport etc.) should be efficienily used:

*

Using as few staff members as possible.

lnterviews with:

*

_{Staff at} this level (the trainers).

*

_{Staff at} the very next level below (the trainees).

*

_{Staff at the districU LGA level.}

*

_{Staff at the FLHF level.}

*

Using as little time as possible (without sacrificing quality)

*

_{Choosing the most} cost-effective _{site etc.}

Describe the oresent situation:

Staff have acquired and adequate training skills. Training is sometimes targeted at specific aspects of the CDTI.

There is minimal integration of CDTI with other trainings.

Describe the situation the vear before:

Training of LOCTs covered the whole CDTI processes because of the LOCTs were transfeped from other non- endemic areas.

lf tralnlno.ir not b"ino

*rri"d ort

in

,n

"ffiri"nffi .

Whv is this?

There is no clear policy on integrated training.

.

_Which steps are being taken to improve the situation?

None

Analysis

.

_When writing the report you have to summar

* *

The evidence _{Reasons for poor} about performance how well this (if any), steps being taken indicator is being achieved.to improve it, and how this is likely to affect sustainability.

.

Examine the trend in training activities

-

.

Your overall judgement: is this indicator for

iu@

FullyIHighlylrvrooei

4.3Checkwhetherstaffatthislevelisplanninganocarry

Thi s i n dic ator assesses the yogramme is

are taking ownerchip of the programme. and effectively, whether managers

Characteristics of

the indicatoi

_Sources

of information

Staff members identify situations

information abouu cornmitment to cDTl, and undertake ac'tivities to inform and pesuade these persons.

HSAM activities are properly planned. They are only carried out where there is an objective need for them, and not as a matter of routine.

such activities should only be camed out at the national level, and at times at the level immediately below (but only when staff at that level asks for help).

There is evidence that these HSAM activities have been effective and have led to action.

b.

d.

.

Examination of HSAM plans/

programmes and reports.

.

lnterviews with:

*

Staff (programme and management) at this level.

*

_Civil authorities at this level.

*

_{Staff and civil} authorities at the next leveldown.

sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

10 Fi

Describe the ^present situation fin relation to efficiencv and outcome):

Briefings are canied out from time to time for decision makers and other staff in the Ministry to advocate, health educate, sensitize and mobilise them. For example when the cunent Permanent Secretary was appointed, HSAM was canied out and as a result she visited LGAs for advocacy and spot checks. ln other instances when ^planning division require clarification on issues HSAM is ^providecl.

Descilbe the situation the vear before (in relation to efficiencv and outcomeL

Advocacy visits were to LGA Chairmen and council members. The concept of CDTI is explained to them and support for CDTI activities solicited.

lf HSAM activities are not beinq canied out efficiently:

.

_Whv is this?

Not Applicable

.

_Which steps are beinq taken to improve the situation?

Not Applicable

Analysis

.

When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

_{Reasons for} poor performance (if any), steps being taken to improve it, and how this is likely to affect sustainabilitv.

.

Examine the trend in HSAM activities

-

^{is it} becoming more efficient and effective?

.

Your overalljudgement: is this indicator for sustainability being achieved?

Fully I Highly I Moderately I Slightly I

^{Not at}

all I ^tlot

^applicable

sffi4s

Describe the present situation:

There are no indications that support programmes are planned and executed in an integrated manner. There are however instances where SOCU memberc are involved in other programmes. For example, one member of the SOCU team is a health educator for CDTI and combines this with other diseases. The Data Manager also

data for other

5.1

Check whether the

various

programme

support activities

are

being

planned and carried

out

in an efficiently and

in an

integrated manner.

Gharacteristics of

the indicator

Sources

of information

a.

^{Staff combines two or} more tasks on a single trip:

x

Monitoring / supervision for CDTI (and other projects, if staff is responsible for them as well).

*

Training for CDTI (and other

p0ects,

if staff is responsible for them as well).

*

HSAM.

*

Fetching records.

*

Deliverino Mectizan.

.

Examination of documents: trip authorisations, log books, trip reports

.

etc.lnterviews with:

*

Staff from this level (managers, administrators, drivers etc.).

*

Staff from the next level below.

Sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

Describe the situation the vear before:

Same as above

lf inteoration between support activities is poor:

.

_Whv is this?

There is no policy on integration

.

_Which steos are beino taken to imorove the situation?

Permanent Secretary intends to hold meetings of all programme managers to ^plan for integration Analysis

r When writing the report you have to summarise:

* *

_The Reasons _evidence for ^poor _about performance (if _{how well this} any), steps being taken indicator is being achieved.to improve it, and

sustainability. ^how this is likely to affect

.

Examine the trend in the integration of aAiv

.

Your overalljudgement: is this inOicato

Fully l ^Highly

^I

F.:."::":".::.fn{.lqwgt{itr:ryqqlf4f

F:_fi fi

ffi

E*rt'.":::

6.1Checkwhetherappropriateamountsarebudgetedforp

Characteristics of the

indicator

-

Sources

of information

b.

d.

The costs for each onchocerciasis control a should be clearly spelt out in a budget.

There is evidence of a const reduction/ containment slrategy (e.g. targeted training, HMAS and monitoring/ supervision; training conduCted

it thsnext

level below etc).

Project managers should have a clear estimate of the funds that will be available to them for onchocerciasis control in the coming year, and should be able to justify this belief.

The total amount budgeted for in the year plan should fall within this estimated income.

.

Examination of the budget documents.

.

lnterviews with the health service

managers at this level.

F

The budget and estimated income:

.

_{For this} year:

The budgeting is canied out following the govemment's procedure. There is a budget line for onchocerciasis control activities, which is listed under subhead 12 in the recunent budget of the MoH. The budget estimate for 2003 is

Nl00,

000.00. There are olher CDTI activities, which are covered under other budget lines like travet,

tllling

stationery etc. This is to ensure that the project has other budget lines from which funds can be sourced for CDTlactivities.

.

For the previous year:

No amount was budgeted in 2002 for Onchocerciasis control activities, the State Govemment released N500, 5OO

from the 2001 budget.

lf budoetino has been inapprooriate:

.

Whv is this?

ln 2002 there was no budget allocation for CDTI. Other sub heads within the MOH budget covered CDTI activities like training, travel, and stationery. This also applies to the 2003 budget.

.

_Which steps are beino taken to imorove the situation?

Advocacy will be intensified

sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

Analysas

l2

.

_{When writing the report you have to summari}

*

The evidence about how well this indicator is being achieved.

*

_{Reasons for poor} performance (if any), steps being taken to improve it, and how this is likely to affect sustainability.

'

Examine the trend in the budgeteo

amouni@

'

Your overalljudgement: is this incticator

FullyIHightylrullo

Checkwhetherthegovernmentisbudgetinganddisbursingiu controlyearly,

and in good time.

This iMicator assesses whether the

pr

_,

ownerchip of the gagramme and can mobilise

the resourC* it

needs.

and wfietherthe govemment _is acceptinig

Gharacteristics of the

indicator

_Sources

of information

a.

The relative budgetary contributions of tfre

to onchocerciasis control should be clearly spelt out.

b.

The amount that the govemment has budgeted in one or more specific onchocerciasis control budget lines (e.9. cunent ancl capital) should be increasing yearly, as a proportion of total expenses. By the end of year 5 o.f

APoc

funding the bulk of onchocerciasis control eipenses at this level should be met from govemment funds; by the end of year ₃ at least half of it.

c.

The amounts actually disbursed from such budget lines shoutd be increasing yearly, _as a proportion of total expenses. (Note that actual disbursement is more important than budgeting, and is a real sign of political commitment).

.

Examination of:

*

Budget documents (govemment, NGDO etc.)

*

Records of disbursement and expenditure (ledgers, oderc, approvals for expenditure etc.)

.

lnterviews with managers at this level (govemment, NGDO etc.).

Budgetary contributions of the govemment _and other partners to Onchocerciasis control _{are not} clearly spelt out.

The amount budgeted by govemment for CDTI has not increased yearly, as a proportion of total expenses. Close to the end of Year 5 of

Apoc

funding the bulk of expenses on

cDil

^is met ny

nebc.

The budget and disbursements from Govemment are as foilows

.

For this year: (2003)

The sum of

Nl.5

million was requesled fnom the state govemment _but there has been no approval yet. However the MoH has so far expended N93, 930 on spares, fueJ, and vehicle maintenance in 2003.

.

For the previous year: (2002)

The sum of N76, 200 was used for stationery and entertainment during meeting and briefing. ApOC

.

Forthe year before that: (2001)

The sum of N550, 500 was requested for the project in 2001. N370, 100 was used for coilection of capital equipments from WHO Lagos, allowances, repairs, fuel and other miscellaneous expenses.

From NGDO records contributions on APOC-WHO sustainability programme is given as follows for lmo Abia project

YEAR APOC NGDO

STATES

1 $139,274.44

*$112,347.00

2 $87,940.53 $108,069.25

$5,244.08

3 $8s,032.0S $63,531.00

$1,093.s0

4 $69,779.s8 $197,202.26

$4,1s6.79

5 $26,275.63

$70,370.48

TOTAL $408,000.00 $550,000.00

$10,500.00

42o/o 56%

1.Zo/o

* inclusive of capital equipment

COMMUNITIES

$2769.96.00 0.3olo

Sustainabilityevaluationinstrumentno.1-nationalffi

13 lf the government proportion of expenditure is not increasing proportionately:

'

^{Why is this?}

This is due to dwindling allocations from the govemment. E.g. in 2002 there were no budget allocations for CDTI.

Other sub heads within the MoH budget covered activities like training, travel, etc.

.

Which steps are being taken to improve the situation?

A supplementary budget has been prepared and submitted to the govemment but there has not been any response

Analysis

.

_When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

_{Reasons for} poor performance (if any), steps being taken to improve it, and how this is likely to affect sustainability.

.

Examine the trend in govemment budgeting and disbursements:

.

_Your overall judgement: _is this indicator for sustainability being achieved?

FuilylHishtylModeraretylstishily[ffi

6.3

^Check whether in case of a

deficit

between estimated costs and the amount

provided

_{by the}

provision is

being made

to

meet it.

fhis

commitment

to

ownerch i p.

assesses whether the goventment is to mobilise the resources rf needg as well as rfs

Characteristics of

the indicator

_Sources

of information

a.

Project management at this level should be aware of the shortfall, if one exists, and of its size.

b.

Project management should have specific and realistic plans to bridge the shortfall.

c.

^{lf it} is planned that non-governrnent souroes of funding are to be used after APOC funding ends, written commitment for this should have been obtained at the highest level in these donor organisations. Projects in Year 3 of APOC funding should also ^be well on the way to achieving such commitment.

.

lnspection of:

*

The budget documents.

(Govemment, NGDO etc.).

*

_Records of expenditure ^(ledgers, orders, approvals for expenditure etc.).

*

Letters of agreement.

.

lnterviews with managers at this level (govemment, NGDO etc.).

There are no plans to source for any dependable funding ^post APOC. lt is however believed that budget shortfalls or deficit could be bridged from other sub heads within the state Ministry of Health budget and the 50/o the MOH keeps from all revenue it generates.

Describe the situation the orevious vear:

Deficit bridged from other subheads in Ministry budget

lf the shortfall cannot be met:

'

^{Whv is this?}

There has been no plans for the post APOC period

.

Which steps are beino taken to imnrove the situation?

The DPH/PHC is considering approaching the supporting NGDO to support some core activities

Sustainability evaluation inslrument no.1

-

^national

^level-

March 2003 version

t4

Analvsis

.

When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

_{Reasons for} poor performance (if any), steps being taken to improve it, and how this is likely to affect sustainability.

.

Examine the trend in shortfall and how it is to be supplemented:

.

Your overall judgement: _is this indicator for sustainability being achieved?

Futty I Highly I Moderately I Slightly I

^{Not at}

all I

Not applicable

6.4

Check whether

funds

disbursed

for

onchocerciasis

control

from

the

budget at

tbis

level are

Characteristics of

the indicator

Sources

of information

The budget holder should be using a control system with the following elements:

*

Approval of each item of expenditure.

*

Allocation of expenditure against specific budget headings.

*

Regular calculation of residual amounts under budget headings.

All the funds released yearly should be spent as budgeted.

a.

b.

.

lnspection of:

*

The budget documents (government, NGDO etc.).

*

Financial control records (ledgers, orders, approvals for expenditure etc.).

,

lnterviews with managers at this level (qovemment. NGDO etc.).

Describe the oresent situation:

.

Approval of expenditure:

The State Oncho coordinator requests for funds through the DPH/PHC. Funds from the State for Onchocerciasis control activities are approved by the Permanent Secretary and go through the accounting system in the ministry before it is released. However there were no clear guidelines as to how this ^is done because there is no project accountant for the State. ln February an APOC account was opened in the State and funds transfened. The APOC financial is due to end in September 2003.

.

Allocation of expenditure:

Funds are allocated based on budget sub heads and is limited to available funds

.

Regular insight into budget line balances:

Expenditures made with reference balances lf the funds are not beinq well manaqed:

.

_Whv is this?

Not Applicable

.

_Which steps are beino taken to imorove the situation?

Not Applicable

Analysis

.

When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

_{Reasons for} poor performance (if any), steps being taken to improve it, and how this is likely to affect sustainability.

.

Your overall judgement: is this indicator for sustainability being achieved?

Fully I Highly I Moderately I Slightly I

^{Not at}

all I

Not applicable Sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

7.1

^{Check if adequate} and appropriate

transport

and other material resources are available

for

necessary CDTI

activities

at

this

level.

l5

Thi s indicator assesses

renources

it

needs. ffogramme is functioning effectively, and whether it is able to mobilise the

Characteristics of the

indicator

Sources

of information

a.

^There are adequate numbers of functional vehicles available for necessary CDTI activities.

b.

^The vehicles are appropriate for the purpose they are intended to fulfil

-

^tough but not luxurious.

c.

^{There is} sufficient office equipment available, in working order:

computers, printers, photocopiers

-

also stationery and materials for training and HSAM.

d.

^The running costs for these vehicles and equipment are met from dependable. sustainable sources.

.

lnspection of:

*

_{Each vehicle in the} pool, each piece of equipment: its source; its functional status.

*

_Training materials and stationery stocks.

.

lnterviews with managers at this level (govemment, NGDO s.).

Describe the availability/ suitabilitv/ functionality of the ^present vehicles. equioment and materials. considerino tfie work still to be done in the coming 5-10 vears:

Available CDTI vehicle will not last for the next 10 years. lt will require replacement in the next two yearc. Computer and accessories will need to be replaced in the next two years.

Adequacy for CDTI

tasks*

Adequacy for CDTI tasks**

* APOC, MoH, NGDO, other

*

_{/s rf working? ls} there enough ot it for the job? ls

it

suitable for the iob?

lf transport. equipment and materials are inadequate and/or funded from sources which are not dependable

.

Whv is this?

.

_Which steps are beino taken to imorove the situation?

Analysis

I When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

Reasons for poor performance (if any), steps being taken to improve it, and how this is likely to affect sustainability.

.

Your overall judgement: is this indicator for sustainability being achieved?

FullylHighlylModeratelylS

sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

7.2

Check if

transport

and other material resources in use at

this

level are adequately and

appropriatelv

maintained.

This indicator assesses whether the programme

ii@

Characteristics of the

indicator

_Sources

of information

b.

There is a routine maintenance schedule for each vehicte, wtrictr is adhered to and recorded. This incluctes weekly driver maintenance, scheduled garage servicing, and replacement of wom tyres.

Equipment such as photocopiers and generators is regularly maintained according to a schedule, and this is recorded.

staff

members have ways of coping when vehicles break down or are not available, so that CDTI activities are not disrupted.

The costs for vehicte and equipment maintenance and repair are met by from dependable/ sustainable sources.

Repairs to vehicles and eguipment are rapidly and efficienflv done.

d.

e.

.

lnspection of:

*

_Vehicle and equipment maintenance schedules.

*

_{Vehicle and equipment}

maintenance records.

'

lnterviews with managers at this level(govemment, NGDO etc.), drivers

Routine maintenance is canied out for the vehicle _{and other} equipment, but there is no record of maintenance (no maintenance book).

The MOH and NGDO provide vehicle when needed to ensure that CDTI activities are not disrupted.

The MOH has been providing funds for vehicle maintenance

Analysis

I When writing the report you have to summarise:

*

_The evidence about how well this indicator is being achieved.

*

_{Reasons for poor} performance (if any), steps being taken to improve it, and how this is likely to affect sustainability.

.

Your overall judgement: _is this indicator for

sust@

Fully I Hishly I Moderatety

I

7.3

^{Check if} the

transpgrt

available at

this

level is appropriatety managed and used.

Gharacteristics of

the indicator

_Sources

of information

b.

Transport is used at this level, and to undertake

sup@

next level. lt should not be used for cDTl implementation activities at lower levels.

The use of transport is properly controlled:

*

_{Trips made for}

_cDTl

purposes should be properly authorised in writing by the relevant official.

*

_{Each trip} undertaken for CDTI purposes should be recoded in a logbook.

*

_Trip authorities and log book entries and regular.ly reconciled, and action taken if there are discrepancies.

.

lnspection of vehicle control documents:

*

Copies of trip

authorities (also noting destination and purpose).

*

_{Log books.}

.

lnterviews with managers at this level (govemment, NGDO etc.).

sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

t7

Describe the present situation:

Transport is adequately utilised for CDTI activities at this level. Authorizations for the use of the vehicle are however verbal based on approved workplan. There is no abuse of vehicle usage

lf the transoort is not being well managed:

.

Whv is this?

Not Appticabte

.

Which steps are beino taken to imorove the situation?

Not Applicable Analysis

I When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

_{Reasons for poor} performance (if any), sleps being taken to improve it, and how this is likely to affect sustainabiliU.

'

Your overalljudgement: is this indicator for sustainability being achieveOZ

FullylHighlylModeratelyls

7

.4

^{Check if} there are appropriate and

rcalistic

plans for the replacement of

transport

and other materia resources, when APOC

support

comes

to

an end.

This indicator assesses whetherthe _Wogrcmme

manage@

to find resources for it.

the gogramme, and are able

Gharacteristics of the

indicator

_Sources

of information

Management should know that replacements will be needed berore tne eno of the programme, and have specific, realistic plans to meet the need at that time.

It should be planned that the govemment will:

*

_Provide replacements for vehicles and equipmeni.

*

Maintain existing vehicles and equipment.

*

_Provide stationery and materials fortraining and HSAM.

lf it is planned that replacement will be from non-govemment sources, written commitment for this should have been obtained at the highest level in these donor organisations (end of Year 5), or negotiations should have started (end of Year 3).

.

lnspection of letters of agreement.

.

lnterviews with:

*

_{Programme managerc}

at this level

(govemment, NGDO etc.).

*

High-ranking Ministry officials and other decision makers at this level.

Vehicle is functional and will serve the proiect only for the next 1-5 years. The MoH is committed to replacing the vehicle and equipment when necessary. Replacement from other sources have not been explored

lf the ^plans for replacino vehicles. equioment and materials are unsatisfactorv:

.

Whv is this?

The issue of APOC pulling out afier five years not seriousty considered and therefore no plan made for reptacing capital equipment.

.

_Which steps are being taken to improve the situation?

None yet Analysis

.

When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

_{Reasons for} poor performance (if any), steps being taken to improve it, sustainability.

and how this is likely to affect

'

^{Your overall} judgement: is this indicator for sustainability neingrechieved 2

Fully I Hishly I Moderately

I

sustainability evaluation inslrument no.1

-

^national

^level-

March 2003 version

l8

8.1

Check whether staff at

this

level is

skilled,

knowledgeable and committed, regarding

the

implementation

of

GDTI

in

its area

of

operation.

This indicator assesses whether the gogramme has been able to develop sufficient resources for itself.

Characteristics of the

indicator

Sources

of information

a.

^The number of staff members in the govemment health service at this level should be appropriate to the task in hand: not too many or too few.

b.

Team members should have enough knowledge and skill to undertake all the key CDTI activities themselves, without help:

* Planning *

Data management

*

_Report

writing *

_{Computer skills}

*

_{Training and}

HSAM *

Mectizan ordering/ distribrution

*

Monitoring/supervision

c.

There should be evidence that the team is committed to the success of the programme (from the evidence of the ^partners, as well as workers at the next level below: from written reoorts and timetables).

.

lnspection of:

Staff files.

*

_{Training reports} and timetables.

.

lnterviews with

*

Managers and other staff at this level (government, NGDO etc.).

*

_Staff at the next level below.

lf the staff at this level lack skills and commitment:

.

_Whv is this?

Not Applicable

.

_Which steps are being taken to improve the situation?

Not Applicable Analysis

When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

Reasons for ^poor performance (if any), steps being taken to improve it, and sustainabilitv.

I

how this is likely to affect

.

Your overalljudgement: is this indicator for sustainability being achieved?

Fully I Highly I Moderately I ^Stigntty I

^{Not at}

all I

Not appticable

.

Number of persors wJto show that they are committed to thier work, and perform it wetl ^: No. of persons quatifred in

this area

No. of persons competent enough to

8.2

Check whether staff at

this

level is stable, and whether

provision

_{is made}

for

passing on CDTI

skills

when a trained person moves away.

This indicator assesses whether the gogramme _{has been} are able to maintain r'fs resources.

Characteristics of the

indicator

_Sources

of information

Staff at this level should remain in one post for at least five ^years.

There should be immediate orientation (in CDTI) of new, unskilled project staff members.

a.

b.

.

lnspec-tion of staff files.

.

The table in 8.1.

.

lnterviews with managers and other staff at this level.

Sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

t9

Describe the oresent situation:

Members of Staff are stable and would remain in their posts. Some of them have been in the programme for 13 years. There plans to train more people on CDTI in case the cunent members of staff are moved.

Describe the situation two vears aqo:

Same as above

Describe the situation when APOC fundinq started beino oiven:

There were 5 staff members then and they have demonstrated commitment to the project because of the motivation and job satisfac{ion.

lf the staff is not stable. and new staff is not beinq trained:

.

WhY is this?

Not Applicable

.

_Which steps are beinq taken to improve the situation?

Not Applicable

Analysis

.

When writing the report you have to summarise:

*

_{The evidence about how well this} indicator is being achieved.

*

_{Reasons for} poor performance (if any), steps being taken to improve it, and how this is likely to affect sustainability.

.

What is the trend in the number and quality of staff?

.

Your overall judgement: is this indicator for suslainability being achieved?

Fully I Highly I Moderately I Slightly I

^{Not at}

all I

Not applicable

9.1

Check whether

all projects

in the

country

(or

districts

in the project) have a satisfactory therapeutic assesses whether the programme

is

effec-tive

-

^{if the rates are poor} the project is clearly struggling,

Gharacteristics of

the indicator

Sources

of information

a.

Alt ^projects in the country (or districts/ LGAs in a project) should have a therapeutic coverage rate of 650/o or higher (the denominator being the total population).

b.

These rates should be stable or increasino.

.

lnspection of distribution reports and statistics at project level,

forthe

past 3 years.

.

lnterviews with staff at:

*

_This level.

*

The next levelbelow.

ind

The theraoeutic coveraqe situation in the ^{proiects (or} districts/ LGAs):

.

At the last distribution:

2003 distribution still ^going on. Final reports were not available

.

The vear before:

Sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version

Sustainability evaluation instrument no.1

-

^national

^level-

March 2003 version 20 ln 2002 therapeutic coverage was 640lo - 81o/o. However it was discovered that therapeutic coverage in some communities was calculated using eligible population as against total population.

.

The vear before that:

ln 2001 coverage was 610lo - Elo/o

lf the therapeutic coveraoe rates are ooor:

.

Whv is this?

Not Applicable

.

Which steps are being taken to improve the situation? (lf such steps are already being taken that is good for sustainability)

Not Applicable

Analysis

.

When writing the report you have to summarise the reasons for ^poor performance (if any); steps being taken to improve it: and how this is likelv to affect sustainabilitv.

.

What is the trend in therapeutic coverage?

.

Your overall iudoement: ^is this indicator of sustainabilitv beino achieved?

Fully (100o/o of ^projects have a therapeutic coverage rate 2650/o - stable

or increasino)

Highly (90- 99o/o of proiects)

Moderately (80-89o/o of

oroiects)

Slightly

(/0-

79o/o of oroiects)

Negligibly (<70o/o of oroiects)

lnstrument 2: districu LGA level

NOTE:

This instrument evaluates

the

level,

which

actually takes

responsibility for

implementing CDTI

in

its area

of

operations.

lt

is

that

level where health services are planned and provided. ^We are

going to

refer

to

it as the

districu

LGA level.

The level below

this

one is the one where the health centres/

clinics/

dispensaries are located. We are going

to

refer

to this

level as

the

_'FLHF'

(front

line health

facility)

level.

Geographical name of this districu LGA: lkwuano, Ukwa East and Umunneochi LGAs Project: Abia State CDTI Proiect, Nigeria

ffikpiBraide,YisaSaka,charlesFranzen,obinnaonwujekweandSundaylsiyaku

Date: 17 -20 June 2003

Abbreviations/ acronyms CDD

CDTI FLHF HSAM NGDO

*,'.".."."-:.'..

community directed distributor

community directed treatment with ivermectin first line health facility

health education, sensitisation, advocacy, mobilisation

-

i.e. activities that are aimed ^at getting all the key playerc to participate wholeheartedly in the ^programme

non-govemmental development organisation

hsq&lqffrqffig{#ffiI;:::

' ^{ttre miin function}

^of

^this

^{level is}

^to

^tafte

responsibility for

the implementation

of

CDTI

in

its area

of

operation.

.

However its

function

is

still

largely one

of

support

of the

FLHF level:

*

Providing targeted

training,

HSAM and

monitoringl supervision.

*

_Arranging

_for

an adequate

supply of

Mectizan.

.

The FLHF level is the one

that

is

for workinq with

the CDDs in the communities.

1.1

Check whether the year plan

for

CDTI appears as part of an overall

written

plan for the

activities of

the districU LGA.

management is beginningto accept ownerchip of the gogramme

-

^{both good for} sustainability.

Th i s i nd icator assesses the programme has become

integnted

into the heafth service, whether

Gharacteristics of the

indicator

Sources

of information

a.

CDT!should be integrated into the overallwritten plan (showing that staff at this level consider CDTI to be part of their ^yearly routine, like any other programme).

b.

^The plan should make provision for

allkey

activities: Mectizan supply; targeted training; targeted HSAM, taqeted monitoring/

supervision.

c.

^Year plans should be drawn up in a participatory way.

d.

^Year plans must take into account community requirements for the timino of distribution.

.

lnspection of:

*

The written year ^plans.

*

Minutes of planning meetings.

.

lnterviews with:

*

Staff at this level: managers and others (pharmacist, transport officer etc.).

*

_Staff at FLHF level.

Sustainability evaluation instrument ^{no. 2}

-

districll LGA level

-

March 2003 version

2 Findin

Describe the present situation:

The comprehensive ^plan for CDTI activities is separate and is included as part of disease control or miscellaneous (subhead 10 & 13) under the PHC in the overall LGA plan. The CDTI ^plans do not in most cases have budgets estimates for CDTi activities. The plans are not drawn up in a participatory manner.

.

_Whv is this?

No explanation

.

Which steps are beino taken to improve the situation?

Steps are being taken by the PHC coordinators to include CDTI control in the overall plan Analysis

.

When writing the report you have to summarise:

*

_{The evidence about how well} this indicator is being achieved.

*

_{Reasons for} poor performance (if any), steps being taken to improve it, and how this is likely to affect sustainabilitv.

.

Your overalljudgement: is this indicator for sustainability being achieved?

Futly I Highly I Moderately I Slightly I

^{Not at}

all I

Not applicable

2.1

Check whether the districU LGA health management team is taking

full responsibility for

the imolementation

of

CDTI at

this

level.

indicator assesses whether management is takrng

Gharacteristics of the

indicator

Sources

of information

a.

lt should be the rnanagernent tearn at this level, and not higher levels/ NGDO leadership, which is initiating the key CDT|ac{ivities: planning, targeted monitoring/ supervision, targeted training and HSAM, Mectizan ordering/ distribution.

b.

There should be a focal ^person for CDTI activities.

.

lnspection of year plans.

.

lnterviews with:

*

Management team at this level.

*

Person responsible for CDTI at this level.

*

_Staff at the project level

*

_NGDO leadership.

*

_{Staff at} FLHF level.

Descilbe the present situation:

The LGA Oncho Coordinator is in charge of activities at this level and initiates activities. There are 3 LOCT members who assisl the coodinator.

lf leadership at this level is nottakinq the initiative in implementino CDTI:

.

_Why is this?

Not Applicable

.

_Which steps are beinq taken to imorove the situation?

Not Applicable

Analysis

.

When writing the report you have to summarise:

*

The evidence about how well this indicator is being achieved.

*

_{Reasons for} poor performance (if any), steps being taken to improve it, and how this is likely to affect sustainability.

.

Your overall judgement: _is this indicator for sustainability being achieved?

FullylHightylModeratetylSrignit

Sustainability evaluation instrument no. 2

-

^{districU LGA level}

-

March 2003 version

indicator a s sesses whether gogramme _become more integratea

into@

f u n ctio ning effectively.

3.1

Check whether routine data

conceming

Gharacteristics of the

i-dicator

_Sources

of information

a. tne

reporting process should take place within the

govemment sysiem, not using other resources. Data being transmitted includes: coverage reports; Mectizan statistics;

training reports _; dislribution reports; financia _t reports.

.

Examination of reports and report forms.

.

lnterviews with:

*

Managerc and staff at this level.

*

_{Staff at} the project level.

Reporting is done on a monthly basis and sent to the State oncho Coordinator. This is however not within govemment system, i.e. through the monitoring and evaluation unit.

.

Why is this?

This was not the initialsystem adopted by the programme

.

_Which steps are beinq taken to improve the situation?

None

Whenwritingtnerep@

*

_The evidence about how welr this indicator is being achieved.

*

_{Reasons for poor} performance (if any), steps bein! taken to improve it, and how this is likely to affect

.

Your overall judgement: _{is th}

3'2 _gg!y!!y 9l?.:[yHP'jl,:.s.:tgpiore

_at

_the

FLHF tevet in an integrated manner.

indicator assesses

whe@

Characteristics of

the

staff at this level shourd routinery onty

sufffi

community level. 'spot checks' may however be done from time to time.

Although one routine supervision visit per FLHF per year is necessary, supervision visits should focus more on FLHFs where there are proven problems

-

each supervision visit must be justified.

supervision visits for

cDTl

should be integrated with supervision of other programmes _(e.9. through a shared checklist). Transport for supervisory visits should be shared with other programmes.

.

Examination of:

*

Supervisorychecklists, plans, itineraries and reports.

*

_{Visitor's books at FLHF}

level.

.

lnterviews with:

*

_{Staff at this level.}

*

_{Staff at the FLHF level.}

supervision is canied out al this level to the FLHF with spot checks to communities. These trips are in most cases routine without definite schedules or targeted to solve problems. There are no suoervisorv checktists or wriiten

and no deliberate efforts are

rre no supervisory checklists or written made to inteorate.

Sustainability evaluation instrument no. 2

-

^{districU LGA}

^level-

March 2003 version