3.2. Analysis of the Health Information System
Objectives (i) and (ii) of this thesis aim at the analysis of the health information system of the Kabarole district health system (compare section 1.4. "Aim and Objectives of this Thesis"). As shown in section 2.1.2. "Analysis", knowledge about the health information system has to be acquired during the analysis phase. This includes knowledge about the structures which already exist, what they should achieve and where they fail to do so.
3.2.1. General Considerations for the Systems Analysis
Since the aim of this thesis does not request a solution for a specific problem within the health information system, the aim of the systems analysis must be to acquire a general picture of the existing health information system. Although the objectives give a broad direction towards a compilation of health indicators by means of a software application product, these indicators were not specified. The system analysis must therefore assess to which extent it is possible to identify a specific set of indicators, which indicators could be of interest and how these can be generated. The strong orientation of objectives (i) and (ii) at the existing health information system structures also limits these indicators to those which can be generated with the existing structures.
Therefore a qualitative approach for the analysis with a broad based and flexible orientation is most adequate to assess opportunities for an improvement of the health information system. In this regard, the following sections elaborate a methodology for the analysis of the health information system of Kabarole district. This section starts with some general considerations towards an analysis according to the multi level model (compare section 2.3. "A Multi Level Model for the Description of Information Systems"). Resulting from these, section 3.2.2. "Logical Stages during the Systems Analysis" lists the logical stages, which the analysis should undergo. Section 3.2.3. "Objects of Interest during the Systems Analysis" identifies areas of interest for the analysis, while section 3.2.4. "Assessment of the Objects of Interest" concludes with a specific qualitative methodology for the analysis.
Due to the various aspects which have an influence on the performance of a health information system, a structured approach for analyzing its components is necessary. Section 2.3. "A Multi Level Model for the Description of Information Systems" describes a way of structuring knowledge about a health information system which pays respect to the different levels of information processing.
In the terms of the multi level model, objective (i) of this thesis aims at the business, the information system and the procedure level of the health information system. On the business level the structures of the health system have to be identified, which require indicators for decision making. The information system and procedure level describes the structures, which must provide these indicators. Resulting from this, it is obvious that the analysis of a health information system has to start with the analysis of the organizational aspects of the district health system. The resulting description of the business level gives a broad idea of the structure of the information system. Figure 3.2-1 shows a very simplified structure of the organizational levels of a health system which involve the health information system:
Fig. 3.2-1: A health information system covers several organizational levels of a health system (modified from [MOIDU, WIGERTZ and TRELL (1991)]).
The organizational levels of a health system in figure 3.2-1 must not be confused with the levels of information processing in the multi level model for the description of information systems. In the latter, the organizational structure from figure 3.2-1 forms the business level as shown in figure 3.2-2:
Fig. 3.2-2: The organizational structure from figure 3.2-1 forms the business level of the multi level model for the description of the health information system.
After having identified the entities on the business level, which use the health information system, the structure of the underlying levels of information processing can be understood more easily. For an analysis of these it seems to be most promising to start with the physical tool level. Discussions on information processing will most likely focus on concrete tools, which are visible and tangible, rather than on abstract structures on the information system, procedure or logical tool level. For example, at a health unit it will be more likely for the systems engineer to be confronted with statements like the following: "We write the number of vaccinated children into this form. Then we send the form to the district health team. They enter it into the computer."
On the other hand it can not be expected that a documentation of the information processing procedures is available. This means that a description of the procedures has to be reconstructed during the systems analysis. Previous to and along with the analysis of the physical tools, the various sites have to be identified where these tools are located.
However, this approach is bound to neglect the informal information processing procedures. For example, it is not possible to assess information flow from the communities to the health units by looking at forms which are filled at the health units. It is obvious that an assessment of these informal information procedures is only possible with a profound knowledge of the involved culture. On the other hand it has to be seen that the objectives of this thesis (compare section 1.4. "Aim and Objectives of this Thesis") already anticipate the need to focus on the computer based part of the health information system. Resulting from this, the analysis of the more formal information procedures is of greater interest for this thesis. In this respect it must not be forgotten that the systems analysis in this thesis can only provide a limited view on the health information system of the Kabarole district health system.
3.2.2. Logical Stages during the Systems Analysis
The considerations from section 3.2.1. "General Considerations for the Systems Analysis" imply the following logical stages of analysis in accordance with the multi level model for the description of information systems:
- Analysis of the organizational structures on the business level.
- Analysis of the sites of information processing on the locality level.
- Analysis of the physical tools on the physical tool level.
- Analysis of application systems on the logical tool level which is oriented at the identified physical tools.
- Analysis of information processing procedures on the procedure level which is oriented at the identified application systems.
The stages are arranged in a logical rather than a chronological order. However, this does not mean that they are fully successive. The need for a further refinement and supplementation of results from an early stage because of results from a succeeding stage will arise as the systems analysis proceeds. During all stages the three aspects of systems analysis have to be kept in mind, regarding the actual and a hypothetically optimized setup of the health information system and the resulting differences.
3.2.3. Objects of Interest during the Systems Analysis
The different logical stages of systems analysis which were identified in section 3.2.2. "Logical Stages during the Systems Analysis" focus on different objects of interest. Since the stages are oriented at the multi level model for the description of information systems, these objects will result in objects on each level of the model. In this section the objects are listed which are of interest during a systems analysis of a district health system. They are arranged according to the identified stages.
Assuming the general structure of a district health system as it is presented in figure 3.2-1, the following objects are of interest on the business level:
- The District Health Team. It has to be analyzed how the organization of health services is managed. Organizational entities within the district health team and the hierarchy in which they are arranged have to be identified. This information is of great importance for the further progress of the systems analysis. Staff members can be identified, who provide further information in how information processing is done in the district. Also, documents can be identified as an additional information source.
- The Health Programs. It has to be analyzed what health services are provided, which health programs exist and how they are integrated into the district health system. Staff members have to be identified, who are responsible for the management of the respective services.
- The Health Units. Health units as the actual providers of health services have to be identified. The general setup of the health units, responsible staff and provided services are of interest.
- The Community. In the context of this thesis, the communities are of interest only in their function of being a target population. With regard to the generation of population related health indicators, it is of special interest how a target population can be estimated and where the weaknesses of an estimation might be.
- The External Parties. The embedding of the district health system into the national health system is of as much importance as the influence of donor agencies. These might have special expectations in the health information system on the one hand, and can provide additional information about the district health system on the other hand.
The analysis of the locality level gives clues on where to find the physical tools of the health information system. This consideration is based on the assumption, that the physical tools are available at the location of the organizational entities of the business level which are involved in the health information system. The identification of the various sites has an essential effect on the time scheduling of the systems analysis, if transport to remote places is necessary for an analysis. The location of the physical tools also plays a role in assessing the performance of the health information system. In the context of this thesis the following objects on the locality level have to be identified with regard to the further course of the analysis:
- The location of offices of the district health team members and external parties. The identification of the offices of the district health team members might be trivial, if all the offices of the district health team are in the same building, but is essential, if the offices are dispersed in the district capital. This has to be considered in the context of this thesis, since the district was hit by a severe earthquake prior to the start of the systems analysis. As a result, many buildings were damaged. The identification of offices of external parties, including those of the ministries which might be useful during the systems analysis, must not be neglected, since these can be expected in the capitol of the country. In the case of Uganda, they are even allocated in two different cities: Kampala, which is the capitol, and Entebbe, which is about twenty kilometers away from Kampala. A special trip to these sites has to be scheduled.
- The location of health units has to be analyzed either to organize transport to these locations, or to adapt the time scheduling of the systems analysis to the availability of transport.
While the above mentioned objects aim at the scheduling of the systems analysis, a more detailed look at the locality level is also of interest with regard to an evaluation of the information system. It has to be observed, whether the location of the physical tools facilitates their use.
The physical tools provide the most obvious manifestations of the health information system. After identifying these, it will be easier to get a description of the more abstractive procedure and application system levels. The following physical tools are of interest:
- Forms, registers and other conventional tools of data collection. It can be expected, that the conventional way of data collection dominates the health information system. Therefore a more or less complete collection of forms and registers can already be used to trace the main lines of information flow in the district. With regard to a further evaluation, sources for incorrect data recording have to be identified and checked during the analysis of the application systems.
- Computer systems. The identification of computer systems which are used as physical tools of the health information system enables a later identification of the software application systems which are installed on these. With regard to the expected implementation of a software application product in the later course of the systems engineering project, the existing computer hardware has to analyzed.
- Human Resources. The identification of staff members who are using and operating the above mentioned tools provides a list of informants who will be helpful in describing application systems and procedures. With regard to a following evaluation, the skills of these staff members are of interest as well.
The above list serves as a guideline for conducting the systems analysis. Therefore it is incomplete with regard to a full description of the physical tool level. Other physical tools will play an important role as well, but have to be identified in the further progress of the systems analysis. The same applies to factors influencing a proper operability of the physical tools.
The application systems of the health information system can be observed in orientation at the identified physical tools. The distinction into computer based and conventional application systems directs the interest of the analysis to the following objects:
- Software Application Products. The software application products which are already in use have to be observed. It is of interest what kind of data processing is done and where the potential for an extension can be. With regard to an evaluation it must be observed, whether staff members are adequately trained to work with these products and how they are integrated into the other application systems. With regard to an supplementation with additional software application products, it has to be observed how compatible these must be.
- Documentations of Forms and Registers. In an optimal setting the forms which are used in the health information system are documented. These documentations would already provide a description of the organizational plans according to which recording and reporting is done with the respective forms. However, it is unlikely that these documentations exist.
- Implementations of Organizational Plans. Where the above mentioned documentations are lacking, the actual implementations of the organizational plans have to be analyzed. It has to be observed, how the forms are used in reality. With regard to an evaluation, sources of incorrect data manipulation have to be observed.
The analysis of the procedure level gives a more abstract view on the health information system. As assumed for the documentation of organizational plans on the application system level, it can be expected that information procedures are documented very rarely, also. Therefore the identification and description of these procedures has to be abstracted from the description of the application systems as a result of the analysis of the application system level.
As mentioned in section 2.3.2. "The Information System Level", the information system level is not more than a summary of the objects on the procedure level. This summary enables a more clearly assignment of the information systems to the organizational structures on the business level.
3.2.4. Assessment of the Objects of Interest
While the previous section aims at what has to be analyzed, this section considers possible sources for information on these topics and how this information can be obtained. As mentioned in section 3.2.1. "General Considerations for the Systems Analysis", a qualitative approach is most adequate for this analysis. The decision for one or the other qualitative assessment method depends on the information source and the required information.
The analysis of documents about the district health system and the involved health information system can provide a very efficient way of gaining preliminary information. The need for the coordination of appointments and trips can be neglected as far as documents are concerned which are available in the district. The information from the documents has to be confirmed elsewhere, though, as the documents might not be up to date and comprehensive. Documents of interest for analyzing the organizational structure could be organigraphs about the district health team and job descriptions for employees. In addition documents on specific health services might be available as part of the communication between the district health team and the makers of health policies on the national level. The involvement of the German Agency for Technical Cooperation (GTZ) as an external donor agency might be an additional guarantor for documents in the form of project proposals, definitions and progress reports.
Of special interest for the analysis of the health information system is the analysis of the reporting forms and registers, as well as their documentations in order to gain an insight into the structure of the conventional tools of the health information system. With regard to an evaluation it is also necessary to review reporting forms which have been filled out already. This can give an impression of how complete the forms were filled out and where there might be inconsistencies and misinterpretations of the demands in the forms.
Subjected to an analysis of documents are also a variety of factors which have an influence on the performance of the district health system such as the cultural embedding, the estimation of target populations and the influence of external parties.
The document analysis has the big advantage, that it can be carried out partially in the home country of the systems engineer already. In the context of this thesis it can be assumed that the GTZ headquarters in Eschborn, Germany possess documents on the district health system in the Kabarole district. Copies of documents could be posted from the district health team, also.
Depending on the availability and quality of the documents to be analyzed, the analysis can be very time exhaustive and bears the danger of producing irrelevant results. Unstructured interviews with informants who are involved in the district health team might avoid these inconveniences.
In order to gain comprehensive, general information about the district health system and its information needs, key informants have to be identified who are in a position that enables them to have an overall view on the district health system. The district medical officer of the district health team is naturally the most obvious informant in this category. Other persons have to be identified during the analysis. Since the aim of the interviews is to obtain a general overview and to identify special expectations in the systems engineering project, they have to be conducted in an unstructured manner.
Besides providing a further refinement of the aim and objectives of the systems engineering project, expected results from these interviews mainly concern the business level. Of special interest is the confirmation of the broad organizational structures, which are identified during the analysis of documents, and the detection of further information sources. These can be additional documents and particularly staff members who are responsible for the coordination of health services or carry out special activities during information processing procedures.
The involvement of external parties into the work of the district health team has to be observed as well as the embedding of the district health system into the national health system. If no special party is of more than peripheral interest, it is sufficient to conduct unstructured interviews with representatives of these parties about their involvement. A visit at the ministry of health is of special interest with regard to the health information system. It can be followed up what happens to the data that were collected in the district. Also it can be checked, whether the forms of the national health information system correspond to those which are available in the district.
Information on population statistics and migrations in the district can be expected at the district administration. In the context of nationwide movements there will be information available at national ministries. Key informants on these issues can also be consulted in unstructured oral interviews. These seem to be more efficient than written interviews with regard to the limited available time. Written interviews might suffer from a long delay of the responses. For a more detailed analysis of population statistics, the oral interviews can give hints on consolidating documents such as population censi. This information will be available to some extent at the district health team, also. The consultation of the external sources will validate these data, though.
It has to be considered whether appointments for the interviews have to be made, or whether it is sufficient to conduct unscheduled visits. In any case it will be necessary to have a letter of recommendation from the district health team.
When the health services and programs of the district health system are identified, their coordinators have to be interviewed. The interviews have to be conducted semi-structured along the following topics:
- What are the aim and objectives of the service?
- Which is the responsible central organization and to which degree is the service integrated into the district health team? In this context especially the involvement of GTZ is of interest as to estimate the influence of the project on the organizational structures.
- What are the main lines of information processing? The forms, which are already collected at this time can be arranged in their order of relevance for the logical tool level. Missing forms can be supplemented.
- Where are the weaknesses of the current information system?
The health units in the district are of a major concern for the analysis. On the one hand the health units are the main providers of data. It is therefore of interest how reliable the collected data can be due to structures on the application system level and to the used forms. On the other hand the impact of the collected data on the management of the respective health units has to be observed. It is of interest if and how the data are being used already at the health unit before being forwarded to the district health team and also if a backward reporting is done. If there is no backward reporting to the health units, needs and ways for it have to be observed. In addition to these areas of interest, it is important for a systems engineer who is not familiar with primary health care activities in developing countries to get a general impression of health services in the health unit level.
The assessment of health units depends mainly on the available means of transport and on the time scheduling of services at the health units. It is therefore difficult to elaborate a methodology without knowing the organizational settings in the district. A mixture of unstructured observation of data collection procedures and semi-structured interviews of the health unit managers seems to be the most promising way. The semi-structured interviews should focus on the following topics:
- What forms are being used at the health unit? The forms, which are already collected at this time can be arranged in their order of relevance for the logical tool level. It can be compared, whether the existing forms at the health unit correspond with the forms that were collected at the district health team.
- How are the forms used? By arranging the forms according to their relevance for the application system level, the implementation of organizational plans for data collection can be analyzed. In the context of the semi-structured interview only the theoretical knowledge about organizational plans can be inquired, but not the actual implementation. This can be supplemented with observations.
- How is the workload for data collection perceived?
- How are data used in the health unit management?
- What can be possible uses of a backward reporting from the district level?
An unstructured observation of data collection procedures can provide supplementing information on what is actually done and reveal additional reasons for wrong data collection. The two main areas of interest for an unstructured observation are
- data recording during consultations at the various services and
- data summarization for reporting to the district level.
The main constraint during the interviews and the observation can be expected in biased information. The interviewed persons might try to reflect on what they are expected to do, rather than reveal what they do in reality. Also the behavior during the observations might change. In order to reduce this bias, it has to be made clear to the concerned persons, that the activities are carried out only for an assessment of the information system, not for an assessment of the specific staff members. Another constraint has to be seen in the incompetence of the author of this thesis regarding medical knowledge and the local language. Therefore an assessment of the quality of the collected data with respect to the quality of diagnoses is impossible. For the observation of data recording during consultations it has to be considered, that the privacy of the clients must be respected.
Because of the geographical distance and the required travelling time, not all health units can be visited. For a representative selection, health units from three different categories should be observed:
- big health units with a broad variety of provided services,
- medium size health units,
- small health units with only a very limited amount of provided health services.
The observation of health units which are known for weak reporting as a fourth category can provide additional information on where there might be weaknesses of the health information system.
The proposed time of five months for the deployment in the district health team provides the chance of a strongly participatory observation of data processing at the district level. Since the author of this thesis is put into the position of giving technical advise especially on computer related topics, observation is facilitated. The participation in district health team activities and meetings can provide a good way to identify needs at the district level. Discussions on topics related to the performance of the health information system are expected to arise almost automatically during the daily activities. With respect to the eventual termination of the deployment, it must be carefully secured, though, that the author is not taking over routine tasks of information processing. It must be avoided to create a relation of the functioning of certain information procedures with the presence of the author.