Methodology for an Evaluation Study: India's Working Women's Hostels Scheme

Learning Outcomes

By the end of the case, students should

• Have a better understanding of the methods adopted in the evaluation study, especially in the context of policy research
• Appreciate the various steps involved in the process of evaluation
• Be able to explain how interview schedules and focus group discussions are used to elicit information for the evaluation study
• Understand how the findings of the evaluation study can be used for policy analysis

Context of the Research

In general, evaluation studies assess the processes, consequences and impact of policies. Evaluation research is considered useful in applied social research where it can be used to analyse existing policies or to examine the need for new policies for the benefit of society. Another significant feature of evaluation research is that it can help to explain social change. In many cases, evaluation research is undertaken to assess the success or failure of a programme, policy or project (Clarke & Dawson, 1999). It also helps to analyse the merits and demerits of the object of analysis. There are many types of evaluations that do not necessarily result in an assessment of worth or merit – descriptive studies, implementation analyses and formative evaluations, to name a few. Perhaps a definition that emphasizes the information-processing and feedback functions of evaluation would be helpful. For instance, one might say, evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object (Clarke & Dawson, 1999). The major objective of most evaluations is to provide useful feedback to a variety of audiences including sponsors, donors, client-groups, administrators, staff and other relevant constituencies (William, 2014). Such feedback often helps in the decision-making process. There is a broad consensus that the major goal of evaluation should be to influence decision-making or policy formulation through the provision of empirically driven feedback (William, 2014).

This case study explains the importance of evaluation research. A study was undertaken for the Government of India to evaluate one of its schemes, or programmes, on providing hostel accommodation for working women and a day care centre for children. This scheme was introduced in the 1970s, and until 2011, there was no study undertaken to evaluate the impact and outcome of the scheme. Therefore, this study aimed at evaluating the implementation process of the scheme as well as examining the outcome and impact of the scheme on working women in India. In the implementation evaluation, the research team tried to monitor the fidelity of the scheme and its mode of delivery. In the outcome evaluation, the team investigated the demonstrable effects on specifically defined outcomes of the scheme. Finally, in the impact evaluation, a broader assessment of the overall effects of the scheme was carried out.

Overview of the Topic of the Study

The development schemes in post-independence India opened a wide vista of education, training and employment opportunities for women. Many women from urban and rural areas sought such employment opportunities in big cities and small towns, but in doing so were confronted with various problems. In India, the entry of women in the occupational world of men is a recent and very modern phenomenon. Many needy women took jobs in various offices and enterprises. An analysis of the roles of working women reveals that they have to act as agents of institutionalized change. Because innovations disturb the established patterns of behaviour, change is typically resisted in every society, often more so in a tradition-bound society like that of India. In such a society, the very idea of a woman taking a job outside of and living away from home is not approved or accepted. Even now, traditional values and norms of behaviour govern the attitudes and way of living of a large portion of Indian society. Traditional inhibitions and stereotypes are still deeply embedded in the ethic of Indian society. Thus, the traditional values and notions regarding the status and roles of women have not changed much. Because the employment of women in itself is viewed as an anachronism in the existing traditional structure, working women living away from home, in hostels, are looked down suspiciously by the society. A stigma is attached to a woman and her family if she takes a job outside her hometown and if she rents a room or a house. In such circumstances, the employed women tend to suffer from various kinds of difficulties resulting from conflicting demands placed on them. Thus, working women who are compelled to live away from their families for employment need a safe and suitable accommodation near their place of their employment at a reasonable rate. Realizing this, the Government of India has launched several schemes for the empowerment of women. One of such schemes is that of providing financial support for the construction and maintenance of working women's hostels.

To provide suitable, safe and inexpensive accommodations to women residing in places away from their hometowns to be able to work, the Government of India in 1972–1973 launched a scheme to construct working women's hostels. Women being trained for employment and female students enrolled in professional courses were also eligible for hostel accommodation. Voluntary organizations (e.g. registered societies, public trusts), women's development corporations, universities, schools and colleges of social work, local governments, cooperative institutions, state governments and union territories administrations were eligible to receive financial assistance under the Working Women Hostel (WWH) scheme. Despite the burgeoning number of women working in both urban and rural areas, only 887 hostels in total have been sanctioned under the scheme since its inception in 1972–1973, with a sanctioned capacity of 64,922 women and 8442 children in the attached 321 day care centres. India's parliamentary committee on empowerment of women has recommended revamping the scheme because it found the following: construction of hostels was at a slow pace, hostels were often unutilized, construction was not being driven by demand, hostel buildings were expensive and unaffordable and the scheme fund was underutilized. Against this backdrop, the Ministry of Women and Child Development conducted a research study from 2011 to 2013 to evaluate the status and condition of working women's hostels sanctioned under the scheme of assistance for the construction and expansion of hostel building for working women with a day care centre for children (Ministry of Women and Child Development, 2011).

Objectives of the Study

As mentioned, since the inception of the WWH scheme, no study has been undertaken to understand the implementation process of the scheme and the outcome and impact created by this scheme. Therefore, the Ministry of Women and Child Development (2011) wanted to

• identify the key areas of achievements and shortcomings in the implementation of the scheme and suggest concrete steps to make the scheme more effective,
• assess the housing needs of working women in states and districts and project how many hostels are required for the entire country,
• compare the hostels run by the state government or its agencies vis-a-vis the hostels run by non-governmental organizations and suggest concrete measures on the modalities of implementation of the scheme,
• suggest policy measures to be adopted for revamping the WWH scheme.

Hence, to meet the above requirement, the following objectives were framed to conduct the evaluation study:

• 1.
  to study the style and pattern of management of working women's hostels
• 2.
  to conduct an in-depth study of the management of working women's hostels and to identify deficiencies in management
• 3.
  to examine the infrastructure and other facilities provided by the hostel authorities to the residents (working women)
• 4.
  to study the socio-economic profiles of working women who are residents of the hostels
• 5.
  to study the opinions and expectations of hostel residents regarding the facilities provided in the hostels
• 6.
  to assess the need for additional hostels for working women
• 7.
  to study the problems faced by management, staff and residents of the hostels
• 8.
  to suggest ways and means for improving the working of these hostels

Research Practicalities

I served as the project coordinator of this evaluation study which was carried out from April 2011 to November 2012. To conduct the study across India, a team consisting of seven members was recruited; each member was trained in data collection procedures and the objectives of the study were explained to them. Of these seven members, there were four females and three males. Team members – both male and female team members – also participated in a gender sensitivity programme. Because the respondents of this study were females, this 1-week programme was conducted to sensitize investigators to the needs of women so that they could properly record information from the female respondents. I have served as a gender sensitivity programme trainer for more than 7 years, so I decided to conduct the gender sensitivity programme for the team members myself. In this programme, gender issues and the reasons for the prevalence and perpetuation of gender inequality and gender discrimination were discussed in detail. Field trips to nearby villages and towns were also taken to observe gender discriminatory practices; at the end of the day, team members discussed their observations. Initially, female members of the team reported that they felt this kind of gender sensitivity should not be required for them, only for their male colleagues. But after the training period, in their feedback, they stated that the sensitivity programme had indeed helped them to understand the issues women face; these issues were mainly due to the unequal relationship existing between men and women. For instance, before attending the gender sensitivity programme, they thought that the problems associated with migration are the same for men and women. But after attending the programme, they changed their opinion, understanding that women migrants face more problems than their male counterparts. They also mentioned that the problems faced by women are not the same across the caste-based system which is prevalent in India. Women from low caste face more discrimination than the women from upper caste. They felt that Indian women cannot be categorized as a single category because their problems are not the same across India. Later the team members were also taken to working women's hostels to observe the facilities available for women and engage in an interactive session with hostel residents. Initially, male members felt that the problems stated by the women respondents were trivial in nature, but later they understood the significance. Training on how to talk to the women respondents in a polite and humble manner to elicit information from them was also provided. Team members were advised to be good listeners and observers.

Furthermore, because India is a culturally diverse country, team members were informed about cultural sensitivity issues so that there would not be any misunderstanding between the investigators and the respondents. The team of seven members was selected from every region of India – one member each from the north, the south, the east, the west and the northeast, and two members from the central part of India. The reasoning behind recruiting team members from different regions is that people from these regions understand their own culture and could then explain it to other team members when they begin data collection.

After the gender and cultural sensitivity training, team members were sent to northern and southern regions of India to collect data. Both these regions are quite different in their culture. Hence, these two regions were chosen initially to get a clear picture on how the hostels operate in different cultural environments. An interim report was prepared after 6 months of the commencement of the study and submitted to the government for appraisal. After receiving feedback, the team conducted the study in the remaining regions of India. Once the data collection was completed, a workshop was organized to validate the data and to seek suggestions and opinions from experts such as academicians, gender specialists and hostel owners. Based on their opinion and the study findings, suggestions were framed at the end of the final study report and the report was submitted to the Government of India. This kind of evaluation research can help researchers to negotiate with governing institutions in a systematic way to establish better practices and policies (in this case, for the benefit of women) as well as add gender perspective to government policies.

Research Design

This study adopted both qualitative and quantitative research methodologies. As per the available records, there were a total of 887 working women's hostels sanctioned by the Ministry of Women and Child Development. Of these, 607 hostels were complete, whereas 280 were under construction. This study included hostels spread across India, but specific attention was paid to the hostels which were completed. However, reasons for delay in the construction of the others were also analysed, along with the problems faced by the agencies in charge of constructing and maintaining those hostels.

The data were collected from women who were using the facilities and from the representatives of the agencies which were entrusted with the responsibility of running the hostels. The data were collected by using two interview schedules, one prepared exclusively for the women residents of the hostels and one prepared for the agencies. The schedules included both closed- and open-ended questions. Equal weight was given to each question in the structured interview; therefore, it is easy and ideal to compare between one and another. To seek the opinion of the respondents about the hostel facilities and operating procedures, open-ended questions were asked and their responses were recorded carefully by the investigators. The expectations from the residents for improvement in the working women's hostels were also asked in the open-ended questions. Focus group discussions were held to prepare case studies to illustrate the best and worst practices in the maintenance of the hostels. In order to know about the maintenance of hostels, focus group discussions were held first with the hostel management team and housekeeping staff of the hostels the team visited. Later, separate focus group discussions were held with the residents of the hostels. These focus group discussions ultimately helped in the qualitative analysis. Further data required for this study were drawn from secondary sources such as government publications, academic publications and reports given by the Ministry of Women and Child Development.

Research Questions

The following research questions were studied in the process of this investigation into India's WWH scheme:

• 1.
  what is the nature of organization and management of the working women's hostels?
• 2.
  what is the socio-economic background of the working women admitted as residents in the hostels?
• 3.
  what, if any, security and protection are provided to the working women staying as residents in these hostels?
• 4.
  are the norms prescribed under the scheme regarding facilities to be provided to the residents in the hostels followed by the management?
• 5.
  what are the problems faced by the residents of these hostels?
• 6.
  is there sufficient staff to take care of the routine management and administration of these hostels?
• 7.
  what is the annual financial outlay of the hostels? What are the sources of income and how is the money being spent?
• 8.
  what are the views of the management, residents and staff about the working of these hostels?
• 9.
  what is the impact on personal and family life of working women staying in these hostels?

Scope and Coverage of the Study

As stated elsewhere, at present, there are about 887 working women's hostels constructed under the WWH scheme of the Ministry of Women and Child Development throughout India. Out of these, working women's hostels in five zones (i.e. north, south, east, west and northeast states) were included in this project.

Out of 887 working women's hostels functioning in the country, only 236 hostels located in those five zones were taken as a sample for the present study. The number of hostels constructed in these zones was not the same. Therefore, 30% of the total hostels constructed in these zones were considered for this study and those samples were picked randomly. However, during the data collection phase when the team visited hostels, despite repeated attempts, some hostels could not be found due to change in location or incorrect addresses, few were under construction and some hostels were closed down. With regard to residents (i.e. working women), a total of 2510 working women were selected under the sampling frame, which was about 10%–15% of the total residents in each hostel. Depending on the availability of the residents, samples were selected for the study. At least one person who is in charge of running the hostel was contacted to gather information from the organizational perspective.

Data Collection

Interviewing was the principal tool used for data collection in this study. As mentioned earlier, two types of structured interview schedules (one for the residents residing in the hostels and the other for representatives of the agencies managing the hostels) were used. This helped in collecting authentic information about socio-economic background of the respondents, availability of facilities in the hostel, accessibility and utility of available facilities. Representatives of the agencies managing the hostels were contacted personally and personal interviews were conducted to get firsthand data. The problems faced by management in running the hostels and the grievances, if any, of the residents and the staff were recorded with the help of open-ended questions. Through focus group discussions among respondents, additional information about working women hostels was collected. A group of 8–12 hostel residents were contacted in their free time to participate in a focus group discussion that investigators facilitated with the help of a checklist of information to be collected. The main purpose of the focus group discussion was to learn about the best and worst practices of the working women's hostel, so investigators began the discussion by asking respondents a lead question about the existing practices in the hostel. Only if the respondents deviated from this focus did the investigator intervene; otherwise, the investigator remained as a silent spectator and recorded the discussion.

Data Processing and Tabulation

The data collected from different sources were processed through computers after careful editing and coding. Tabulation was done with the help of the analytical software SPSS for all quantitative information and presented in the form of tables based on variables. For qualitative analysis, computer-assisted qualitative data analysis software was used minimally. Most of the information obtained through focus group discussion and open-ended questions was narrated elaborately in the final report.

Analysis

After the completion of the whole process of data collection, the team performed data analysis. While analysing the data, the team followed the guidelines for policy analysis proposed by Eugene Bardach (2005) in an influential book titled A Practical Guide for Policy Analysis:

• 1.
  describe the problem (without diagnosing its causes or proposing a solution).
• 2.
  assemble the evidence: gather information in the form of factual data which has meaning.
• 3.
  construct the alternatives: each alternative must address not only the basic intervention strategy, but also indicate how the strategy will be financed and implemented.
• 4.
  select the criteria: evaluative criteria are used to judge the best outcomes.
• 5.
  project the outcomes: determine as far as possible the benefits and costs of the policy options. Indicate the values and dimensions of the indicator to assess the outcomes.
• 6.
  confront the trade-offs: measure trade-offs across outcomes rather than alternatives.
• 7.
  decide: select the best alternatives given the analysis.
• 8.
  tell your story.

Based on these guidelines, the data were analysed and the final report was prepared. The major objective of the study was to investigate the perception of working women hostellers about the impact of their stay on themselves and their families. The analysis examined the positive impact of the women's stay in the hostels with regard to physical, psychological, economic and social aspects of the women's lives. In almost all the cases, the women hostellers perceived that their stay in the hostels enriched their physical life by providing them physical security, fulfilling their basic physical needs and improving their health status. They have also stated that their stay in the hostel enriched their mental capacities, mostly ‘need for achievement’, ‘preparing for adjustment in life’, ‘capacity to accept and face the challenges of life’. In most of the cases, the women residents were ‘pulled’ by the desire for ‘economic independence’ while the remaining were ‘pushed’ by the necessity of ‘fulfilling the basic needs’ or ‘earning source for livelihood’. By and large, the women residents found that their stay in the hostel enriched their social life by providing them friends, opportunities to develop social contacts, social acceptance, status and prestige, a ‘family-like place’ to stay and exposure to social realities.

Data analysis also examined the negative impact of women's stay in hostels on their personal and family life, mainly with regard to the physical, psychological, economic and social aspects of life. Many respondents reported negative impact in the form of health problems, feelings of insecurity and being away from dear ones and families. Conversely, about 40% of the women residents perceived no negative impact on their physical life. Those who perceived some negative impact reported emotional problems mostly, such as loneliness and insecurity.

Agencies which are committed to women empowerment performed better than other agencies in maintaining working women's hostels. Most of them complained about lack of government support in extending further financial assistance for hostel maintenance. All positive and negative consequences were documented with adequate data, and the report was finally submitted to the Government of India.

Suggestive Measures as an Outcome of the Evaluation

When one thinks of a hostel in India, it is often associated with unappetizing food, rudimentary furnishing and appliances and cramped, dingy spaces. The research team thought that every hostel that is to be constructed and maintained in the future should try to change this notion. To do so, the team provided several suggestions regarding the construction and maintenance of hostels based on the outcome of their evaluation. For example, the location of any hostel should be such that commuting is not a problem (i.e. close to a metro or bus stop). There should be provision for natural lighting and ventilation in the rooms. Wholesome meals should be provided to the residents, as the health of women is a key concern in our society. Ensuring that the infrastructure and facilities provided are up to date is also a good idea. The environment should be maintained as pleasant and congenial, and the surroundings should be safe. Overall, cleanliness, hygiene, security and facilities are all important concerns when running a hostel facility. It may be feasible to repurpose large buildings that are not being used and rent them out as working women's hostels. This is an idea that has gained popularity in recent years.

Because the need for women's work hostels emerges from the changing face of women's work in India, the research team noted that these hostels could be the pull factors encouraging more women to explore employment opportunities in better locations. To increase awareness about the existence of the hostels, the team recommended proper and visible advertisements. For the women who stay in these hostels, issues such as security and reputation are very important, so the research team suggested that the hostels be in populated areas and that they establish themselves as reputable places for women to stay. The final and perhaps most important of the research team's suggestions: the regular evaluation of these hostels by the authorities to ensure the proper functioning of these hostels.

Exercises and Discussion Questions

• 1.
  In this case, gender sensitivity and cultural sensitivity training were given to the research team before collecting data through interview schedules. Explain their importance.
• 2.
  Explain the process involved in evaluation research.
• 3.
  How can the findings of an evaluation study be used to frame policies or schemes of social development?
• 4.
  Examine the limitations of the interview schedules in the light of this case study. Discuss how these limitations could be addressed.
• 5.
  Analyse the association between the objectives of this study and the recommendations of the study.