Developing and Executing a Data Collection Plan for Archival Research

Abstract

This case details the process of planning and carrying out archival research on municipal associations. Municipal associations are an understudied area of the political science and local government fields. This case study discusses the challenges and opportunities that arise from researching an understudied field, including the need to identify, locate, and collect primary data. The case study details the process of developing a Data Collecting Plan and executing a plan of archival research to conduct original, empirical analysis. By carrying out this process, I found and developed the data needed to carry out mixed-methods research and analyze the relationship between municipal associations’ memberships and their behavior in intergovernmental relations.

Learning Outcomes

By the end of this case, students should be able to

  • Develop a Data Collection Plan
  • Describe methods of digitizing archival records
  • Develop a data indexing and organizational system
  • Understand how qualitative research can contribute to a mixed-methods approach that employs quantitative analysis
  • Discuss the challenges of research an unstudied or understudied field

Project Overview and Context

I started my doctoral program with the broad idea that I wanted to study the relationship between provincial and municipal governments. Prior to starting my doctorate, I worked as a research fellow at the Institute of Island Studies. In this role, I co-wrote the Institute’s submission to the Prince Edward Island Commission on Land and Local Governance, which argued that municipal governments needed a stronger relationship with the provincial government to ensure that their local needs were heard and addressed when provincial land use policies were developed.

From this work, I began to wonder what factors contributed to the relationship between municipal and provincial governments. Were municipalities in some provinces more successful in lobbying for policy changes? If yes, why? What determined which policies municipalities advocated for? How did issues vary between provinces? These questions were in my mind when I began the local government coursework for my doctorate degree.

During my doctoral coursework, I became particularly interested in the role of municipal associations. Municipal associations, also known as local government associations or municipal leagues, are membership organizations for the elected officials—mayors and city or township councilors—from municipalities within a geographic area, such as a country or a province. Elected officials use municipal associations to collectively lobby senior levels of government on common objectives. The associations act as interest groups but have greater leverage than other interests groups because their members are elected officials. They have the credibility of directly representing the interests of voters.

I soon learned that municipal associations were an understudied topic within the local government field. No empirical research had been published on municipal associations in Canada. Basic questions about municipal associations in Canada and elsewhere had not yet been studied. For example, when discussing the fact that some associations in Canada are split along rural and urban lines, Andrew Sancton’s (2011) textbook on local government in Canada notes, “whether this makes much difference in the overall scheme of things is unclear” (p. 39). When I saw that question posed, I knew that I wanted to answer it.

Choosing an unstudied topic presented an opportunity to contribute original, foundational research to the broader field of local government. It also presented a unique set of challenges. Because there was no existing literature on Canadian municipal associations, and minimal, primarily descriptive, research on associations in other countries, I could not build on prior research questions or designs to develop my own approach. Instead, I had to continually narrow down the wide range of potential research questions and designs to determine the research questions in my dissertation.

Conducting Fieldwork: Accessing and Digitizing Archival Records

Planning Archival Research Travel

After selecting municipal associations as my dissertation topic, I needed to identify what data were available for analysis. Because I was unable to draw on secondary literature as a starting point for identifying available data, I began researching municipal associations on their own websites, to get a sense of how they are structured and how they interact with provincial governments. I found that the 18 municipal associations in Canada share similar broad behaviors. They vote on their policy priorities and have officers and full-time staff to carry out their lobbying agendas. With few exceptions, the associations hold and document annual conventions, publish annual reports and operating budgets, and distribute monthly or quarterly updates on the progress of their lobbying efforts. However, while the associations were founded between 1899 and 1957, only the last several years of records are available on their websites. If I wanted to conduct a longitudinal analysis, I needed to collect additional data.

I contacted associations to ask whether they had digital copies of their older records. From this, I learned that most associations only had digital records dating back 10–15 years. However, most kept hard copies in their offices and donated copies of their records to archives and public libraries. The volume of records for 18 associations, many of which were more than 100 years old, meant that the associations did not have the in-house capacity to digitize their own records. Thankfully, most associations would allow me to digitize their records if I visited their offices.

In addition, I could digitize the records available through provincial archives and public libraries. Most archives offer copying services, but they often cost upward of CAD $0.50 per page. After I estimated the volume of records available, I determined that it would be significantly less expensive for me to travel to digitize records in person than to pay copying fees. Ultimately, I digitized nearly 115,000 pages of association records myself rather than paying CAD $57,500 in copying fees. In comparison, my total travel expenses were just over CAD $3,500. I was able to keep the costs of my travel low, relative to the length of my fieldwork, by staying at hostels or on university campuses and traveling by bus, train, or walk-on ferry where possible. I also reduced my individual expenses by applying for and receiving a Graduate Research Award at my university that covered CAD $1,500 of my expenses.

Developing a Data Collection Plan for Archival Research

Within political science, most researchers develop a Data Collection Plan (DCP) prior to conducting fieldwork or archival research. A DCP is used to identify and conceptualize the independent and dependent variables and the factors and outcomes that will be used in a research question. Developing a DCP can include thinking about specific measures and indicators and figuring out how, where, and when to get the data required for the selected measures and indicators (see Table 1). The foundation of rigorous data collection is a systematic approach to avoid biases with a constant relation to, and emphasis on, theory building or testing.

Table 1. Data Collection Plan (DCP) Matrix.

Variable

Concept

Indicator

What: Data needed to measure

How: Data collection technique

When: Data collection sequencing

How long: time needed to get data

Dependent variable

Independent Variable 1

Independent Variable 2

With only descriptive secondary research on associations available, I needed to study their primary records in-depth to identify factors and outcomes and to develop potential independent and dependent variables. Reviewing all association records would help me in conceptualizing variables and factors. As noted, digital records for most associations were available for the last decade, but I could not conduct a full longitudinal analysis without first conducting field research.

In the long run, it was faster for me to collect all municipal association records before conceptualizing my dependent and independent variables, rather than first identifying my research questions. That is, I wanted to understand the factors that were influencing the behavior of municipal associations. It was my goal to evaluate how different aspects of association membership—for example, the percentage of members who were rural versus urban or the total number of members (independent variables) influenced how associations acted when lobbying senior levels of government or what issues they prioritized when lobbying (both dependent variables). As a first step in my DCP, then, I focused on the how (data collection technique), when (data collection sequencing), and how long (time needed to get data).

My approach was also shaped by the timing of the academic year. While enrolled in my doctoral program, I worked as a teaching assistant during the fall and winter terms. May through August was the only time period I had each year to conduct extended fieldwork. I selected municipal associations as my research topic in March 2013, which meant that I could either conduct my fieldwork before finalizing my research questions, or I would have to wait an additional 8 months to travel. Rather than wait until after I had determined my research questions and risk having a significant gap between finalizing a variable-based DCP and my ability to collect the records that aligned with that conceptualization, I proceeded with the plan of strategically collecting all data available on associations. This determined the “when” of my DCP.

Ahead of my fieldwork travel, I used the AMICUS Canadian National Catalog and Archives, a central search engine for all public libraries and archives in Canada. Through AMICUS, I located all available municipal association records in provincial archives, legislative libraries, and university libraries in each of the provinces I would visit. By searching for the name of each association, I was able to identify what records were available and the archive or library where each available record was located. When the records for an association were incomplete or missing years or record types, I contacted that association to ask where I could visit their office to access the missing records. I used the records listed on AMICUS to gauge how long to stay in each province, building in at least two extra days per city in case I faced issues in record collection or found additional documents on site.

I also made a spreadsheet that listed every document I wanted to digitize, where they were located, and their call numbers for those records. This would speed up the process of retrieving records once I was at the archives and libraries I planned to visit. In some instances, I was able to submit a request of the records I wanted to view along with the dates I would be in the archives so that they were pulled and waiting for me when I arrived on-site.

I also planned my trip geographically to minimize travel time. During my doctorate, I lived in London, Ontario in Central Canada. I first traveled to Eastern Canada. I was able to stay with friends in Halifax, Nova Scotia and use the city as a hub to travel to the other eastern provinces by bus or plane for data collection trips. For the second leg of my trip, I flew to the provincial archive located furthest west and worked my way east back to Central Canada, through British Columbia, Alberta, Saskatchewan, and Manitoba. I did not travel to Quebec because, in my initial research, I learned that its two associations do not make their records publicly available. From this strategic, comprehensive approach I was able to answer the first three categories of my DCP: how (digitizing records), when (immediately and in geographic order), and how long (4 months). In total, I conducted archival research at 26 locations in 12 cities over 114 days, traveling more than 15,000 kilometers by plane, train, bus, and ferry.

Digitizing and Indexing Records

While conducting my fieldwork, I set daily schedules for myself to digitize, check, label, and index the records I collected. I coordinated with the municipal associations to determine which days would be best for me to visit their offices and scheduled days at provincial archives and libraries around those dates. The archives were open to the public, and most were open 10–12 hr per weekday, creating greater flexibility in when I accessed their records than the standard business hours kept by municipal association offices.

I began my research using a portable digital scanner to scan each page but found this process too time-consuming. The digital scanner took 10–15 s to scan each page, and many older documents were written on legal-size paper, requiring multiple scans per page. I quickly switched to using my digital camera to digitize association records. With my camera, I was able to flip pages and photograph a page at a rate of nearly one page per second. To further speed up my process, I took a photograph of a brightly colored, blank piece of paper between each document. This made it easier to see the breaks between documents when sorting the photos.

After I finished digitizing for the day, I would check, sort, merge, label, and index all of the records I recorded that day. I used a software program to merge a set of digital photos into a pdf. I ensured that each merged pdf matched one document, such as an annual report, brief, or convention proceedings. In short, I produced one digital copy for each document I reviewed.

I then checked each pdf document to ensure that each page had been recorded legibly. I noted any blurry or missing pages and re-digitized them the next day. I then labeled each document using a system that identified: (1) the association, (2) the record type, and (3) the date. For example, “Alberta Urban Municipalities Association Annual Convention Proceedings 1993” or “Union of Manitoba Municipalities Board Meeting 1982-05.”

I organized the records by creating a series of folders on my computer, one for each association, and within each association a folder for each record type. The record types were then organized in order from oldest to newest. At the same time, I created an index of all associations’ records I digitized. This provided an overview of the records I had collected across all associations. I also created a spreadsheet for each association, with record types as columns and each year a row. From this, I could identify any gaps in the data and determine later how incomplete records might impact case study selection.

The process of merging and indexing typically took 2–3 hr at the end of day in an archive, legislative library, or municipal association office. After completing this process, I backed up all of the records I digitized that day on both an external hard-drive and a cloud storage account. This two-step process ensured that I was able to digitize all available records within my limited summer timeframe. It also ensured that I would not arrive back in Ontario with incomplete or illegible records or lost records if my computer was damaged. It also enabled me to begin developing my research questions immediately after completing my fieldwork, rather than completing the merging and indexing process following the travel. Once I had digitized all available records, I returned to Ontario to identify my research questions.

Determining Research Questions

After I completed my archival research, I needed to determine how I would use the data I collected to analyze the behavior of municipal associations. I could now focus on completing the rest of my DCP: independent and dependent variables, concepts, and indicators (see Table 1). I knew that I wanted to examine the relationship between municipal associations and senior levels of government, but the minimal research within the field again presented a unique set of opportunities and challenges. Effectively any aspect of municipal association I studied would be an original contribution to the local government literature, but I did not have prior research to build from or use as an indicator of research questions that provided meaningful analysis.

Instead, I reread what minimal literature existed to identify questions about municipal associations had been posed but not yet answered. In her descriptive analysis of local government associations in the United States, Anne Marie Cammisa (1996) noted,

while government lobbies are interested in particular policies, they, unlike other groups (or at least to a greater extent than other groups), are also interested in the spatial dimension of any policy, that is, who will have the authority in implementation and control of funds. (p. 25)

When municipal associations lobby senior levels of government for policy changes they have a clear choice: to ask senior levels of government to act on their behalf or ask for the resources and authority to enact programs at the municipal level.

I determined that one of the research questions in my dissertation would focus on factors that impact this jurisdictional aspect of lobbying senior levels of government. That is, when associations lobby for policies or programs, which level of government do they want to carry out the proposed changes? My preliminary review of association documents during fieldwork indicated that member cohesion shapes how individual municipalities understand and approach their association. Do most municipalities perceive themselves as having similar interests to other members? From this, I decided to measure the distribution of member populations against the jurisdictional aspect of lobbying requests. My dependent variable was the lobbying activities of municipal associations, in particular the jurisdictional aspect of these activities. The first independent variable was the diversity of associations’ member populations.

My qualitative research enabled me to use mixed methods, including the quantitative technique of hypothesis testing. In particular, I tested how the similarity of member municipalities’ populations impacted whether they believed other members shared their interests. I expected that associations with similar member populations would make specific requests for the provincial government to carry out uniform policies that treated all municipalities the same, under the assumption that most other municipalities shared their interests. If municipalities perceive the interests of other members as similar to their own, they can request specific policy changes that would impact all members uniformly.

Second, in associations with a high variation in member populations, it can be expected that members will request the resources to institute programs at the municipal level. I expected that they are unlikely to think that the majority of other members share their interests and would want greater control over policy implementation. The distribution of member populations varies across all associations which enabled the research question to be tested empirically. As noted, I was able to carry out quantitative analysis in my research because of the volume of qualitative research I conducted.

Practical Lessons Learned

Planning Archival Research Travel

Using archival research and document coding, I was able to build and analyze an original dataset on a previously unstudied topic. This methodological approach and the outcomes of my research design met the original excitement I had when selecting this topic. However, I faced a particular set of challenges and trade-offs from the lack of secondary literature and prior research in this area. These included the time-consuming nature of archival work, the importance of fieldwork preparation, and the trade-offs associated with case study selection.

As noted, I conducted more than 4 months of field and archival research. I was afforded this opportunity due to the schedule and funding of my doctoral program, but the time-intensive nature of this work shaped my DCP. If I had selected my dissertation topic earlier in the academic year, it is possible that I would have been able to determine my research questions—and the records needed to carry out that research—before starting my fieldwork travel. This approach could have reduced the total number of records I needed to digitize, and subsequently the time needed to conduct archival research.

Accessing Archival Records

The second practical lesson I learned was the importance of checking and verifying the digitization policies of archives. The Provincial Archives of Alberta was the seventh of nine provincial archives I visited during my research travels. Because I had been able to digitize records at the previous six archives, I incorrectly assumed that I would be able to make digital copies of association records at all provincial archives.

When I arrived on-site in Edmonton, Alberta, I learned that records at the Provincial Archives of Alberta could not be photographed, scanned, or copied. I was only able to take handwritten notes on documents housed in the archives. I had scheduled 2 weeks in the province and wanted to digitize more than 100 years of records for each of 2 associations in the province. This period of time would not be sufficient for recording that volume of documents.

Because I had identified that all of the records I needed were available at the Provincial Archives, I had not checked whether they were available elsewhere. I quickly contacted Alberta’s Legislature Library and confirmed that they had all of the documents I needed and that I was able to digitize them. If this was not the case, there would have been a major misalignment in the records I needed to collect and the time needed to collect them.

I also learned to work closely with archivists to ensure that I was collecting all relevant records. When I began my research at the Provincial Archives of New Brunswick, I was unaware that there were three municipal associations that were formed in the late-1940s and then dissolved in the mid-1960s. Because I was unaware of them and what they were called, I had not identified their records when using the AMICUS to determine what records I needed to digitize. However, the archivists at the Provincial Archives informed me about these associations and I was able to digitize those records. The existence of these short-lived associations became the basis of one of the other research questions in my dissertation. By speaking with archival staff about my research, I was able to expand my research and build on their knowledge of what records were available.

Conclusion

I learned a lot from the months I spent planning for and conducting archival research. I learned that studying an underresearched field is exciting, although it requires more upfront planning because you cannot build on the prior work of others. Upfront planning can reduce the time and cost associated with archival research considerably, but there were events I could not have anticipated.

Building extra time into my research plan was vital for handling unanticipated issues in archival research, particularly because I was traveling between multiple cities. In addition, building in time to reflect on my digitizing process enabled me to identify places where I could save additional time—for example, using a digital camera instead of a scanner. This extra time also enabled me to digitize the records that archivists identified as related to my topic that I had not found through online systems.

Having a clear, straightforward organizational system was also a major timesaver. Once I returned to Ontario, I was able to dive right into developing my research questions because my files were already sorted. By indexing all of the records I collected, I was able to get a comprehensive picture of the primary data available to me for analysis. Finally, archival research and the travel associated with it was an enriching experience. If you have the interest and opportunity to pursue it, it is an opportunity I would recommend pursuing.

Exercises and Discussion Questions

  • I chose to strategically collect all association records before identifying my research question or variables. Would you follow the same approach? Why or why not?
  • I evaluated the records of associations and use resolutions an indicator of an association. Do you think I should have interviewed municipal association officials to ask firsthand how they conceptualize the interests of other members? What do you think would have been the advantages and disadvantages of an interview-based approach?
  • What resources (e.g., at your university or library) might you use for identifying records at other locations? What challenges do you foresee with the use of these resources?
  • What organizational system would you suggest to organize a large number of documents or articles for research? Why?

Further Reading

Barrett, C. B., & Carson, J. W. (1997). Identifying a site and funding source. In C. B.Barrett & J. W.Cason (Eds.), Overseas research: A practical guide (pp. 626). Baltimore, MD: Johns Hopkins University Press.
de Widt, D., & Laffin, M. (2018). Representing territorial diversity: The role of local government associations. Regional Studies, 52, 15851594. doi:10.1080/00343404.2018.1462488
Elman, C., Kapiszewski, D., & Vinuela, L. (2010). Qualitative data archiving: Rewards and challenges. PS: Political Science & Politics, 43, 2327. doi:10.1017/s104909651099077x
Moravcsik, A. (2010). Active citation: A precondition for replicable qualitative research. PS: Political Science & Politics, 43, 2935. doi:10.1017/s1049096510990781
Shott, A. K. (2015). Municipal associations, membership composition, and interest representation in local-provincial relations (Unpublished doctoral dissertation). University of Western Ontario, London, Canada.

References

Cammisa, A. M. (1996). Governments as interest groups: Intergovernmental lobbying and the federal system. Westport, CT: Praeger.
Sancton, A. (2011). Canadian local government: An urban perspective. Don Mills, Ontario, Canada: Oxford University Press.
locked icon

Sign in to access this content

Get a 30 day FREE TRIAL

  • Watch videos from a variety of sources bringing classroom topics to life
  • Read modern, diverse business cases
  • Explore hundreds of books and reference titles