Feminist Research Ethics
6. Critically evaluating your data: reflexive data analysis
(a) Fact checking and contextualizing your data
Fact-checking is an important step in assessing the quality of your data. Fact-checking ensures that information such as dates, events, and descriptions are accurate.
Fact-checking involves cross-referencing statements with sources such as municipal records, news media coverage, and other research to determine the validity of the data. An example of the importance of fact-checking is when using internet content as a data source, or verifying the historical dates of events mentioned in an interview.
It is also important to contextualize the primary data you have gathered from interviews and your fieldwork observations by using secondary data sources, such as census data, or other government statistics. Understanding how your data is connected with secondary data and information can help you understand the broader social, political and economic context of your participants’ experiences and strengthen your analysis.
(b) Dealing with recalcitrant data and recognizing cognitive bias
It is inevitable that researchers bring their own assumptions and biases to the data analysis process. However, it can be a challenge to recognize how your own cognitive biases shape the interpretations of your data.
One way to become aware of your cognitive biases is to watch for recalcitrant data, or data that does not fit with your current analysis. As you begin to see patterns of common themes or ideas emerging in your analysis, and start to form hunches about particular arguments you can make, make note of cases that do not fit, seem ambiguous or that appear to contradict the patterns you’ve identified in your analysis.
“Do not be afraid of complexity and ambiguity about themes. The world is complicated and your writing must reflect that complexity. There is a tendency of social scientists to describe patters as if they were uniform and monolithic. To do that slights the complexity of things. Don’t fall in love with early, plausible theories.” Karp, in Hesse-Biber 2014
Use the data that does not seem to ‘fit’ to ask further questions about your analysis, test your theoretical frameworks, and to deepen your understanding of the data that you already have.
Here are some things to consider when you encounter examples of recalcitrant data:
- Consider why and how these cases contradict others.
- What debates do these raise?
- What might these differences be saying about your initial hunches?
- How does this challenge your current understanding of your research?
- If possible, consider what other questions can you ask future research participants – or perhaps follow-up questions with a past participant – to clarify these contradictions. If not, then what conclusions can you draw from this supporting material, as well as that which refutes it?
- Do these cases raise questions that show directions for future research?
(c) Sitting with data
Sitting with your data – that is, suspending your analysis to reflect further on your data, or simply walking away from your data for a period and returning to it later – can be useful strategies to gain some distance from your data and your interpretations of it. Creating distance from your data is important to create space for reflexivity about your data analysis process and to gain perspective on the data itself.
Sitting with your data or walking away from is an important part of the data analysis process. That is, data analysis continues even when you spend time away from your data because it gives you space and time to develop your analysis further and to see your data from different perspectives.
Coming back to your data with fresh eyes and a fresh perspective often makes it easier to see new and interesting connections in your material. Be careful not to take such a long break that you have to spend hours reviewing your analysis again, but don’t be afraid to take some time to sit with your data, and see what comes of it.
(d) Listening to silences, recognizing limitations, and revision
Listening to your data includes paying attention to the silences in your data, or the things that are not said. Listening to the silences in your data may reveal new issues or connections that are not readily apparent the first time you analyze your data.
It is also important to recognize the limitations of your data and ensure that your analysis adequately supports the arguments or conclusions you wish you wish to make.
Recognising silences and limitations in your data can be important ways to deepen the complexity of your analysis by forcing you to ask new questions, or to revise your existing questions, analytical and theoretical frameworks.
Constantly reviewing and, where necessary, revising your analytical approach, your conclusions and even your research questions is important to ensure that analysis is as thorough, rigorous and well-founded as possible.
Review and revision during the data analysis process are also central to to meeting ethical obligations, such as a commitment to a politics of empowerment, transformation and decolonization, by enabling you to exercise self-reflexivity about your positionality in relation to your research, about the political meaning of the way in which you are interpreting your data, and the way in which your analysis is connected with the communiites you are studying.
References:
Hesse-Biber, S., ed. 2014. Feminist Research Practice. 2nd ed. Thousand Oaks: Sage Publications.
Murchison, J.M., 2010 Ethnography Essentials: Designing, Conducting, and Presenting Your Research. San Francisco: Wiley.