Insightful Discussion on Operationalization

Admittedly, thus far in the course readings I have preferred Luker's readings to Knight's. This is not to suggest that a favourite must be selected, but I sometimes find Knight to be so heavy in theory that it can be challenging to grasp the main points of his chapters (although after a few attempts I usually get there). This week, however, I found both readings to be helpful and complemented each other well.

I found Luker's chapter "On Sampling, Operationalization, and Generalization" especially insightful this week. Luker discusses in the section on operationalization that home something is defined or named is very important (which I think we all recognize), but in particular for researchers. She demonstrates this by using as an example rape statistics in the United States by providing four different estimates of the number of rapes that occur each year (from 102,560 to 683,000). If considered individually, all of the statistic numbers provided in the chapter are certainly significant and may not otherwise be questioned. But when considered together, as four different statistics on the same topic, one would automatically question why there is such a discrepancy. This demonstrates that what is considered part of a statistic depends on how researchers and respondents operationalize a particular term. One of the reasons this section of the chapter really made me pause and think after I read it was because it clearly demonstrates the power of the researcher and the responsibility that this entails. When researchers prepare questions to ask respondents, they (we) must think about our definition of a term, and how different people would define and interpret the term because this can drastically change the results.

Finding my 'outcropping'

The readings this week got me thinking about all of the surveys that I have filled out in my life and the level of planning that went into them compared to the amount of thought that I put into filling them out (generally very little).  This makes me a bit sad for researchers out there who follow all of these guidelines and spend time piloting them and fussing about wording of their surveys, only to have them hastily filled out or ignored all together. 

I was also interested in the idea of the ‘halo effect’ and can say that I have definitely been guilty of this in my own survey responses. The method of crafting surveys that “trick” respondents into being truthful by having them respond as if they were someone else seems flawed to me, however, as some respondents would actually respond as someone else would instead of themselves – defeating the purpose of the trick. 

In terms of my own research proposal, the Luker chapter was more relevant this week. I am researching museums fundraising in rural Canadian museums, therefore my ‘outcropping’ for my sample is easier to find as it is already fairly defined. My interviews will be with museum staff from rural museums in Canada. I still have work to do, however, in narrowing my sample and deciding which museums to focus on since there are a lot of them out there!  

Frustrating

I read this week's Knight reading today and was surprisingly interested by it. While I had considered how important the structure of a survey was, I hadn't realized quite how systematic rigidly structured the existing frameworks are. As I had planned on using a survey as a part of my fictional research for my SHHRC proposal, I re-thought exactly how I would create my survey and, in my mind, completely revamped my research process. Unfortunately, when I returned to edit my proposal to reflect those changes, I realized that the limited space I had faced when writing it had restricted me to a very vague description of the survey to begin with, and I was unable to make any tangible changes to the proposal itself. I found this frustrating because in my mind, my process became much better, but I was unable to translate that improvement into the only non-theoretical element of my research, the proposal. I'm sure this knowledge will be valuable for the final proposal later this semester, but it annoys me that I can't employ it now.

Speaking of frustrating, anyone ever try and work with the Ministry of Education on a research project? I was hoping to gain access to the Ontario School Information System (OnSIS), a frequently cited source in much of the prior research for my proposal. According to the extremely vague guidelines to gain access to the system, I need to apply to a Local User Authority, at least one of whom "must be appointed to each publicly-funded school board". As my local school board is the TDSB, I attempted to find the LUA for them. Of course, it isn't mentioned anywhere on the TDSB website, or anywhere on the wider internet as far as I can see. I emailed some key people from the TDSB website, but of course none of them have responded. I even checked the Toronto Catholic District School Board, which proved equally unhelpful. At this point I don't know who the LUA is, I don't really know how to apply or whether I'm at all qualified to access OnSIS, and I don't even know how to access it if I was granted permission. The whole thing is just a frustrating mess.

Telephone Interviewed by MTO

Here's another interesting incident that occurred to me. My mom had just received a phone call from the Ministry of Transportation requesting to conduct a phone interview survey.  I was present with my mom when the call took place and I had no idea what was going on.  Apparently they mailed out a letter months back informing us of the phone survey.  Anyways, I ended up listening in on the call because my mom used speaker phone.  Based on the questions asked such as "where and when did you go yesterday? How did you get there?", I was wondering how consistent these answers would be.  With my mom's limited english speaking and with her answering the same questions on my behalf, I found that the answers wouldn't be that precise, but an estimation.  The questions asked, I would say it could've been for both qualitative and quantitative purposes.
 


  

Luker's Four Features

Luker’s discussion on the four features of a true research question really helped me put things into perspective. First, the research question proposes a set of relationships between concepts. Second, understanding that relationship in order to helps us explain something important about social life. Third, the research question allows a range of possible answers which can be examined to see if answers fit the data. Lastly, if properly answered will advance the state of play in one or more intellectual conversations that is already happening in some area of the academic world. Luker further states that just because the research you have in mind is of interest does not mean it is the right question; you have to come up with a clear question that fits these four features. Always narrowing your focus and asking the right questions, who cares? This gave me a clear understanding of how to frame my question and to see the relationship of the concepts involved.

Examining the sample SSHRC proposal on Cyborg, our group found that it was kind of unusual that the person introduced themselves as being in a wheelchair from the start. Another surprising element was that the proposal didn’t even discuss how the research was going to be conducted, no methodology... I thought it was one of the criteria's!

Face-to-face inquiry method

Knight reading this week talks about various face-to-face methods, and I’m excited to learn about these real methods and their pros and cons. I appreciate the humanistic approach to research taking interest into subjective interpretation, as I always feel that qualitative methods have the tendency to overlook the beauty of individual uniqueness. A good face-to-face inquiry is also a highly controlled process, largely relying on the interview/interpersonal skills of the researcher. As Knight suggested, the role of interviewer plays, his/her body language and physical emotion, the technique of facilitating interviewee(s) to elaborate while keeping them stay on the topic, and the flexibility to empathize… I can only imagine it takes a lot of training and experience to become a good face-to-face method researcher.

I’m also interested in the feministic method, memory work mentioned in the reading. It seems to be a highly subjective and even personalized approach. I wonder what kind of research needs to adopt this method, how to select sample (for it requires a certain level of commitment and trust among the participants, as well as towards the researcher), and how research can control the reliability and validity of data.

Ethics and Feelings!

I really enjoyed the discussion in class today about the differences between empathy and sympathy, and the ways the bias of the researcher can be involved in face-to-face research. These sorts of ideas are things I've never really considered, because I haven't really done any social research like the kind we discuss in class. The research I have done has been with end users, testing interfaces or labelling. That's not to say that there aren't issues of bias and things to consider when doing user research (I always changed what I was wearing depending on the interviewee), but I feel like the stakes aren't as high. I'm not going to be horribly offended if a person that I'm interviewing doesn't like the way the global menu is organized, like I might be if I were interviewing a person who thought that women shouldn't be allowed to have access to birth control. There are a lot more angles to research in the social sciences that I won't ever really get to explore, but I find the discussions about them to be fascinating. I don't see many problems with empathizing with research subjects, but I think once you start to sympathize with them, it's a bit of a slippery slope. It's deceptively easy to start subconsciously leading people to certain responses, and I think that the more you sympathize and identify with someone, the more you want to see that their point of view resembles your predetermined biases or conclusions.