Multilitteratus Incognitus
Traversing the path of the doctoral degree
Validity...or Trustworhiness?
27-08-2017, 01:30 dissertation, EDDE802, EDDE805, ethics, MDDE702, methodology, proposal, reliability, research, validity Permalink It's been a crazy few days! If it weren't for my brother coming down to hangout for a while I probably would have more in common with Nosferatu than a regular human being😹 (having been stuck indoors for most of the weekend). When I started off this summer I gave myself a deadline to be done with my methods chapter by August 30th (chapter 3 of my proposal). After reading...and reading...and reading...and re-reading (select articles form EDDE 802), I reached a point of saturation when it comes to methods. I really wanted to read all of Lincoln & Guba's 1985 book called Naturalistic Inquiry during this round, but it seems like I will just need to focus on specific aspects of the book.
So, in this whirlwind of activity, I went through the preamble to my methods section, my target participant descriptions, my data collection, my data analysis techniques, and any limitations. I added to these sections, explicated, went more in-depth in each section, I corrected issues that were brought up by Debra in EDDE 805, some outstanding issues and comments from the feedback on the parts I had worked on MDDE 702, and some of the initial comments I got back from my dissertation chair. **phew** That was hard work! The only parts that I still have left to complete in order to be "done" with my methods section are (1) The ethics section; (2) the validity/reliability/bias section; (3) a conclusion section for the chapter bringing it all together; (4) an appendix with a sample survey; and (5) an appendix with the participant consent form. I am considering adding (6) the REB application to an appending as well before I call this section "done". I am not sure if I will be done by August 30th (as was the original plan) but I think I will be damn close.
That said, there is one thing that is tripping me up, and that has to do with the validity/reliability/bias section. Bias is actually not that hard. I think I can write up procedures and things to be on the lookout for in order to avoid bias in both data collection and analysis. The thing that is much more concerning is philosophical: Do I go with Trustworthiness, Credibility, Dependability, and Transferability as what I talk about in this section (coming from the Lincoln and Guba tradition of qualitative research work)? Or do I choose the more traditional Validity and Reliability and discuss my methods in that frame of reference? Of course, for qualitative studies the Lincoln & Guba approach makes sense (at least it does to me, and it's references in a variety of other texts I've read on qualitative approaches to research), but at the same time quite a few of the texts that I've read (both on case studies and on qualitative studies in general) still use reliability and validity as terms in qualitative researcher. So, so I "translate" validity and reliability (from the texts) in Lincoln & Guba terms? Just discuss in the framework of Lincoln & Guba? or try to smash both together? Perhaps start with Validity & Reliability and transition to L&G terms since they make sense? I need to re-read Chapter 11 of Naturalistic Inquiry this week to help make up my mind (any thoughts are more than welcome in the comments).
As of this point I am at 23 pages (with 1 paragraph of lorem ipsum text, and quite a few scraps of though patterns for items 1 and 2 above), so I am thinking I should be wrapping this up soon and not getting logorrhea.
So, in this whirlwind of activity, I went through the preamble to my methods section, my target participant descriptions, my data collection, my data analysis techniques, and any limitations. I added to these sections, explicated, went more in-depth in each section, I corrected issues that were brought up by Debra in EDDE 805, some outstanding issues and comments from the feedback on the parts I had worked on MDDE 702, and some of the initial comments I got back from my dissertation chair. **phew** That was hard work! The only parts that I still have left to complete in order to be "done" with my methods section are (1) The ethics section; (2) the validity/reliability/bias section; (3) a conclusion section for the chapter bringing it all together; (4) an appendix with a sample survey; and (5) an appendix with the participant consent form. I am considering adding (6) the REB application to an appending as well before I call this section "done". I am not sure if I will be done by August 30th (as was the original plan) but I think I will be damn close.
That said, there is one thing that is tripping me up, and that has to do with the validity/reliability/bias section. Bias is actually not that hard. I think I can write up procedures and things to be on the lookout for in order to avoid bias in both data collection and analysis. The thing that is much more concerning is philosophical: Do I go with Trustworthiness, Credibility, Dependability, and Transferability as what I talk about in this section (coming from the Lincoln and Guba tradition of qualitative research work)? Or do I choose the more traditional Validity and Reliability and discuss my methods in that frame of reference? Of course, for qualitative studies the Lincoln & Guba approach makes sense (at least it does to me, and it's references in a variety of other texts I've read on qualitative approaches to research), but at the same time quite a few of the texts that I've read (both on case studies and on qualitative studies in general) still use reliability and validity as terms in qualitative researcher. So, so I "translate" validity and reliability (from the texts) in Lincoln & Guba terms? Just discuss in the framework of Lincoln & Guba? or try to smash both together? Perhaps start with Validity & Reliability and transition to L&G terms since they make sense? I need to re-read Chapter 11 of Naturalistic Inquiry this week to help make up my mind (any thoughts are more than welcome in the comments).
As of this point I am at 23 pages (with 1 paragraph of lorem ipsum text, and quite a few scraps of though patterns for items 1 and 2 above), so I am thinking I should be wrapping this up soon and not getting logorrhea.
Comments
The publication emergency
03-08-2017, 12:55 academia, PhD, pondering, publishing, research PermalinkPaul Prinsloo has a wealth of thought provoking posts on his facebook ;-) I wasn't planning on blogging until tomorrow, but this got my mental gears moving and thinking (not about my dissertation, but it's thinking nevertheless). This blog started as a continuation of a comment I left on Paul's facebook feed. The article that got me thinking is an article on the Daily Nous titled The Publication Emergency.
In the article a journal editor (in the field of philosophy) opines (although not with his editor hat on) that graduate students (I guess this means doctoral students) should be barred from publishing until they are done with their degree. He says that this is not a barring of people who don't hold a doctorate, but rather of people who are in process of earning their doctorate. So, in theory, some with an MA, but not pursuing a doctorate would be welcome to publish their stuff. So, even if an article is good and has merit, if its author is in process of earning a doctorate they'd get an automatic rejection.
It was an interesting read, and a good thought piece - and in all honesty I think that that since the author is a philosopher he is either trolling us or trying to get us to think more deeply about this topic. I certainly hope so at least because I don't think there is a way for me to disagree more with the position expressed (my disagreement has reached 11 😉 ).
The proposal seems to be trying to address the problem of the proliferation of credentials amogst the professoriate, of which I'd argue that publications is one kind of credential. And, this proliferration of credentials is making it much more competitive for those in the professoriate at a variety of levels. To deal with this problem the proposal is to arbitrarily eliminate one segment of the population, barring them from publications. It's arbitrary because someone without a doctorate, as mentioned above, is still NOT barred from publication, except if they are in the middle of earning one. So someone like me publishing prior to entering a doctoral program is fine, but if I enter that gray zone it suddenly is not? (not trying to make this about me, I promise 😛 ).
I see the problem as being more systematic, and the credentialing is a symptom of a larger issue (something we see at level beneath the doctoral level as well!) Doctoral programs have increased and saturated the market. Doctoral programs have gone from artisanal experiences, where few people applied, fewer were selected, cohorts were smaller and members shared the 'pain' of the experience. Sort of like going through an elite military program, but in academia. Nowadays doctoral programs have (mostly) morphed to being cash cows for universities. More PhD†students graduating means more competition in the tenure stream job market; that is of course unless your program was thinking ahead and designed doctoral programs whose goals are not just to create new tenure stream faculty, but also work in other facets of society. More programs means more candidates competing for fewer jobs and resources (journals being one of those resources). Hence, you need additional credentials to make it, such as doing post-docs (which I'd call exploitative labor), other type free work and volunteering, more papers published, etc., in order to get an edge. It reminds me a lot of high school when people joined as many extracurriculars in order to pad their college application 😒
So, now you have a problem, my friend! You see, more early academics‡ publishing because current professors claim that they don't have the ability to publish as much because the journal market is saturated - with the author of the article citing 500-600 submissions to a journal per year. Add this to the budding market of predatory journals, and you've got a really bad mix. An invalid argument that was proposed in the article is "well if the argument of a graduate student is good now, imagine how good it will be in the future!" I don't buy this. Academia has setup a perverse set of incentives, including keeping track of citations. If the goal is to write something good and get citations for it, why wait? Ideas don't have a monogenesis, so if you wait someone else will beat you to the punch. Why handicap younger scholars who should actually be apprenticed into this? Another proposal is to not have anything published before you got your current tenure-stream job not count, so that people can see what you can do on the job. This also made me roll my eyes a bit. 🙄 Why practice selective amnesia? Simply to sour the milk and prevent people from doing something?
I think academia needs to seriously look systematically at the perverse incentives it has provided over the past 40 yearsâ™ and deal with those rather than just dealing with a symptoms in ways that are arbitrary, and at best, a bandaid on a bleeding patient. Doctoral students should be encouraged to publish, it's good practice for those who want to go into the professoriate. Why not have the benefit of mentorship while undertaking this? Furthermore, the publish-or-perish modus operandi for academic units needs to finally be put to rest. Faculty positions need much more granularity than what the current system allows for. Faculty hired as lab scientists should be evaluated more on their research and publication performance. Faculty who do mostly teaching should be evaluated more on that. Tenure and academic freedom should not just the playground of research PhDs, and we shouldn't try to shoehorn everything into a research PhD framework. This to me seems like a vestige of positivism, and our profession needs to think more seriously about a more dynamic, better representative, professoriate. More holistic means of evaluation will most likely take the pressure of the publication system.
NOTES
†PhDs is just my short term for any sort of doctorate, I don't really want to get into the whole PhD/EdD thing...
‡ By early academic I don't just mean tenure-track folks, but doctoral students
â™ just an estimate on my part, based on my conversations with others in field more time than I
Archive
Nov 2020
Oct 2020
Sep 2020
Jun 2020
May 2020
Mar 2020
Jan 2020
Nov 2019
Sep 2019
Aug 2019
Jun 2019
May 2019
Apr 2019
Jan 2019
Dec 2018
Nov 2018
Oct 2018
Jul 2018
May 2018
Apr 2018
Mar 2018
Feb 2018
Jan 2018
Dec 2017
Nov 2017
Oct 2017
Sep 2017
Aug 2017
Jul 2017
Jun 2017
May 2017
Mar 2017
Feb 2017
Jan 2017
Dec 2016
Nov 2016
Oct 2016
Sep 2016
Aug 2016
Jul 2016
Jun 2016
May 2016
Apr 2016
Mar 2016
Feb 2016
Jan 2016
Dec 2015
Nov 2015
Oct 2015
Sep 2015
Aug 2015
Jul 2015
Jun 2015
May 2015
Apr 2015
Mar 2015
Feb 2015
Jan 2015
Dec 2014
Nov 2014
Oct 2014
Sep 2014
Aug 2014
Jul 2014
Jun 2014
May 2014
Apr 2014
Mar 2014
Feb 2014
Jan 2014
Dec 2013
Nov 2013
Oct 2013
Sep 2013
Aug 2013
Jul 2013
Jun 2013
May 2013
Apr 2013
Mar 2013
Feb 2013
Jan 2013
Dec 2012
Nov 2012
Oct 2012
Sep 2012
Aug 2012
Jul 2012
Jun 2012
May 2012
Apr 2012
Mar 2012
Feb 2012
Jan 2012
Dec 2011
Nov 2011
Oct 2011
Sep 2011
Aug 2011
Jul 2011
Jun 2011
May 2011
Apr 2011
Mar 2011
Feb 2011
Jan 2011
Dec 2010
Nov 2010
Oct 2010
Sep 2010
Aug 2010
Jul 2010
Jun 2010
May 2010
Apr 2010
Mar 2010
Feb 2010
Jan 2010
Dec 2009
Nov 2009
Oct 2009
Sep 2009
Aug 2009
Jul 2009
Jun 2009
May 2009
Apr 2009
Mar 2009
Feb 2009
Jan 2009
Dec 2008
Nov 2008
Oct 2008
Jul 2008
Jun 2008
May 2008
Oct 2020
Sep 2020
Jun 2020
May 2020
Mar 2020
Jan 2020
Nov 2019
Sep 2019
Aug 2019
Jun 2019
May 2019
Apr 2019
Jan 2019
Dec 2018
Nov 2018
Oct 2018
Jul 2018
May 2018
Apr 2018
Mar 2018
Feb 2018
Jan 2018
Dec 2017
Nov 2017
Oct 2017
Sep 2017
Aug 2017
Jul 2017
Jun 2017
May 2017
Mar 2017
Feb 2017
Jan 2017
Dec 2016
Nov 2016
Oct 2016
Sep 2016
Aug 2016
Jul 2016
Jun 2016
May 2016
Apr 2016
Mar 2016
Feb 2016
Jan 2016
Dec 2015
Nov 2015
Oct 2015
Sep 2015
Aug 2015
Jul 2015
Jun 2015
May 2015
Apr 2015
Mar 2015
Feb 2015
Jan 2015
Dec 2014
Nov 2014
Oct 2014
Sep 2014
Aug 2014
Jul 2014
Jun 2014
May 2014
Apr 2014
Mar 2014
Feb 2014
Jan 2014
Dec 2013
Nov 2013
Oct 2013
Sep 2013
Aug 2013
Jul 2013
Jun 2013
May 2013
Apr 2013
Mar 2013
Feb 2013
Jan 2013
Dec 2012
Nov 2012
Oct 2012
Sep 2012
Aug 2012
Jul 2012
Jun 2012
May 2012
Apr 2012
Mar 2012
Feb 2012
Jan 2012
Dec 2011
Nov 2011
Oct 2011
Sep 2011
Aug 2011
Jul 2011
Jun 2011
May 2011
Apr 2011
Mar 2011
Feb 2011
Jan 2011
Dec 2010
Nov 2010
Oct 2010
Sep 2010
Aug 2010
Jul 2010
Jun 2010
May 2010
Apr 2010
Mar 2010
Feb 2010
Jan 2010
Dec 2009
Nov 2009
Oct 2009
Sep 2009
Aug 2009
Jul 2009
Jun 2009
May 2009
Apr 2009
Mar 2009
Feb 2009
Jan 2009
Dec 2008
Nov 2008
Oct 2008
Jul 2008
Jun 2008
May 2008