Quantity vs. Quality: the never-ending debate

An anonymous commenter yesterday brought up an incredibly important point to remember when we discuss what kinds of publication records make our institutions (and tenure committees) happy.

"My point is that impact factors and so-called "top tier" journals are supposed to give some indication of quality and "toughness" in publishing. But there will always be someone who looks at numbers of papers (crap crap crap = 3) and others who only look for CNS and IFs. Keep track of your IFs, mark your CV with student "stars" and invited papers, and be on your toes constantly to defend why you chose the journal you submitted to."

I've now reached that stage in my career where a number of my friends are at a variety of institutions and nothing is clearer to me right now than every institution seems to have it's own crazy "rules" on this matter ("rules" are in quotes because these are not always officially sanctioned rules). I know some high quality programs where it seems to be all about numbers (i.e., 3 papers in low-tier journals are better than 2 papers in high-tier journals). I know some places have a formula combining number and journal "quality". And I know one place that apparently has a rule about where you are on the author line because I know a variety of their young faculty will throw people under the bus (including their own students) to get that particular slot. One of the messages that comes through loud and clear from my anonymous commenter is that there is no standard rule of 'success' and people can run into issues if the rules they are operating under are different from the ones their institution values (I would encourage everyone to read the comment in its entirety).

General Disarray and I have spent hours and hours discussing tenure criteria from various universities and what to do if you realize that your university does not value what you value in terms of the quality x quantity interaction (the teaching/research trade-off is beyond my abilities to deal with, so I will leave that for others). We decided that there are essentially two strategies: 1) to follow your institution's tenure criteria like they are commandments from on high or 2) focus on being as competitive as possible for the job market. We have been taking the latter path (I'll let you know how that works out for me as I get closer to the big tenure decision). Our decision was based on one core concept: if we are good enough to leave for a university of similar or better quality, we are good enough to stay. Now, there are risks with this approach, no doubt. We could potentially screw ourselves if our institution doesn't like our CVs and we actually cannot get a job somewhere else. But I would also argue that there are inherent risks to clinging to your institution's criteria - especially if in chasing those ideals you leave behind the science you love. I have friends who chased their institutions mantra of numbers above all else. To meet those criteria they  traded science they were passionate about for science that was easier to publish to large quantities in low-tiered journals. They have tenure now, but have drifted so far from the cutting edge that they can't figure out how to get back to what they loved. In part, I have to admit that I focus on option 2 for chasing tenure because it fits better with the type of science I want to do and the type of scientist I want to be...and an inherent realization that if I can't chance the questions I'm passionate about then my entire reason for doing all this in the first place is gone.

And this leads me to what I think is the most important lesson I learned from my advisor - do the science you love, publish it the best place you can, and never stop looking for the institution that is excited by that. There are so many ways to make a "successful" (and by successful I mean one that makes you happy) career in ecology that I honestly see no point in chasing someone else's ideal of "success" if it doesn't make you happy. So, if you're the woman referred to by the anonymous commenter, taking heat for only having 2 papers last year in the most prestigious journals in our field instead of 5 in some lower-tier journal, take heart, I'd hire you in a heartbeat. And to the guy who took heat for all his papers with undergrads in lower-tier journals, take heart there too, I think my university would probably over-rule me and hire you instead!

Journal Choice

There is an intensely fascinating discussion occurring over at DrugMonkey right now on the importance of journal choice strategy for manuscript submission. Instead of polluting their nice discussion, I thought I would use my own blog to provide an ecologist's commentary. Let me start this discussion with a quote from PhysioProf's original entry:

"If your papers are being routinely accepted with only minor revisions, you are almost certainly not aiming high enough with the journals you are submitting to. If you want to publish in high-end journals, the sweet spot to aim for is that only about 1/3 of your papers should end up accepted at the journal you originally submit them to, and the rest should have to filter down."

When I read this, I actually wondered if PhysioProf had bugged my home. This summer, General Disarray and I debated this point with regards to two paper I was an author on. I knew beyond a shadow of a doubt that both papers would fly into one of my standard haunts and wanted to shoot higher. Afterall, if I was sure they were sure things at one tier, wasn't that the best indication that it might be worth trying a little higher? A co-author (the same one in both cases) thought it a waste of time and wanted to dump them lower. I was swayed by the arguments because the co-author was someone I highly respected. General Disarray finally convinced me that this strategy only ensured that I would not get either manuscript into a higher tiered journal. (Hmmm, maybe PhysioProf didn't bug my house, but surely I would know if General Disarray and PhysioProf were the same person....right?). There are many assurances that make me happy in life (the love of my husband, medical insurance, 100% on-time arrival for a flight), but a guarantee for not getting a paper into a better journal is not one of them. So I currently have two manuscripts trickling through the system. (And yes, PhysioProf, most of my papers have been getting into the journals I first submit them to, which was also part of my reasoning for shooting higher...but surely you already know this since you have apparently bugged my home).

Part of what I wanted to emphasize for my more ecological friends is not to get distracted by the comments over at DrugMonkey that predominately focus on what they call C/N/S or GlamourMags, (Cell, Nature, Science; obviously Cell is completely meaningless to us and I am endlessly fascinated by Cell being on that list because to an ecologist it's like hearing Gold, Diamonds, and...Seashells? Really, you rank seashells with Gold and Diamonds? But that's for another discussion about the differences among fields and was mainly put in here as a friendly poke at my medical/molecular/cellular blogging friends.) Obviously, getting papers into Nature, Science, PNAS, and PLoS Biology are great career boosts for us but the argument is not that this is only relevant to that top tier. I think that CC said it most clearly: if you routinely get papers accepted on the first submission, you're obviously leaving impact factor points on the table. If you try to aim higher across the board (submitting what you think is a B paper to A and what you think is an E paper to D) you'll reap rewards more thoroughly. Conversely, if your first submission acceptance rate is zero or close to it, you're being too aggressive and annoying everyone. I have reviewed a number of papers in low-tiered taxon-specific journals that when asked on the review form about alternative journals I wanted to say: Yes, rewrite and send to Ecology! (For my friends outside the field, that is one of the top ecology journals).

While the approach of shooting a little higher than one might intuitively reach for is a good one for maximizing the "payoffs" for each paper, it should be noted that this strategy is not necessary to have a career as a scientist, especially an ecologist. How important it is to follow this approach all depends on whether or not you want a career at a high wattage R1, a less brilliantly lit R1, or a more laid-back R2 (which may sometimes experience brownouts). At my current locale, which is an R2, I could make a handy career for myself without ever stretching myself to more stellar journal heights. However, at one of the top R1 programs I might need to stretch more. The fact that I am currently stretching is not because I need to, but because I want to. Given the type of science and career I want for myself, PhysioProf has the right advice. Now, if only I could find that damn bug he put in my house.

Whispers from the NSF frontlines

After a more light-hearted post about ISI rankings, it's time to return to more serious topics: NSF.

The deadlines for many...if not all...of the ecology and evolution-related panels at NSF are looming and we have hit that time when those of us preparing to storm the castle are getting feedback from those that the castle repulsed. The funding "no"s are trickling out now from the last round at NSF and some of the feedback on why certain proposals have been turned down is...interesting.

Colleagues on at least two different proposals that went to at least two very different panels have now spoken to at least two different program officers. Their proposals were ranked with "excellent"s and given "funding recommended" stamps of approval by their panels. They were turned down because the PIs already had money and/or tenure. In other words, untenured people who did not have money were apparently being given priority. (I should note that this does not mean that they were the only ones being given funding).

I want stop here and make it crystal clear that I am NOT ranting about NSF (though I understand this would be a funnier posting if I was). When funding rates hit 8% (or lower, which I suspect happened on this last round), program officers have to make some tough choices. I may have my opinions on their recent decisions, but all I know with any certainty or clarity is that I REALLY do not envy them right now. So, what might this new, unofficial policy mean? Well, if the whispers are true, then this means that NSF has stepped in to prevent the kind of decimation of the assistant professor ranks that was prophesied over at the DrugMonkey blog. I voiced my skepticism at that site that such mass non-tenurings were feasible for universities, though I was also accused of living under a rock for having said opinions. I will qualify my previous stance on this to say that a) mass firings in the medical sciences may more common (I'm not in the medical sciences and I'm happy to assume that the people over at DrugMonkey understand their field better than I do) and b) I should add that I agree increased non-tenurings over the short-term (next 1-2 years) might happen but that over the longer-term, would not be sustainable for most universities. When I'm in one of my cynical but amused moods, I'll play out what I think would happen to the State University of the Short-Sighted if they implemented the mass-firings policy over the long-term.

I'm still digesting what the new filter at NSF might mean for individual researchers and also broader impacts on collaborative research (what happens if the lead PI has money but the co-PIs don't or vice versa), university practices (do people now have problems being promoted to full, and do I even care about that?), and a host of other things. Whether in the long-term I agree with what NSF is doing (which hopefully will only have to happen for the short-term), I at least find it comforting that someone in a position of power (i.e., not me) is thinking about these things too!

NSF funding rates and University tenuring policies

I read with interest a posting by DrugMonkey on the "undoubling" of the NIH budget. I poked around for similar information on the National Science Foundation, but did not find anyone who put the NSF budget in the context of inflation.

NSF's funding rates have been at ~8% for my field for the past few years. There were rumors that NSF was to undergo a doubling like NIH, but it really never materialized. I suspect, that given various budget cuts, NSF has at best kept up with inflation. At worse, it hasn't even done that.

General Disarray and I have speculated on what the 8% funding rate at NSF means for the tenuring process at universities. Our university, like many others, requires a "federally-funded" grant as part of the promotion and tenure process. Really, though, the requirements for the grant are usually more stringent than that. Grants run through other institutions do not count for the tenure process because the tenuring university does not "see" that money (seeing has nothing to do with it being visible on your CV and everything about whether the accounting office can roll around in the Facilities and Administration returns in some secret back room). At my institution, the grant must be "in operation" at the time of tenure (i.e., if you had a grant while an assistant professor but it ran out before you came up for tenure, it wouldn't count.) I have a friend at another institution who was told the grant must be at least $500,000 to count, meaning his $150,000 in NSF funding did not. (I always meant to ask him if he had 5 grants each $100,000 or 10 grants each $50,000, would that count, but it always sounded too much like an SAT or GRE question).

The current 8% funding rate has us curious. Can universities afford to be so picky about grants now or do they risk losing their untenured faculty because their expectations no longer meet funding reality? I haven't done the math on this, but I am very curious to see what tenure decisions really look like for assistant professors in my cohort. Will the less-mighty and more scarce dollar really play such a big role for us as it has for cohorts past? While I suspect that the answer is "no", I have no intention of being the guinea pig for our brave, new, less-funded world, so back I go to my NSF proposal.