Did you read this fascinating bit of....whatever...from Nature? I picked this up from writedit's thread. The title and subtitles are, I kid you not, this.
Nature's choices: Exploding the myths surrounding how and why we select our research papers.
You might almost think they had read my recent post on the business of spin. Apparently they think they are taking some unfair knocks about their process for publishing papers and want to spin the story back around to their liking. Fair enough.
I don't buy their argument so I'll take my hand at spinning it back my way.
Now, what are these evil myths you might ask? They have a list.
One myth that never seems to die is that Nature's editors seek to inflate the journal's impact factor by sifting through submitted papers (some 16,000 last year) in search of those that promise a high citation rate. We don't.
Right. Sure you don't. You don't have to* "sift through submitted papers" to satisfy your shared interest with other GlamourPubs. Every bloody thing you do is directed at this goal! From defining the acceptable article as that which is the latest and greatest, to prioritizing first over best, to schmoozing PIs, fields and techniques which have a recent track record of astronomical citations, to engaging in co-publication shenanigans.....yeah. GlamourMags, of which Nature is unmistakably one, all seek to inflate their impact factor. Of course nobody can lay a glove on them with an accusation of a specific practice like sitting around the editorial table writing down an expected citation value. It doesn't work like that. This doesn't mean that the substance of the critique isn't valid though. Doesn't mean that editors aren't steeped through and through in policies and decision making that are designed to select papers for publication which will be cited frequently. So no, sorry, your protestations aren't going to do much to dispel the 'myth' that your journal is motivated to publish papers which you expect will result in high citation numbers.
but wait, they have data which will blow this idea clean out of the water...
Indeed, the papers we publish with citations in the tens greatly outnumber those in the 100s, although it is the latter that dominate our impact factor. We are proud of our full spectrum.
HAHAHA! So because you can't predict perfectly this is evidence that you aren't making the predictions in the first place? Or that since you have a broad mission across fields with different citations practices / rates / timelines and because "tens" might be the very top of some fields we're supposed to believe you don't know this? And oh, just btw, how hard do you work to make public the entire distribution of citations to show us just how proud you are of your "full spectrum"?
On to the next howler.
Another long-standing myth is that we allow one negative referee to determine the rejection of a paper. On the contrary, there were several occasions last year when all the referees were underwhelmed by a paper, yet we published it on the basis of our own estimation of its worth. ... But we make the final call on the basis of criteria such as the paper's depth of mechanistic insight, or its value as a data resource or in enabling applications of an innovative technique.
I think this is a straw argument and a straw defense. Of course any disgruntled rejected author is going to view one negative and two seemingly positive reviews as evidence the editors should have accepted their paper! This happens all up and down the taxonomy of scientific journals. The only knock would be whether Nature does this at a different (higher) rate than other journals. This defense does not address the real issue, instead rebutting the easy scenario of "we did overturn a negative review that once so there". Pfagh. [As always, I'll leave it as an exercise for the reader to determine if arguing for definitive proof based on single examples, instead of plural data, is discordant with what Nature seems to consider good science *cough*representative gel*cough*error bars*cough*cough*]. How often do you do it? How often do real journals do it?
Considering this issue we come to the next obvious question which is whether some negative reviews are more equal than others. This may be getting closer to the truth of this particular myth. After all, the Nature
editors spend several weeks a year in scientific meetings and labs
so when they assert
our decisions are not influenced by the identity or location of any author... we commonly reject papers whose authors happen to include distinguished or 'hot' scientists...another myth is that we rely on a small number of privileged referees in any given discipline. In fact, we used nearly 5,400 referees last year,
it does not really tell us anything about what happens with close calls or highly contested reviews. Is the cadre of hot scientists listened to more regularly on those cases? What happens when a paper is initially rejected (or gets a devastating request for unending additional data)? Do they take the phone calls of the distinguished scientists more frequently? Are you more likely to accept their revisions or resubmissions? And in any case, what a tired defense in this continued mode of throwing up irrelevant chaff. Who cares that you used 5,400 referees. Who cares that you occasionally reject the papers of a Nobel Laureate or your top Ten cited PIs ever? The question is whether on the whole, statistically, considering your entire publication approach...is there an edge? 10%? 20%? 50% improvement in acceptance rate? Personally I doubt they even bother to look at their data and they sure haven't presented even a taste of a real analysis here...
Myths about journals will continue to proliferate.
Yes they will. Including the myths about the "best" science perpetuated in your own interest. As with all myths, there is often a lot of truth behind them. It can take more than a trite denial of the "nuh-uh" variety to deflate the ones that are untrue and actively harmful. I appreciate they feel a low cost, low effort martyred denial helps with the spin game. It sure doesn't get down into the truth of the matter though.
Since I'm all about productive suggestions, let me end by reacting to the final statement:
We can only attempt to ensure that the processes characterized above remain as robust and objective as possible, in our perpetual quest to deliver to our readers the best science that we can muster.
Objective? Okay, I can help out there.
A, Numero Uno- stop with the interpersonal interactions about manuscripts. No informal email or phone inquiries about editorial interest from the authors and no soliciting papers on the part of editors. No more conference schmoozing or spending time in labs. No arguing on the phone or in email for reconsideration after rejection. Everything done by the book of the submission process which is open to all.
Second- look at the reviewer data. How many reviews are coming from the same lab group or reviewer? In mixed-review conflict situations, whose opinion coincides with the editorial decision? How often is it the junior, unknown, third world institution scientist out of these fabled 5,400 referees and how many times is it the "distinguished or hot scientist" that sways the editor?
Third- get real about subtle misconduct timelines and stop pretending you don't know anything about it. You *know* when you've sent something to Group A to review, they sit on it, ask for more experiments and surprise, surprise, have a hurried draft to you juuuuuuust in time for you to decide co-publish. Or even if this stuff goes on between GlamourMagz...would it really kill you to pick up the phone and call your counterpart once a raging author has leveled all sorts of accusations? Wouldn't you all want to tamp down that whole rush-to-first competitive stuff? Since, you know, you are after the best possible science and not just trying to beat your GlamourMag competitors to the get.
Fourth- you know, maybe it IS time to get real about blinded review. I tend to be skeptical because I think the radical restructuring of the typical scientific paper that would be required for blinded review, i.e., totally de-citing it, would break something important. That link to other scientific work. But I thought to my self, "Self, you know the manuscript submitted to a GlamourMag already doesn't look very close to the published article (in startling contrast to real journals), does it? So heck, just go ahead and de-cite that puppy for the review and we can put that back in at some later stage, after the heavy lifting of review is complete".
Let me end on one final counter-spin.
The supposed "myths" of the process of getting published in a GlamourMag journal could be just sour grapes from those authors who have been rejected or from those subdisciplines that will never publish in such journals. Could be. I think that is what this Nature editorial would like you to believe and to infer that these "myths" are totally unfounded. But what the spin meisters at GlamourMagz fail to remember is that science can be a very small and very well connected business at times. Laboratories that publish frequently in their pages tend to be very large, meaning a whole lot of people are right there at the primary point of interaction. People who have friends, neighbors, relatives and the like who are also in the sciences. Lots of trainees who go on to other labs. Or even jobs as Glamour Mag editors. Not all of these latter drink the KoolAde. Some of them tell tales out of school.
Most of these people don't have any particular reason to lie or even embellish all that much to their friends. OTOH, whomever is writing an official Editorial offering from one of the GlamourMagz does have a clear interest in.....well, let us just leave it at spin. So I'm going to need something a little better than your tone of self-righteous woundedness to get on your side, 'k?
__
* Because you use negotiation with ISI over what counts as citable matter and what doesn't instead!
Education
Information Science
Comments
Could those douchebags be any more full of shit?
Posted by: Comrade PhysioProf | February 22, 2010 7:50 PM
CPP @1, I think you're thinking of colostomy bags.
DM, you make excellent suggestions. Perhaps if Nature is unresponsive, some other GlamorMag might want to take them on board, to gain a competitive edge as far as love of the (science) game.
Posted by: Dr. Free-Ride | February 22, 2010 8:31 PM
It certainly is a lame attempt at refuting those 'myths' without any hard data. But honestly Nature bashing gets really tired too. Once you have to filter so heavily and publish only a few percentage of your submissions it's human nature to end up with sociological biases. I think of GlamMags as top-down attentional systems: "Look here, it's cool". Attention is a limited resource so GlamMags serve a useful purpose but they should not be construed as a stamp of quality. And here is the rub! Why do we as a community end up evaluating other scientists based on their ability to get published in GlamMags? I consider this a much more serious issue.
Posted by: PY | February 22, 2010 8:48 PM
"a low cost, low effort martyred denial"
This.
Posted by: Abel Pharmboy | February 22, 2010 8:58 PM
But honestly Nature bashing gets really tired too...Why do we as a community end up evaluating other scientists based on their ability to get published in GlamMags?
At the risk of belaboring the obvious, GlamourBashingTM serves the same role as the little boy in the crowd crying out that the Emperor is in fact nekkid as a jaybird whilst all the sophisticates nod sagely at his fine raiment.
Posted by: DrugMonkey | February 22, 2010 9:31 PM
DM, even just the first paragraph of your dissection ("sifting through submitted papers") is worth the price of admission on this one. That bit had me laughing when I read the editorial, and all my publishing experience is in a completely different field.
Posted by: Stephanie Z | February 22, 2010 9:52 PM
At the risk of belaboring the obvious, GlamourBashingTM serves the same role as the little boy in the crowd crying out that the Emperor is in fact nekkid as a jaybird whilst all the sophisticates nod sagely at his fine raiment.
So then your real problem should be the crowd of sophisticates and not the Emperor.
Posted by: PY | February 22, 2010 10:16 PM
Actually, PY, if you read the story, you'll understand that the problem was the con men who defined dissent as incompetence.
Posted by: Stephanie Z | February 22, 2010 10:33 PM
SZ, FTMFW!!!11!!!!
Posted by: DrugMonkey | February 22, 2010 11:24 PM
There's something else that bothers me about this bit:
There are, by the last rough estimate I saw, just over a thousand or so people actively working in my subfield of physics research. Then there are several subfields of physics that have more people than that (and several more that don't, but the numbers add up), and then the chemists and biologists and... (I kind of get the impression there are more biologists floating around than physicists, too.) Even taking into account the fact that most of those researchers are grad students and postdocs, it seems like once you integrate over all the different fields Nature is catering to, 5400 is not such a huge number of reviewers. I might be interested to see how their ratio of reviewers to readers compares to other mags.
Posted by: adagger | February 23, 2010 7:51 AM
Honestly, I'm very embarrassed for Nature having printed this. It's painfully un-self aware, and has the tone of, like, a high school newspaper or something trying to defend the homecoming court elections.
Look, seriously you guys, there are some "rumors" floating around that Billy won because his dad paid for the new football uniforms, but it was fair and square, I swear to GOD!!
Posted by: Dr Becca | February 23, 2010 9:49 AM
I think its worse at the glamour mags offsprings (Nature Immunology, Molecular Cell) where you have 35 yo failed post-docs with limited broad knowledge deciding what gets reviewed. At least at CSN, you've got professional editors who have been doing it for a while.
Posted by: physician scientist | February 23, 2010 11:31 AM
no physician scientist, you don't. There are plenty of relatively noobish editors even at the flagship CNS journals.
whether they are "failed" postdocs or not is up to you, of course, and I have been known to FWDAOTI a bit myself. but do recognize that perpetuating this idea that if you don't get a TT faculty job you are a "failure" is 1) unfair and 2) unrealistic.
before you all postdocs start foaming at the mouth again, I'll note you might want to wander over to the Alternative Scientist blog...
http://alternative-scientist.blogspot.com/
Posted by: DrugMonkey | February 23, 2010 11:50 AM
True enough drug monkey. "Failed" is not a good term and should not be used.
Its just a bit frustrating when you pubmed some of these editors and see their contributions to the scientific literature. The two journals I mention above are particularly disheartening.
Posted by: physician scientist | February 23, 2010 11:58 AM
Can someone let me know why using the 'highly citable' metric as a measure for accepting a paper is bad?
Posted by: Venkat | February 23, 2010 1:22 PM
They are considered "failed postdocs" to the extent that they didn't publish jack shit as postdocs, not because they didn't get TT positions.
Posted by: Comrade PhysioProf | February 23, 2010 1:46 PM
All's I know is, when a Glamour Lab of my acquaintance got a manuscript back from a Glamour Mag with three "meh" reviews, the editorial interjection was, "Oh, please send us a revised manuscript because *I* really liked it so maybe we can get it in somehow anyway."
Oddly, this never seems to happen to non-Glamour Lab people.
Posted by: Anonymous | February 23, 2010 1:53 PM
DM, this particular crusade of yours is misdirected, and your rant is silly. Where did you get the idea that Nature (or any other journal) is in the business of policing/regulating/guiding/etc science? Or that it should be?
NATURE PUBLISHING GROUP IS A MAGAZINE PUBLISHER! Their job is to publish things they think their readers will want to look at. Just like the guys who publish Time, Sports Illustrated, Dog fancy, and Penthouse. And they do a pretty good job. That's why Nature is probably the most widely read scientific magazine in the world.
If they think publishing papers from big-name labs will attract readers, then so be it.
If they think publishing crappy scientific papers on hot topics will attract readers, bully for them.
If they think their editors have a better idea than reviewers what attracts reader attention, then awesome.
...because they're probably right!
It's their fucking magazine. They clearly know what they are doing. Quit your arrogant whining.
You know how Comrade PhysioProf so often suggests that outspoken readers who think they can do better Get Their Own Fucking Blog? You might consider starting your own journal. It's pretty easy these days. (c.f. http://www.biomedcentral.com/info/authors/startajournal)
(Good luck with that, by the way. Sounds like it'll be boring as hell to read. I guess you'll have to make the authors pay to publish, because subscription sales might be slim.)
Posted by: DM is talking out of his ass on this one | February 23, 2010 1:55 PM
DMITOOHAOTO:
If you're arguing that Nature is making business decisions rather than scientific merit-based ones, fine. This is probably the case. However, this is exactly what the writer of the editorial argues NOT to be the case, so DM's point is that the editorial...doth protest too much. That said, as long as Glamour Mag publications remain the standard by which we as scientists are judged, Nature does, in fact, owe it to the community to operate ethically. It would be best, of course, if we could let go of the Glamour Mag as the primary indicator of our worth, but I don't see that happening any time soon.
Posted by: Dr Becca | February 23, 2010 2:22 PM
DoucheMonkey's point in this post isn't that what Nature does sucks--although I believe he has made that point before--but rather that what they say in their editorial is a big huge pile of bullshit.
Posted by: Comrade PhysioProf | February 23, 2010 3:53 PM
the editorial interjection was, "Oh, please send us a revised manuscript because *I* really liked it so maybe we can get it in somehow anyway."
This kind of thing is the BEST potentially possible use of the power of the GlamourMag, when done judiciously by an experienced Senior Editor. Ben Lewin (RIP) of Cell was famous/notorious for this kind of thing.
My favorite story comes from Judah Folkman (also sadly RIP). He related how he was unable to publish (or get funded?) his idea that controlling angiogenesis might be a key cancer therapeutic intervention opportunity. He relates how he was giving a talk at a meeting and the editor of the New England Journal of Medicine heard him and told him to write it up and he would publish it, and screw what any reviewer might have to say about it. The resulting paper has since been cited over 3000 times, for those of you keeping score at home, and is one of the few examples of truly paradigm-changing work.
Folkman, J. Tumor angiogenesis: therapeutic implications. N. Engl. J. Med. 285, 1182–1186 (1971).
Posted by: whimple | February 23, 2010 3:54 PM
I never realized that Nature was a business trying to make money. In that case, I love DMITOOHAOTO's idea. Nature Penthouse would take peer review to a new level and I'm sure it would outsell any of their other specialty journals.
Posted by: bsci | February 23, 2010 4:06 PM
while it is true that I think GlamourMagSkienze is all that is Evul and Wrong with science today, significantly *interferes* with the conduct of science and for damn sure deprives the taxpayer of the full output of their investment, that is a critique for another day. What REALLY chaps my ass is when they strike this holier-than-thou tone and pretend they are not doing what it is so palpably obvious that they are.
Posted by: DrugMonkey | February 23, 2010 4:20 PM
CPP put into words exactly my point. The scientific production of these editors prior to their involvement in the scientific literature does not give one confidence that they can actually recognize quality science.
Posted by: physician scientist | February 23, 2010 4:31 PM
The problem here is that DM doesn't understand what Nature means by "best science", which is what Nature says it is interested in publishing. DM asserts that Nature is interested only in citation rates. DM has not read or thought about the Nature editorial very carefully.
Of course, high citation rates will tend to accompany great science papers, but these two things should not be confused. DM seems to have them confused, and projects his confusion onto the Nature editors. The Nature editorial tries to disentangle the sort of confusion DM has by, for example, comparing an award-winning but lesser-cited chemistry paper to another more highly-cited one to show that importance does not always equal high citation rates. Which is something I think DM agrees with.
So why can't DM wrap his head around the idea that Nature might actually do just what they say -- try to publish the most important and interesting papers from a variety of scientific disciplines... period?
Does DM also complain that literature anthologies tend to contain an inordinate number of works from famous and highly-quoted authors? Does DM lament the fact that the most famous artists also tend to be among the most talented? Perhaps Michelangelo got the Sistine Chapel job only because the pope could see into the future and know that Michelangelo would be famous and historically important?
And THAT is the problem.
Notice it's not a problem of Nature's doing. It's a problem of OUR doing. Scientists have become not only too lazy to actually read papers, they have even become too lazy to even look at the journals papers are published in. Instead they rely on 'citation indices', and then (like DM) get all in a huff when they realize how stupid this is.
Posted by: Dm is talking out of his ass on this one | February 23, 2010 4:46 PM
DMITOOHAOTO,
Here's an experiment for someone who has more time than me right now. I suspect those low citation articles are very disproportionally from people who have already published a high citation article in Nature or are already very prominent.
If this is the case, then the low citation articles aren't the editors plucking great science from obscurity. They're setting a lower bar for already prominent people to publish in the journal. I'll admit this isn't the the case for the specific low citation article highlighted in the journal, but they might be very good at picking a good example.
Posted by: bsci | February 23, 2010 5:18 PM
DMITOOHAOTO,
hy can't DM wrap his head around the idea that Nature might actually do just what they say -- try to publish the most important and interesting papers from a variety of scientific disciplines
This would require me to lower my opinion of their intellectual capacity to an insultingly low level. No, I don't credit that they honestly, in their heart of hearts, believe this. There is just no way they do not understand the inherent circularity of their publication game.
regarding the arts- I don't have a lot of dogs in the hunt but when it comes to the business of promoting recording artists pre-Internet....damn straight GlamourMusicPub was a problem.
re: "I've seen the problem and it is us", I think you are having a problem wrapping your brain around the fraction of my audience that is "us" and that which is "glamour pub editors".
Posted by: DrugMonkey | February 23, 2010 5:52 PM
Blah-blah-ass-blah there seems woefully unaware of the concept of a tastemaker. Does s/he really think that Michelangelo is hailed as a supreme painter because we all believe women are ideally depicted with the kind of thick necks and ropy muscles you find in gay wank mags, preferably in carnival colors? No.
We don't know what great art didn't survive or was never created because of the flaws in the patronage system. We do know plenty of talent was wasted. Who wants to see that happen in science?
Posted by: Stephanie Z | February 23, 2010 8:41 PM
Asshats, the fucking problem ain't what goes out for review it's the good shit they return without reading intelligently.
Posted by: Eli Rabett | February 23, 2010 9:01 PM
I love that the evidence for how open-minded the editors are is b/c they do publish articles that got all mediocre reviews... is this perhaps how they also manage to publish papers with lots of media oomph but little scientific substance?!
I have been thinking of asking a Nature-love-child to tack his name onto one of my submissions as a test of the hypothesis of name-influence.
Given what I've seen up-close-and-personal of how this person interacts with Nature, this defense is complete BS. I don't get phone calls from editors asking why I sent paper X to journal Y instead of Nature, but my friend does...
Posted by: inBetween | February 23, 2010 9:03 PM
CPP @16
They are considered "failed postdocs" to the extent that they didn't publish jack shit as postdocs, not because they didn't get TT positions.
physician scientist @24
CPP put into words exactly my point. The scientific production of these editors prior to their involvement in the scientific literature does not give one confidence that they can actually recognize quality science.
I doubt that recognizing great science correlates too well with "scientific production". But either way, some neuro editors FYI:
http://www.ncbi.nlm.nih.gov/pubmed?term=chou_ih
http://www.ncbi.nlm.nih.gov/pubmed?term=gray_nw
Posted by: Anonymous | February 23, 2010 9:58 PM
Well shit me a frying pan sideways, cause as I see it DM is just barfing populist pie with this one. What would he have Nature do? Publish boring descriptions of shallow minutia in order to somehow enfirmen the stature of the nonglamour science DM seems so enamoured of? Maybe they could toss him a bone by having a special MDMA issue with him as guest editor? Heck, maybe Nature should somehow concoct all do-it-yourself issues. Everyone can stick in their own papers. It'll be like those places where you get to dress up and pretend you are a wild west gunfighter, with a photo to take home. Except in this case it's Nature on your CV instead of a cowboy hat. Whoohoo! We can all be talented and famous, if only Nature would let us.
How about writing something important and informative for once, DM, instead of just trolling for bloghits and a rise in the Sb active-o-rama scale? After all, I'd hate to see you sink to the level of Nature.
In the mean time, I'm going to write American Idol a stern letter. Why am I not on the show? My mom thinks I sing great.
Posted by: Dm is talking out of his ass on this one | February 23, 2010 10:50 PM
Whimple @21, it's fine if editors selectively rescue some manuscripts. What's NOT fine, and what I was asserting, is that they preferentially rescue Fancy Lab manuscripts over Lowly Lab manuscripts, because of the name cachet.
Posted by: Anonymous | February 24, 2010 12:09 AM
Anonymous #31:
Are you saying those editors production is impressive? I see junior people with on a couple of first author papers from other people's labs with no evidence that they can run their own labs and run a larger scientific operation. Give me JCI, MCB, JCB or any other journal that has SCIENTIST (running their own lab) editors any day of the week.
Posted by: Physician scientist | February 24, 2010 7:23 AM
Just for the record, although I did not participate in the decisions surrounding this editorial, one possibility for why it emerged is the following: Nature/Science/Cell were recently targeted by a BBC article suggesting there was a Free Mason society regulating what got published RE: stem cell manuscripts. The article was a little bizarre, with the story gaining traction about 7-8 months after the original accusation was made by a separate secret society of stem cell researchers. This accusation could have safely been ignored, but the editorial provides a little bit of perspective as a counter-balance to that BBC story.
The data requested in this post would be great, but unfortunately is difficult to assemble since the boundaries of classification for specific decisions would in itself by subjective. It is almost easier to just randomly select files and discuss them anecdotally! Nevertheless, there is data on the distribution of papers published by Nature as compared to other journals. An editorial in Nature Neuroscience from 2003 explored this concept of how the IF is dominated by a small fraction of papers that receive a lot of citations, while the proportion of papers receiving only a few citations is roughly similar for both the glamour and working-man's journals. Although dated, the trends remain the same today. It is behind a pay wall, so I do hope you will get an opportunity to read the entire piece and look at the graph, but here's a block of the relevant text:
We looked at the distribution of citations to individual papers in Nature Neuroscience (2002 IF = 14.857), and compared this to the distributions for neuroscience papers in Nature (overall IF = 30.432), and for samples of papers published in two larger journals, Journal of Neuroscience (IF = 8.045) and Brain Research (IF = 2.409), during the same period.
The most obvious feature of these distributions is that they are highly skewed (Fig. 1); in every case, the medians are lower than the means, reinforcing the point that a journal's IF (an arithmetic mean) is almost useless as a predictor of the likely citations to any particular paper in that journal.
Although the distributions overlap, they are all significantly different from each other by a non-parametric test. Unsurprisingly, the peaks are systematically shifted in a direction consistent with the overall IF, such that, for example, the median paper in Nature would be at the 68th percentile for Nature Neuroscience and the 99th percentile for Brain Research.
What is most distinctive about the higher-impact journals is their long tails, corresponding to a relatively small number of papers that are exceptionally highly cited, and which therefore contribute disproportionately to the IF and, presumably, to the overall prestige of those journals. At the other end of the distribution, the lower-impact journals tend to publish more papers with few citations.
[SNIP]
Properly interpreted, citation data can be a valuable tool for evaluating journals, papers, authors and perhaps even editors. But it is a blunt instrument at best, and when complex distributions are reduced to simple averages, then much of the usefulness is lost. Journal impact factors cannot be used to quantify the importance of individual papers or the credit due to their authors, and one of the minor mysteries of our time is why so many scientifically sophisticated people give so much credence to a procedure that is so obviously flawed.
So while we deserve many criticisms and since the peer-review and paper-selection process, being of a human endeavor, is inherently flawed, I suggest discussing how to wean the study sections/committees making grant funding decisions and faculty hiring/TT decisions off of counting publications and instead explore the best strategies to have these groups take a hard look at the scientific contribution of the grant/scientist. Academic research science is the only big time entity I can think of that currently outsources its most important decisions (who to hire and to whom to give money) to a bunch of "failed" members within it's community. Interestingly, biotech/pharma companies, who have boards and stockholders to whom they answer, do not practice this same "outsourcing" culture...
Posted by: Noah Gray | February 24, 2010 7:54 AM
Physician scientist:
In my field, faculty jobs are usually given to junior people (postdocs) with a couple of first author papers from other people's labs that have had no experience running their own lab. I mistakenly thought that was typical, but it's encouraging to hear that senior scientists with long C/V's and who come from their own labs are still getting the faculty job offers these days to start their own lab (again?), instead of these young wet-behind-the-ears whippersnappers with a hunger for doing science!
Posted by: Noah Gray | February 24, 2010 8:01 AM
Why is this NOT fine? It's their magazine!
And given a choice, I would rather know what FancyLab is up to rather than no name lab. It's a SCIENCE magazine. Part of it's purpose is to tell us what is going on in the world of professional science.
Should 'People' magazine stop following celebrities and instead publish photos of average people on the street?
Should NewsWeek reduce their coverage of Obama so they can do equal coverage of the congressional reps from North Dakota?
Posted by: DM is talking out of his ass on this one | February 24, 2010 8:25 AM
Noah-
That's the point. Incredibly inexperienced scientists (in the early 30's) are being given powerful positions that vastly shape the scientific literature without having the scientific experience to really know diverse fields. This is less true at CSN as those editors typically have to work their way up. The editors at journals like Molecular Cell, Nature Immunology and the like have been given positions that are above their experience level.
My point is exactly the one you make. Approximately 50% of Asst professors given startups wash out. We don't automatically make them chairpeople of their department coming out of a post-doc. They must prove themselves first. The same should be true at the subsidiary journals of the glamour mags.
Posted by: Physician scientist | February 24, 2010 8:30 AM
Uh, seriously folks... It's the top Science journal in the world for a reason -- they know what the fuck they are doing! Maybe you all could shut up and learn from what they are saying, rather than pretend they are liars and you know better.
The armchair quarterbacking here is just as annoying and stupid as the armchair quarterbacking that went on in the Amy Bishop thread.
'...if I were there, I'd have jumped across the table and karate-punched her in the throat while she re-aimed...'
'...if I were in charge of Nature, I'd publish all nontechnical confirmatory observational papers from no-name labs, because they never get retracted...'
Posted by: DM is talking out of his ass on this one | February 24, 2010 8:35 AM
Noah-
I should also point out that I don't think these people aren't smart or that I don't think they won't be really good in the future. I just question the experience level.
Posted by: physician scientist | February 24, 2010 8:36 AM
Physician scientist:
Thanks for clarifying. I mistakenly thought you were referring to faculty positions. Anyway, I understand now.
It's a fair criticism.
Posted by: Noah Gray | February 24, 2010 8:41 AM
Incredibly inexperienced scientists (in the early 30's) are being given powerful positions that vastly shape the scientific literature without having the scientific experience to really know diverse fields.
Kind of a nutty critique. Are you suggesting that the "incredibly inexperienced scientists" are given these positions instead of all of the vastly more experienced scientists that are applying for these positions?
Posted by: whimple | February 24, 2010 9:15 AM
Whimple-
Are you for real? The following are the top editors at some very good journals:
Ira Mellman - JCB
Michael Yaffe - Science Signaling
Jeremy Boss - J. Immunology
Roger Davis - Molecular and Cellular Biology
Lawrence Turka - J. Clinical Investigation
Jonathon Eisen - PLOS Biology
Clearly there are senior scientists taking on these jobs.
Posted by: physician scientist | February 24, 2010 9:30 AM
So, you're saying Nature hires inexperienced junior people instead of senior people? Why? Pay them less? Enjoy their fresh, naive perspectives? Lose a bet to JCB & MCB?
Posted by: whimple | February 24, 2010 10:21 AM
Whimple - You asked for senior people running high-tier journals. I gave you a list. Interpret the data how you want.
Posted by: physician scientist | February 24, 2010 10:36 AM
That's not what I asked for. I asked why you thought Nature is hiring these "inexperienced junior people" into "powerful positions that vastly shape the scientific literature"?
Posted by: whimple | February 24, 2010 10:39 AM
Whimple-
I don't know why, but it seems clear that its happening.
Posted by: physician scientist | February 24, 2010 10:42 AM
whimple- yes, they pay their editors poorly, give them enormous loads of papers to read, and make them travel almost constantly. it's a shit job, and senior scientists don't want it.
DM- I'm totally with you on this post. but I also agree with the people who say the problem is the still the reliance on glamour pubs for evaluating young scientists (like myself). There are two parts to this crime: the emperor, and the obedience.
Posted by: msphd | February 24, 2010 6:14 PM
I think the reason is clear and obvious. Because Nature can. It is cheaper to hire inexperienced junior people, and it doesn't reduce their revenues very much. Good marketing will always be cheaper than good science.
It only affects the scientific literature to the extent that scientists allow it to affect the scientific literature.
GlamourMagz fit right in with the fad driven nature of scientific research right now. Everything is genetics and -omics. Everything is glamor driven because the gate keepers for funding can't understand what it is they are being asked to fund, so they go to second and third order metrics.
I am less worried about the scientific literature. There is already a lot of junk in there, more junk isn't that much of a problem. The bigger problem (and it is a much bigger problem) is that good stuff isn't making it into the literature because people doing good stuff can't get funding because the funding agencies are enamored with the sizzle of the latest fad, puffed up by the glamor of the GlamourMagz. The good stuff never gets funded, so it never gets done. That is the real loss. Eventually it will get done, but eventually is a long time, and a lot of dead ends geet investigated in the mean time.
Posted by: daedalus2u | February 24, 2010 6:23 PM
daedalus2u has a great point. If it takes a Nature paper to get a t-t job and another Nature paper to keep that job, then Nature editors and reviewers set a lot of scientific direction.
Posted by: whimple | February 24, 2010 6:30 PM
When did DM suggest that nature should publish less interesting work? Sounds like a straw man to me.
Posted by: pinus | February 24, 2010 7:08 PM
"The bigger problem (and it is a much bigger problem) is that good stuff isn't making it into the literature because people doing good stuff can't get funding because the funding agencies are enamored with the sizzle of the latest fad, puffed up by the glamor of the GlamourMagz."
I've been hearing a lot lately about how this has led to the demise of the taxonomic branch of biology, which is a great loss, and worrisome considering the loss of diversity we are facing. And we still only know 10 -20% of what's out there, and for most of those species we know very little about them.
Posted by: Isabel | February 24, 2010 8:59 PM
Nature is so full of it. Editorial staffs should not be swayed by such things as "citation impact", "page views impact", "visitor impact", etc. I work for a life science magazine and we have a policy of seperating sales, web, and editorial. It keeps the product at a high quality level. Besides if we followed pageviews/visitor trends - every article would be creationist versus evolutionist. lol
Posted by: NJ Wedding Photographer | February 25, 2010 2:35 AM