Can Social Media Replace Pre-Publication Peer-Review?

Richard Smith (former editor of BMJ) commented on the case control study of the XMRV (xenotropic murine leukemia virus related virus) as a cause of Chronic Fatigue Syndrome.  The study was published in Science and he comments that there were several problems and people called for a better peer-review process to avoid these problems in the future.

[Added 12/24/2010: There have been several comments for this post, highlighting some of the controversies regarding this topic.  "CFS" has had a lot of recent research and studies with very conflicting results.  The comment by Richard Smith mentioned above was made in March 2010, and a lot of research has been reported since then.  Hopefully we will soon find out the truth and hopefully get closer to providing a cure for our patients.  This post is NOT about appraising the evidence regarding the "CFS" literature and thus this is NOT a commentary on the Science study mentioned above.  It is about the problems with peer review process in general as identified by a former editor of a major journal,  and a tentative exploration for an alternative model, and barriers to such a model. The statements in the paragraph above referring to CFS and XMRV are there just to provide context.  For the purpose of this post, it could well have been another condition and a different study]

Richard Smith points out the problems with the current peer-review process:

• Faith based not evidence based
• Slow
• Expensive
• Largely a lottery
• Poor at detecting errors and fraud
• Stifles innovation
• Biased

He suggests that we move away from our bias for top journals and move away from the traditional peer-review process and use a "publish and then filter" process.  

This got me thinking about how this could work.

1. A central resource for online hosting of all research articles in each area of biomedical science.  We would not have multiple journals competing and catering to the same audience
2. There would be some kind of simple review process to filter out "junk" and "spam" publications
3. The articles would need to include all the necessary raw data so anyone could rerun the statistical tests and verify the results.
4. There would be a robust authentication scheme for authors.
5. Each article would have a place for commenting much like a blog, but you would need to have to be authenticated before submitting your comments.  There would be no anonymous comments.
6. Readers after logging in could rate each article on various criteria e.g. study design, practical value, etc...  
7. The comments could also be rated up or down
8. It would be possible to track how many times the article was cited, tweeted and posted on Facebook; how many times it was downloaded, favorited,  etc.
9. Other studies on the same topic would also be linked from the article making it easy to find all the studies in one place.
10. Part of the publication process would be to search for all the previously published related articles in this central repository and provide links to all of these.
11. Viewers could see a timeline of development of literature on a specific topic 
12. Over a period of time, some studies, authors, commentators would rise to the top.  
13. There would be a robust search and tagging system.
14. Some articles could be accompanied by "editorials".
15. Every time the IRB at an institution approved a protocol, it would create an entry in this central repository.  Investigators would have to provide their data and a short summary at end of the study even if they did not write it up fully.  This would remove the problem of publication bias for positive studies and make meta-analyses more complete.  If they did not provide this information, their ratings would go down.  

Most of this functionality already exists - just look at YouTube, Ebay, Amazon etc.  It would not take a lot to get this working.  The problem is breaking down the traditions and existing norms.  How can you replace the thrill and ego-boost that authors get from having their article accepted in a "top-tier" journal.  Would the really big multi-center randomized double blinded trials with positive results get submitted to this central resource instead of to a top tier journal?  Would universities change their criteria for promotion and tenure?

We need to break down some of the walled gardens of some of our "top" journals and level the playing field but it will be an uphill battle.

[Added 12/24/2010 - Looking at some of the comments for this post, there is clearly a lot of energy surrounding the research on "CFS".  Would it not be easier for folks looking to study this condition if all the studies reporting on "CFS" and possible connection to XMRV were published in the same repository, so they would not have to go to multiple journals and databases to find this information, all the raw data was available, the pros and cons of each study were transparently viewable and authenticated users could post comments in unmoderated fashion (like to this blog post) to add to the richness of the discussion?  Why do we need to have so many barriers to collaboratively finding solutions to such vexing problems?]

Barriers to use of Social Media in Medical Education

I just came across 2007 this study - an online survey- of UK medical students, house staff and physicians regarding their opinion regarding use of social media in medical education.  Invitations were sent by e-mail to 6000 people and 21% responded.  The barriers identified by the responders were:

1. Would like to use social media for my education but don't know how
2. Don't like to use technology for my education
3. Lack of awareness of quality resources - don't trust the content
4. Lack of access at workplace and lack of time.

These were 4 themes that emerged from 60 free text comments in the survey responses.  That's a pretty small percentage of people from those who responded who bothered to enter a free text comment.  Combine that with the 21% response rate and it suggests that we have to interpret the results with a degree of caution.  Unfortunately for studies like this (email invitations to large number of people) these types of numbers are not unusual. The authors need to be commended for the effort they made to get this data.  

The authors commented that respondents appeared to be generally interest in Web 2.0 tools including social media. 

Do these 4 themes cover all the reasons why doctors don't use social media?  In my experience, I think there are even now (in 2010) a lot of physicians who think that using social media in healthcare is a waste of time (different than not having time to use it) and that apps like FB and Twitter are used by a bunch of narcissistic mutual back slappers.  There are numerous ideas and examples of how social media are, or can be, used in medical education (examples).  The problem is that these discussions about uses of social media occur in the blogosphere or on Twitter or FB.  The Average Joe physician is unlikely to get exposed to these, unless it reaches the mainstream print journals.

Measuring true outcomes (Kirkpatrick Level 4) from educational interventions is difficult  but it will probably take a study that shows an impact on outcome measures that is published in a reputed journal to change some people's minds.   To make matters worse, the print journals are constantly publishing case studies and guidelines regarding professionalism (or the lack thereof) and social media and this scares away some of the physicians who might be interested in this medium.  Physicians have already been burned by adopting clinical practices without sufficient good evidence and then having to go back when these were proven wrong (e.g. hormone replacement therapy).  So one can understand why some will "look" before leaping!  Even with educational practices we have seen the rise and fall of Learning Management Systems.  How can medical educators and physicians be sure that social media is not just another fad?  

It reminds me of a talk I gave in 1995 titled "The Internet and Medicine: Why Physicians should Pay Attention" and later wrote up for a medical journal.  There is a great The New Yorker cover by Edward Sorel which shows Whistler's Mother looking skeptically at a telephone.  I had referred to this cover when I gave that talk in 1995.  Maybe it is time to bring out that old issue of The New Yorker!

A Practical Medical Informatics Curriculum for Medical Students

How can best prepare our medical students to practice medicine in world of technology?  A number of efforts have been made to create curricula for medical schools and also for special fellowships in medical informatics.  This is a very rapidly moving target and the products of these efforts need constant updating. 
The American Medical Informatics Association website states that "biomedical and health informatics applies principles of computer and information science to the advancement of life sciences research, health professions education, public health, and patient care"


If we had to develop a list of concepts that we need our graduating medical students to be familiar with; what would it include today?  Maybe, if our students were all self motivated, reflective, life long learners, all we would have to do it ensure that they know how to find answers to their problems and questions, share this list of concepts with them and we would be set; we would not even have to develop a course for this curriculum!

This list of Core topics for all medical students would look something like this: (clearly a work in progress):

1. Computers (desktops, laptops), mobile devices, smartphones, portable data storage devices
2. Networking and connectivity – Intranet, Internet, WWW, VPN, Wireless, Bluetooth, 4G etc
3. Software – Office (word processing, presentation, spreadsheet etc), Statistical software, Bibliography tools, Cloud computing, Browsers and add-ons, mind mapping etc.
4. EBM concepts – Information resources, information retrieval, Appraisal of literature, Application to clinical practice, guidelines etc, also related to this are the concepts of public health informatics
5. Communications – e-mail, list serv, text messaging, discussion groups, social media, professional and patient communication, presentations (live and online),  Web 2.0, blogs and Wikis, social networking
6. Data – data collection, organization, storage and representation, data standards (including HL7), data interchange,
7. Personal knowledge management, project management, organization tools
8. EHRs/EMRs, Personal (patient) health records, meaningful use, physician report cards, Quality metrics etc. ePrescribing, RxHub, etc.  (Some topics like PACS might be more relevant to those going to radiology)
9. Decision science (decision analysis, probability, test characteristics, likelihood ratios etc) , online and point of care decision support tools,  Clinical decision support (Alerts, reminders),
10. Ethics, professionalism, legal and regulatory issues, privacy, security, authenticity and encryption
11. Ontology, terminology (Thanks to comment from Anonymous)

There are areas that might be optional for some students and be offered as electives.

1. Education informatics – Learning management systems, virtual learning environments, personal and social learning environments 
2. Bioinformatics (genomics etc)
3. Research informatics 

What do you think?  What are some other concepts that every graduating medical student should have that are not included here or could be better defined/organized?  


For reference, here is a recent JAMIA article with a detailed curriculum for clinical informatics specialists.
J Am Med Inform Assoc. 2009;16:153–157. DOI 10.1197/jamia.M3045.
And
Here is the AAMC Medical School Objectives Project (MSOP) Phase II report that covers medical informatics and Public Health.  https://services.aamc.org/publications/showfile.cfm?file=version88.pdf&prd_id=199&prv_id=240&pdf_id=88

Google Reader to Facebook Journal Club Part II

The previous post on a model for using Google Reader and Facebook to create a Journal Club got a lot of responses.  I also added some refinements to the Model.  Here is a summary:

1. Concern about residents/students having to friend the faculty members of the group:  Group members do not have to be friends. So this should not be a concern. 
2. Using Groups vs Pages.  The biggest problem with FB Pages is that they cannot be closed or secret.  Students and residents would have concerns about their comments being visible to future employers and thus Pages are out.  
3. Secret vs. Closed Group.  If you make the group secret, them folks cannot even find it and so the group admin has to invite each member to the group.  This requires, I think, for the invitee to be a friend of the admin.With a closed group, the group name is still visible to non-members who can ask to be invited.  The advantage of a secret group is that comments made by members and not visible to their non-member friends. Group creators should not change their group from secret to closed without approval of the members.
4. Keeping the group active:  Members may not remember to visit the group even though they are logged into their Facebook account.  Posts on the Group wall will not show up on the members news feed.  Members will see posts by group members who are friends.  Thus members with friends who are active posters in the group will keep getting reminded to visit the group.  Admins can send e-mail notifications to group members to alert them about some critical posts.  Judicious use of this might help improve group activity.  Members can elect to get notified of posts.  None of these are ideal.  Wish FB would make for automatic posting of group messages on member walls an option.
5. Too many posts could be intimidating.  Would be a good idea to classify/organize the posts by specialty for example.  One way to do this is to have an RSS stream for each specialty's journals.  (or create tags for each specialty in Google reader and create a public RSS feed for each tag). Them import that feed into the Group using RSS Graffiti but under "Post as" select a different person.  Thus one could create a fictitious Pulmonary Doc FB account, make that person one of the admins for the group, and post all pulmonary journal articles under that person's name.  Use an image of the lungs for that Pulmonary Doc's profile pic.  Thus all posts of journal articles related to the lungs would be tagged by an image of the lungs.
   

Example of creating a cardiology feed with an image of the heart for quick identification

 So that's it for a quick update.