After MOOCs, the next challenge to Academia

Increasing Challenges to Traditional Academia 

The huge surge in interest in MOOCs underlines more the problem with the current status of academia and less the qualities of MOOCs as a replacement model. Part of the attraction of MOOCs is that they present a potential solution to address the eternal triangle of education - Access, Quality and Cost.  

As if the ridiculous cost of higher education combined with the absence of matching quality is not a problem enough, the traditional publish-or-perish tenure process is being called into question. Peter Higgs, the man behind the Higgs boson, and recipient of the Nobel Prize was quoted as saying 

"(He) doubts work like Higgs boson identification achievable now as academics are expected to 'keep churning out papers"

 Doing high quality research and disseminating the results should not be controversial.  The problem is that the publication process has been tied with tenure.  A high-stake summative examination promotes teaching to the test, and creates an industry around the examination and preparation process, while causing distractions for the actual process of education.  The publish-or-perish tenure process is similarly flawed and has been compounded by the issues with the peer-review itself.

Problems with the peer-review process

Problems with the peer-review process have been well-recognized.  While it is a valuable process, it is not perfect.  In a previous blog (from 2010) we looked at some of these problems as highlighted by Richard Smith (former editor of BMJ).

• Faith based not evidence based
• Slow
• Expensive
• Largely a lottery
• Poor at detecting errors and fraud
• Stifles innovation
• Biased

Can Web 2.0 play a role?

In that same blog, I outlined a model for using technology and Web 2.0 tools to create a post-publication review process.  Here is an outline of the model:

1. A central resource for online hosting of all research articles in each area of biomedical science.  We would not have multiple journals competing and catering to the same audience
2. There would be some kind of simple review process to filter out "junk" and "spam" publications
3. The articles would need to include all the necessary raw data so anyone could rerun the statistical tests and verify the results.
4. There would be a robust authentication scheme for authors.
5. Each article would have a place for commenting much like a blog, but you would need to have to be authenticated before submitting your comments.  There would be no anonymous comments.
6. Readers after logging in could rate each article on various criteria e.g. study design, practical value, etc...  
7. The comments could also be rated up or down
8. It would be possible to track how many times the article was cited, tweeted and posted on Facebook; how many times it was downloaded, favorited,  etc.
9. Other studies on the same topic would also be linked from the article making it easy to find all the studies in one place.
10. Part of the publication process would be to search for all the previously published related articles in this central repository and provide links to all of these.
11. Viewers could see a timeline of development of literature on a specific topic 
12. Over a period of time, some studies, authors, commentators would rise to the top.  
13. There would be a robust search and tagging system.
14. Some articles could be accompanied by "editorials".
15. Every time the IRB at an institution approved a protocol, it would create an entry in this central repository.  Investigators would have to provide their data and a short summary at end of the study even if they did not write it up fully.  This would remove the problem of publication bias for positive studies and make meta-analyses more complete.  If they did not provide this information, their ratings would go down. 

Progress

This was a proposal made almost 3 years back.  Some of the pieces are beginning to fall into place.

Authentication in the online space

Google is stepping up by having people create a profile on Google+ with their real names.  This is getting tied with comments in other Web 2.0 spaces like YouTube and Blogger.  When you search on Google, you can see results that were "1+'ed" by people you know on Google+.  You can see "likes" by Facebook friends on public sites.  These are the first steps to creating authentic personae in the online space.  

When scholars "1+'s" an article or comment on a website, their online persona is tied to that article and will increasingly carry the same weight as commenting on a print article e.g. a letter to the editor.

Public comments on PubMed

Just recently, PubMed Commons was opened to invited researchers to comment on any article indexed on PubMed.

PubMed Commons is a system that enables researchers to share their opinions about scientific publications. Researchers can comment on any publication indexed by PubMed, and read the comments of others. PubMed Commons is a forum for open and constructive criticism and discussion of scientific issues. It will thrive with high quality interchange from the scientific community. PubMed Commons is currently in a closed pilot testing phase, which means that only invited participants can add and view comments in PubMed.

Online communities using reference and citation managers

Sites like Mendeley, CiteULike and Zotero allow scholars to collect online publications in their libraries and share these within groups/communities.  This activity provides data on the frequency with with each publication is included in a library and the number of people doing this.

Tracking impact of an article in the digital world

Altmetric is a terrific tool that tracks the impact of an article in social media and on sites like Mendeley, Zotero and CiteULike. There is a very cool bookmarklet and a Chrome extension that lets one see the Altmetrics of an article.  For example a recent article that I co-authored was published in Academic Medicine in October 2013 just over a month back.  It has been cited twice in literature according to Google Scholar.  It will take a long time to see how many times it will actually be cited in peer-reviewed journal articles.  But using the Chrome extension, I could easily see the impact factor it has in social media.  

Clicking on the link brings up detailed statistics:

It shows the overall impact of the article compared to all articles in Academic Medicine and also provides the impact for all articles of similar age, i.e. published 6 weeks on either side of it.

Altmetric has made is API public and this is being used to create innovative tools.  Thus PaperShip recently created an app for iOS that connects to Zotero or Mendeley and also displays Altmetrics data for each article.

The foundations of traditional academia are being challenged on multiple fronts and are beginning to show their age!  How long before the foundation is rebuilt or the house comes tumbling down?

Reading Journals in the Era of Tablets

I recently wrote a blog post on why readers don't use their tablets to read journal articles.  There were several possible reasons.

1. Wanting to work with PDFs rather than HTML
2. Difficulty annotating the PDFs on tablets (not knowing best tools for this)
3. Hyperlinks may not work in PDFs
4. Wanting to flip between a citation# in the text of the article and the reference in the bibliography.  This is easier to do on paper.

The workflow that might meet most of these needs (except #4 scroll to bottom for a work around for this):

Model workflow for annotating Journal PDFs on Tablets

1. On you Desktop/Laptop
1. Get table of contents and abstracts of Journals sent to my Google Reader via RSS feeds
2. When you want, you get the full text of the article and save to Dropbox
3. Journals are now increasing providing PDF with OCR which allows for highlighting.  If the PDF cannot be interpreted as text by the PDF reader, then the words cannot be highlighted, or the hyperlinks will not work.
2. On your tablet
1. Use EZPDF which is a full featured PDF reader and annotator.  It has some awesome features like:
1. Highlighting
2. Adding notes
3. Drawing free text
4. Adding arrows, circles or rectangles
5. Night more (background becomes dark and font is white)
6. Text to speech - actually reads the article for you.
2. Go to Dropbox on you tablet and open the PDF with EZPDF
3. Start reading and annotating (Long tap).  
4. Choose not to make a copy of the article for annotation. Annotate the original in Dropbox. This will leave the annotated article in Dropbox as opposed to another folder on your tablet

This is a video of how to use EZPDF on an Android tablet.  There are more details on a blog here by Sleepless Ninja

The annotated PDFs are stored in Dropbox which has pros and cons.  This is good because the library automatically syncs with all your computers.  The negative is the inability to use this library to cite when writing an article or when creating bibliography.

The alternative work flow that supports citations and bibliography is to use something like Zotero.  Since Zotero introduced the standalone product that works with Chrome and Safari in addition to FireFox it has become even more attractive.  There is a tool called Zotfile that allows connecting your Zotero library to Dropbox and syncing with the tablet.

I have not explored Zotfile yet but I love Zotero so look out for an update in a future post.

Add:
If the journal does not have hyperlinked references in its PDF format, go to the HTML format and check if these are available.  If they are, then use Adobe Acrobat to create a PDF [File>>create PDF>>from web page>> enter URL
This will maintain all the hyperlinks that are in the HTML version.

Practical model for using the Information Processing Theory to stay current with (medical) literature

The Information Processing Theory compares the human brain to a computer and suggests that just like we can make a computer work better by changing the processor, motherboard or RAM and the programming, the human can also become more sophisticated thinkers with appropriate strategies, sensory inputs and rules.

While I don't know how well validated this theory is, it does describe an interesting flow of information from the sensory register to short term memory to long term memory.
The wikipedia describes it thus

Sensory register: the mental processing unit that receives information from the environment and stores it fleetingly.
Short-term memory: the mental processing unit in which information may be stored temporarily; the work space of the mind, where a decision must be made to discard information or to transfer it to permanent storage, in long-term memory.
Long-term memory: the encyclopedic mental processing unit in which information may be stored permanently and from which it may be later retrieved.

The way I understand it, Constructivism works great for understanding how we can facilitate learning of new concepts e.g. medical students learning anatomy in their first year. Active learning, Problem based small group discussions, etc.

When we think about how to help a practicing physician keep up with medical updates in his or her specialty, it is a different setting. The person already knows the subject, is busy and probably does not have much time for collaborative learning in small groups. The constructs have already been formed during medical school, residency and subsequent experiences. It seems self directed learning using the principles of the Information Processing Theory would very helpful to help the physician refine these constructs on a longitudinal basis. Let me show you a model of how this would work.

Software needed:

1. Google Reader This is a free web application and all it requires is a free Google account.
2. Zotero I have referred to Zotero in a previous post. It is a free bibliography and citation tool that works as an add on to Firefox.

Screen shots of the Model:

Overview of Google Reader



Google Reader Detail:



Zotero Overview:


Zotero Details:


Retrieval from Zotero:

Steps in the Model:

1. Set up RSS subscriptions to your specialty medical journals in Google Reader
2. This allows abstracts of articles flow into your Reader automatically and allows you to view these in one place
3. This is equivalent to the sensory register mentioned above. When you browse through the RSS feeds, you "receive the information from the environment and store it fleetingly"
4. As you scan through the abstracts, you can in Google Reader mark the ones that seem significant and relevant with a star. The Google Reader automatically tracks the ones you have browsed and removes them from the "home screen".
5. You have thus gone through the process of determining whether the abstracts are to be discarded or to be processed for storage in long term memory. This is similar to the short term memory proposed in the Information Processing Theory.
6. When you have more time and inclination, you return to the Google Reader using Firefox that has the Zotero plug-in installed.
7. You click on the hyperlink to the abstract and from there to the full text article if you have access. You read the abstract and determine that it is worth "storing". Zotero allows you to take a "Snapshot" of the article and annotate it with notes, keywords and highlights. The bibiliography information of the article along with the annotated snapshot and notes and keywords are all stored in your Zotero library
8. Zotero allows each reference item to be included in several collections. Thus as an Internist, I have collections for all subspecialties of medicine - e.g. Cardiology, Pulmonary etc.
9. This process of analysing the article, annotating it and adding keywords, notes and sorting into collections is similiar to encoding that is described in Constructivism and helps move the information into a more permanent long-term storage.
10. The entire Zotero library is searchable and allows for easy retreival of previously processed information. As information is recalled and applied more often, it is more likely to become true knowledge.

Try it out, you have nothing to lose and possibly a lot to gain!

Using Zotero in an ePortfolio

Zotero has become one of the most popular Reference management and bibliography tools. Created by the Center for History and New Media at George Mason University with funding from the US Institute of Museum and Library Services, Andrew W. Mellon Foundation and Alfred P. Sloan Foundation. It is an extension (add-on) for Firefox and is free.
Zotero is among the many excellent applications developed by the Center. The appear to have some very creative minds and an excellent team of designers, user interface experts and programmers along with some strong funding.

Zotero is in some ways like other reference managers:

1. Allows one to build up a library of references along with specific information like Author names, dates, journal name, pages etc...
2. Has a Word (and Open Office) plug in to insert citations into articles you write and to create a bibliography of the citations you use at the end of the article.
3. Has multiple styles for citations and bibliographies that you can use based on which journal you are sending your manuscript to.

Zotero is different from out reference managers too:

1. It is free
2. It is an extension of a browser - this feature means Zotero understands that most databases of information are now online and people use a browser to search these databases. Even if you do want to add an article from a print journal into your library, you could still go to the appropriate database where this journal is indexed and find that article.
3. Adding the article's appropriate information to your library is often seamless. A large number of these online databases (e.g. PubMed) are now Zotero ready. What this means is that when the browser opens a web page with information re' say a journal article, Zotero "senses" this and a small icon appears to the right of the URL box. Clicking on this icon adds all the metadata about that article into the appropriate fields in the Zotero Library.
4. Zotero allows you to tag the article with keywords and also to add it to various "Collections". It also has a powerful search tool to search for through your references.
5. Zotero allows you to get a "snapshot" of the webpage and annotate your copy of this snapshot (much like highlighting or commenting in a PDF file).
6. Zotero also allows you to add your notes to the reference.

There are numerous features like this and Zotero is under constant development and new versions are being released regularly with additional features. Zotero user forums help to gather ideas and suggestions from the users and are regularly monitored by the development team. So is Zotero PERFECT?

I have one specific problem when using Zotero for a specific purpose. I am working with a couple of medical students to use Zotero for their ePortfolios. We have a home-grown ePortfolio system where all medical students get narrative formative and summative feedback through out their 5 years. These assessments are competency based (similar to ACGME competencies) and organized into areas of improvement and areas of strength. These are all done online and when submitted, they appear in the students' ePortfolio workspaces. The students write essays (ePortfolios) regarding their meeting various standards in each competency citing the evidence from the assessments.

We use EndNote or RefWorks to allow students to download all their references (assessment information), write their essays, cite the references, and then upload it all into a review area. The citations work as hyperlinks so when reviewer reading the essay wants to look at one of the cited references s/he has to just click on the citation to open a new window where the assessment form is displayed.

The problem I am facing with Zotero is that we cannot figure out how to make the citations in the text of the essay or in the bibliography clickable without going to each one and editing them. Since some of the students have as many as a 100 references in their bibliography this is not a viable solution.

It is very surprising that though Zotero is built on the philosophy of getting the references from online databases, it seems to have neglected that the manuscripts written by people using Zotero would be reviewed online where it would be critical for these citations to be hyperlinks!

I am relatively new to Zotero and am quite possibly missing something. I have scoured the Zotero forums, Googled the web and found nothing so far. Maybe the CSL does not at present allow this to occur? Would it be possible to add a "Web style" to Zotero's growing list of styles? If someone can do that, it will be the most appreciated!