Category Archives: Medical research

Academic Medicine Hacks

How is Impact Measured?

A scholar’s impact has traditionally been measured via bibliometrics, which include citation impact scores like h-index and i10-index. These indices quantify the number of times someone’s work was cited by others in established databases, such as Google Scholar.

Two broader forms of evaluating the usage and impact of research have emerged in the past decade as an adjunct to traditional citation metrics:

  1. Article-level metrics (ALMs) provide a wide range of metrics about the uptake of an individual journal article by the scientific community after publication. Starting in 2009, the Public Library of Science (PLoS) introduced ALMs for all articles.
  1. Altmetrics, proposed in 2010 measure the broader impact of a work such as how many data and knowledge bases refer to it, article views, downloads and mentions in news and social media.

Several services calculate ALM and altmetrics statistics, including Plum Analytics , Altmetric and ImpactStory. Depending on your goals for tracking and disseminating your impact, one or more of these services may be best for you.


Track your Impact Metrics

Track Impact Metrics for Your Published Work

Most published work has a DOI—a persistent interoperable identifier—that is in a system indexed by one of the services that calculated ALMs.

  • The subscription form of ImpactStory ($60/year) draws on numerous sources of existing data to compile an overview of your overall impact. The free version draws on your ORCID identifier and Twitter feed to compile a more abbreviated overview of your impact. Here is mine as an example.
  • Altmetric tracks more diverse sources and thus provides more granular data about the context in which your work was cited (who, where, how, when etc.) It provides this information at the publication-level rather than at the scholar-level (like ImpactStory)—a task made easier by the Altmetric bookmarklet. So, for example, here is the Altmetric data for one of my recent publications.
  • Plum Analytics was acquired by Elsevier and seems more oriented towards selling larger research institutions altmetric data related to comparing researchers and institutions.

Track Impact Metrics for Your Unpublished Work

Unpublished work includes diverse bodies of scholarly work including but not limited to

Educational resources: curricula, lecture slide decks, podcasts, outreach, simulation scripts and other educational modalities.

  • Grey literature: this includes online reports not indexed in the National Library of Medicine as well as conference proceedings, quality improvement reports, academic blog posts and other output.
  • Community impact: if work you have done—published or not—has impact on the community, this is not always captured by altmetrics sources. This impact includes media coverage, changes to policy and program implementations or improvements.
  • Tracking impact not connected to a traditional academic URL database requires more creativity. Although each publication has its own PubMed-indexed URL, other academic output does not.

Create unique URLs for each of your web-based outputs. If there is an existing academic sharing repository for these, use it. Some examples are:

  • Medical education materials: MedEdPORTAL
  • Other educational resources and grey literature: I use SlideShare to maintain a URL for each non-PubMed output. This includes materials I make “public” such as lecture slides and posters, as well as materials I share on a more limited basis, such as program description documents.  Two other repositories for sharing scholarly output are:
    • FigShare A repository where users can make all of their research outputs available in a citable, shareable and discoverable manner.
    • Zenodo A repository service that enables researchers, scientists, projects and institutions to share and showcase multidisciplinary research results (data and publications) that are not part of existing institutional or subject-based repositories.

To track your metrics for output on these academic sharing repository sites, you can

  • Follow metrics on the site.
  • Convert each SlideShare URL for each of these outputs into a unique URL that you can track yourself. I use Google URL Shortener because of its great analytics. If you are going this route, use these shortened URLs every timeyou share your content. Follow your use metrics on the Google URL Shortener website.
  • The subscription version of ImpactStory tracks SlideShare and figshare.
  • If something you do cannot be hosted on an academic sharing repository site, host it somewhere (like Google Docs) and use Google analytics to track its usage metrics.
  • On each poster you present at a conference, post the poster on SlideShare and then include a QR code for the poster’s unique URL on the poster itself. There are many free QR code generators online. Here is the QR code of the SlideShare version of a poster: 2BD8FA82-71B1-4CC2-BCD0-0D0786589C1F

Amplify your Reach

Social media and blogging can help disseminate the reach of your work. Much has been written about the downsides of social media, including its association with negative mood, wasted time, loss of privacy and identity theft. If your institution has social media amplification resources—accounts that promote the work of faculty—this is a win-win, as it can amplify your work without the personal risk to you. The downside of this is that the university accounts own your metrics—although you can work around some of this via the “Track your Metrics” hacks above.

If you are on social media, promote your own content (using your unique URLs) and the hashtags that get the most reach in your area. To identify the hashtags that are trending, consider searching the Healthcare Hashtag Project. Conference hashtags like #PAS17 are often promoted by the meeting organizer and are a good way of boosting your reach.

I also recommend this compendium of resources for social media use in academic medicine.

Present your Metrics

In academic medicine, we present the impact of our work both intramurally—in our annual reviews and promotions portfolios—and extramurally, in our NIH Biosketches, personal webpages and other venues.  You can also include a link to the ImpactStory or Altmetric details relevant to your cited impact metrics to support your statements.97DF9312-E28D-4AF7-88AB-F4D0F531FCC7

Networking Sites

There are numerous sites to post an online compilation of your work; many will do some of the work for you.

  • NCBI will import your PubMed-indexed citations into “My Bibliography”, and then generates a sharable URL for this bibliography. It’s also a useful site for storing collections, setting up PubMed searches and managing your SciENcv biosketch generator.
  • Google Scholar also allows you to create a profile including all your publications in the Google Scholar database.
  • ORCID, mentioned above in relation to ImpactStory, compiles your publications and federal grants in one place, and allows you to enter other professional information. It is used by some publication managers as a login ID.
  • ResearchGate allows you to keep up on publications in your field and allows others in academic medicine to follow your output.
  • LinkedIn is not tailored to an academic online CV (hard to import publications, for example), but it is used by so many sectors as a networking resource that it is worth being on, if you don’t mind its association with Microsoft.
  • Doximity was built as a LinkedIn for physicians, and is more compatible with PubMed and some altmetrics. It is only accessible to physicians.
  • Many universities, including mine, host faculty profiles that can be edited by the faculty to some extent. As examples, mine are at:University of Colorado profileColorado Clinical and Translational Sciences Institute profile and Children’s Hospital Colorado profile.

Two most common pediatric migraine medications no more effective than placebo

Neither of the two drugs used most frequently to prevent migraines in children–amitriptyline and topiramate–is more effective than a placebo, according to results of the Childhood and Adolescent Migraine Prevention (CHAMP) trial published this week in The New England Journal of Medicine. The investigators found no significant differences in reduction in headache frequency or headache-related disability in childhood and adolescent migraine with amitriptyline, topiramate, or placebo over a period of 24 weeks.

The active drugs were associated with higher rates of adverse events. One child on topiramate attempted suicide. Three taking amitriptyline had mood changes; one told his mother he wanted to hurt himself, while another wrote suicide notes at school and was hospitalized.screen-shot-2016-10-28-at-4-01-47-pm

Migraine headaches are common in children. Up to 11 percent of 7- to 11-year-olds and 23 percent of 15-year-olds have migraines.

AHRQ and CMS Public Reporting Measures Fail to Describe the True Safety of Hospitals

A new study from the Johns Hopkins Armstrong Institute for Patient Safety and Quality, published in the journal Medical Care, performed a systematic review and meta-analysis of two sets of safety measures used for pay-for-performance and public reporting The measures evaluated in the study are used by several public rating systems, including U.S. News and World Report’s Best Hospitals, Leapfrog’s Hospital Safety Score, and the Center for Medicare and Medicaid Services’ (CMS’) Star Ratings.

The two sets of measures evaluated are:

The investigators first performed a systematic review of all published medical research since 1990, looking for studies that addressed the validity of the HAC  and PSI measures.  They identified only 5 of these 40 safety measures with enough data in these prior studies to permit a pooled meta-analysis:

  • A. Iatrogenic Pneumothorax (PSI 6/HAC 17)
  • B. Central Line-associated Bloodstream Infections (PSI 7)
  • C. Postoperative hemorrhage/hematoma (PSI 9)
  • D. Postoperative deep vein thrombosis/pulmonary embolus (PSI 12)
  • E. Accidental Puncture/Laceration (PSI 15)

The investigators then performed a meta-analysis, pooling the results of all studies about the validity of each of these measures.  Their findings in the figure below show that in pooled studies (the diamond at the bottom of each lettered rectangle) only one measure–Measure E, PSI 15 (Accidental Puncture and Laceration)–met the investigators’ criteria for validity: a positive predictive value of at least 80% (indicating that at least 80% of the patients determined by the measure to have an accidental puncture or laceration truly had an accidental puncture or laceration.) Actual occurrence (reference standard) of each reported safety event was determined, in each individual study, by medical chart review.

Screen Shot 2016-06-02 at 6.12.26 AM

Measure D, PSI 9 (Postoperative hemorrhage or hematoma) came close to the 80% PPV threshold, with a pooled PPV of 78.6%.

Based on these findings, the investigators conclude that these measures, widely used for public reporting and pay-for-performance, should not be used for either purpose:

 CMS and others have created payment incentives based on hospitals’ performance for a variety of hospital-acquired complications, which are measured with the respective PSIs and HAC measures. Policy makers and payers have argued that the PSIs and HAC measures are good enough for reporting and pay-for-performance, whereas many providers believe they are not. Our results suggest that the PSIs and HAC measures may not be valid enough and/or have insufficient data to support their use for these purposes. This is especially true given the potential financial impact these pay-for-performance approaches may have on the narrow financial margins on which most hospitals function.


Medical ecology: tending the microbiome

Today, the Obama administration announces the new National Microbiome Initiative, intended to create scientific tools, discoveries and training techniques related to the human microbiome, the 100 trillion microbes that live in the human body. Tending the human microbiome may help in the treatment of infections, as well as disorders that would seem unrelated to microbes, including obesity and diabetes.

The microbiome represents the only organ that can be replaced without surgery,” said Jo Handelsman, a microbiologist at the White House. “Just by eating differently, taking drugs, exercising and other things, you can have fairly immediate effects on your microbiome and your health, if we only knew how.”

Microbiome research also has important public health implications. New research, for instance, suggests that much of the world’s childhood malnutrition arises not from a lack of food, but from problems with children’s intestinal microbiomes caused by poor sanitation.India-Sanitation-web-Artboard_1

Should Doctors Undergo Opioid Prescribing Risk Training?

Earlier this week, an advisory panel recommended that the Food and Drug Administration require doctors who prescribe painkillers s to undergo training aimed at reducing misuse and abuse of the medications. The New York Times notes:

It is the second time since 2010 that an F.D.A. panel has recommended expanding safety measures for painkillers. But the training plans instituted about four years ago are voluntary, and data shows that under half of the doctors targeted by the effort have completed the training.


Despite the rising opioid-related death rate since the initial FDA panel’s initial recommendation in 2010, the panel strongly recommended training physicians. Given the amount of training we all underwent in preparation for US cases during the most recent Ebola epidemic, the physician training for this pandemic, far more deadly on our shores, seems both feasible and urgent.

What are the priority Emergency Department (ED) presenting conditions for which ED-based Shared Decision-Making should be most urgently developed?

An article in this week’s Wall Street Journal focused on the development and use of shared decision-making (SDM) tools in the emergency department (ED).  Decision tools can help engage patients in making decisions about their care, including decisions about which tests and treatments to pursue.  In the ED, relevant decisions are (1) decisions involving two similarly reasonable options (“preference-sensitive health care decisions”) and (2) decisions that patients or their families can take some time to consider. The examples mentioned in the WSJ article are

  1. children with uncomplicated acute appendicitis: IV antibiotics versus surgery
  2. adults with chest pain and normal initial test results: watchful waiting (not doing any tests) versus extensive testing (you can try out this tool here)
  3. mild traumatic injury: watchful waiting versus head CT scan

Other scenarios for which SDM tools have been studied include

  • testing for bacterial infections among children with fever-without-an-obvious-source
  • choice of anesthesia and sedation for the repair of small lacerations in children
  • choice of hydration methods (IV or oral) for children with dehydration from a vomiting/diarrhea illness.
  • management of acute musculoskeletal pain in adults

shared-decision-making-were-in-this-together-3-728Selection of SDM is the focus for the 2016 Academic Emergency Medicine consensus conference, “Shared Decision-making in the Emergency Department: Development of a Policy-relevant Patient-centered Research Agenda”. I pose here one of the many questions to be addressed at that conference: What are the priority ED presenting conditions for which ED-based SDM should be most urgently developed?


Fall in life expectancy for white Americans

Newly released 2014 data from the National Center for Health Statistics on life expectancy showed a worrisome decline in life expectancy for whites in the US, to 78.8 years in 2014 from 78.9 in 2013.

The good news is that, in contrast, life expectancy increased by 0.2 years for the Hispanic population (from 81.6 to 81.8 years) and by 0.1 years for the non-Hispanic black population (from 75.1 to 75.2). This continues a trend of a decreasing gap in life expectancy between black and white populations, with improving black life expectancies attributed to decreases in death rates due to heart disease, cancer, HIV disease, unintentional injuries, and perinatal conditions.db218_fig1

Life expectancy rates had steadily for decades, then flattened in 2010-2013. joi150117f1The decline in life expectancy is driven by increased death rates among young and middle-aged whites (mid 20’s to mid 50’s), especially among those with no more than a high school education.  A recent study attributed the rise in mortality in this group  to rising rates of suicide, liver disease and drug overdoses.


Life expectancy has dipped before–most recently in 2005, the year of a severe influenza epidemic.