Academic Medicine Hacks

How is Impact Measured?

A scholar’s impact has traditionally been measured via bibliometrics, which include citation impact scores like h-index and i10-index. These indices quantify the number of times someone’s work was cited by others in established databases, such as Google Scholar.

Two broader forms of evaluating the usage and impact of research have emerged in the past decade as an adjunct to traditional citation metrics:

  1. Article-level metrics (ALMs) provide a wide range of metrics about the uptake of an individual journal article by the scientific community after publication. Starting in 2009, the Public Library of Science (PLoS) introduced ALMs for all articles.
  1. Altmetrics, proposed in 2010 measure the broader impact of a work such as how many data and knowledge bases refer to it, article views, downloads and mentions in news and social media.

Several services calculate ALM and altmetrics statistics, including Plum Analytics , Altmetric and ImpactStory. Depending on your goals for tracking and disseminating your impact, one or more of these services may be best for you.


Track your Impact Metrics

Track Impact Metrics for Your Published Work

Most published work has a DOI—a persistent interoperable identifier—that is in a system indexed by one of the services that calculated ALMs.

  • The subscription form of ImpactStory ($60/year) draws on numerous sources of existing data to compile an overview of your overall impact. The free version draws on your ORCID identifier and Twitter feed to compile a more abbreviated overview of your impact. Here is mine as an example.
  • Altmetric tracks more diverse sources and thus provides more granular data about the context in which your work was cited (who, where, how, when etc.) It provides this information at the publication-level rather than at the scholar-level (like ImpactStory)—a task made easier by the Altmetric bookmarklet. So, for example, here is the Altmetric data for one of my recent publications.
  • Plum Analytics was acquired by Elsevier and seems more oriented towards selling larger research institutions altmetric data related to comparing researchers and institutions.

Track Impact Metrics for Your Unpublished Work

Unpublished work includes diverse bodies of scholarly work including but not limited to

Educational resources: curricula, lecture slide decks, podcasts, outreach, simulation scripts and other educational modalities.

  • Grey literature: this includes online reports not indexed in the National Library of Medicine as well as conference proceedings, quality improvement reports, academic blog posts and other output.
  • Community impact: if work you have done—published or not—has impact on the community, this is not always captured by altmetrics sources. This impact includes media coverage, changes to policy and program implementations or improvements.
  • Tracking impact not connected to a traditional academic URL database requires more creativity. Although each publication has its own PubMed-indexed URL, other academic output does not.

Create unique URLs for each of your web-based outputs. If there is an existing academic sharing repository for these, use it. Some examples are:

  • Medical education materials: MedEdPORTAL
  • Other educational resources and grey literature: I use SlideShare to maintain a URL for each non-PubMed output. This includes materials I make “public” such as lecture slides and posters, as well as materials I share on a more limited basis, such as program description documents.  Two other repositories for sharing scholarly output are:
    • FigShare A repository where users can make all of their research outputs available in a citable, shareable and discoverable manner.
    • Zenodo A repository service that enables researchers, scientists, projects and institutions to share and showcase multidisciplinary research results (data and publications) that are not part of existing institutional or subject-based repositories.

To track your metrics for output on these academic sharing repository sites, you can

  • Follow metrics on the site.
  • Convert each SlideShare URL for each of these outputs into a unique URL that you can track yourself. I use Google URL Shortener because of its great analytics. If you are going this route, use these shortened URLs every timeyou share your content. Follow your use metrics on the Google URL Shortener website.
  • The subscription version of ImpactStory tracks SlideShare and figshare.
  • If something you do cannot be hosted on an academic sharing repository site, host it somewhere (like Google Docs) and use Google analytics to track its usage metrics.
  • On each poster you present at a conference, post the poster on SlideShare and then include a QR code for the poster’s unique URL on the poster itself. There are many free QR code generators online. Here is the QR code of the SlideShare version of a poster: 2BD8FA82-71B1-4CC2-BCD0-0D0786589C1F

Amplify your Reach

Social media and blogging can help disseminate the reach of your work. Much has been written about the downsides of social media, including its association with negative mood, wasted time, loss of privacy and identity theft. If your institution has social media amplification resources—accounts that promote the work of faculty—this is a win-win, as it can amplify your work without the personal risk to you. The downside of this is that the university accounts own your metrics—although you can work around some of this via the “Track your Metrics” hacks above.

If you are on social media, promote your own content (using your unique URLs) and the hashtags that get the most reach in your area. To identify the hashtags that are trending, consider searching the Healthcare Hashtag Project. Conference hashtags like #PAS17 are often promoted by the meeting organizer and are a good way of boosting your reach.

I also recommend this compendium of resources for social media use in academic medicine.

Present your Metrics

In academic medicine, we present the impact of our work both intramurally—in our annual reviews and promotions portfolios—and extramurally, in our NIH Biosketches, personal webpages and other venues.  You can also include a link to the ImpactStory or Altmetric details relevant to your cited impact metrics to support your statements.97DF9312-E28D-4AF7-88AB-F4D0F531FCC7

Networking Sites

There are numerous sites to post an online compilation of your work; many will do some of the work for you.

  • NCBI will import your PubMed-indexed citations into “My Bibliography”, and then generates a sharable URL for this bibliography. It’s also a useful site for storing collections, setting up PubMed searches and managing your SciENcv biosketch generator.
  • Google Scholar also allows you to create a profile including all your publications in the Google Scholar database.
  • ORCID, mentioned above in relation to ImpactStory, compiles your publications and federal grants in one place, and allows you to enter other professional information. It is used by some publication managers as a login ID.
  • ResearchGate allows you to keep up on publications in your field and allows others in academic medicine to follow your output.
  • LinkedIn is not tailored to an academic online CV (hard to import publications, for example), but it is used by so many sectors as a networking resource that it is worth being on, if you don’t mind its association with Microsoft.
  • Doximity was built as a LinkedIn for physicians, and is more compatible with PubMed and some altmetrics. It is only accessible to physicians.
  • Many universities, including mine, host faculty profiles that can be edited by the faculty to some extent. As examples, mine are at:University of Colorado profileColorado Clinical and Translational Sciences Institute profile and Children’s Hospital Colorado profile.

AHCA Medicaid Cuts Would Hit Rural Areas Hardest

According to an analysis conducted by the Georgetown University Center for Children and Families, the Affordable Care Act’s (ACA) Medicaid expansion disproportionately benefited rural Americans over their urban counterparts.  The American Health Care Act’s (AHCA) proposed cuts to Medicaid, rolling it back to below funding levels established by the ACA, would negatively affect millions of kids and adults.

Rural and small town voters–those  often credited with Trump’s 2016 election victory, could stand to lose the most if the president’s health-care overhaul is signed into law. Both nationally and in Colorado, the report found, higher percentages of people in small towns and rural communities are covered by Medicaid than in cities.

Medicaid plays a larger role in providing health coverage to families living in small towns and rural communities than it does in metropolitan areas, a trend that is particularly striking among children. About 45 percent of children in small towns and rural areas rely on Medicaid for their coverage, compared to 38 percent in metro areas. In 14 states, more than half of the children outside of metro areas receive health benefits from Medicaid and the Children’s Health Insurance Program (CHIP). County level urban and rural Medicaid coverage across the US are available on this map:

in Colorado, rural area Medicaid coverage grew 11% with the ACA. 42% of rural kids in CO are covered by #Medicaid.

Impact of AHCA Medicaid Cap on Children

A recent analysis by Avalere projected the impact of federal Medicaid cap proposals on overall and state-by- state federal spending on children currently in Medicaid. The report estimated that the House-passed #AHCA would cut Medicaid for kids by $43 billion by 2026. In Colorado, these cuts would amount to $571 million.

While this study focused exclusively on non-disabled children, the high costs associated with disabled children in Medicaid would likely make the total impact of the AHCA on kids significantly greater.

Fact Check: the Colorado Case for Repeal of the Affordable Care Act (and replacement with the Affordable Health Care Act) is a False Narrative

The Affordable Health Care Act (AHCA or “Trumpcare”) will cut 24 million from insurance by 2026 including 410,600 from Colorado (use this interactive feature to see how many will be impacted in your state).  Colorado’s own Senator Cory Gardner has expressed his reservations about the 14 million projected to lose Medicaid, describing Medicaid as providing “access to life-saving health care services” for the 1.3 million Coloradans—and 72 million Americans—who get their insurance through Medicaid, including the Medicaid expansion part of the Affordable Care Act (ACA). Given the ominous projections about the AHCA’s impact compared to staying with the ACA, how did Republican legislators end up with such a destructive bill after 8 years of decrying the disastrous results of the ACA? The unfortunate reality of what the ACA-to-AHCA transition would mean highlights the accomplishments of the ACA as it is on the verge of elimination: it has improved healthcare access and outcomes for millions of Americans.

The mythology on which the Republican members of congress have based their anti-ACA rhetoric is exemplified by the talking points used by our Colorado legislators. Senator Gardner has reiterated his anti-ACA talking points in his two recent tele-town halls.   As he has before, Senator Gardner gave three arguments for why the ACA must be repealed:

  • Health insurance premiums in Colorado are rising because of the ACA
  • Coloradans lost their health insurance because of the ACA
  • Coloradans can’t access their doctor because of the ACA

Each argument bent the truth, casting the ACA as causing the problem rather than what the data shows it is: part of the solution. If the ACA did not cause the problem, repeal is not the solution.

  • Premiums are rising.  True, but we don’t know how much is due to the ACA and the rise is exaggerated. Premiums were rising before the ACA. This year’s rise brought lower-than-predicted premiums into line with the 2017 rates predicted by the CBO in 2009. The 400% rise cited by Senator Gardner in both tele town halls– of “people whose premium rose from $300 to $1500 a month”—grossly overstates the average rise of 20.4% in Colorado–an increase that primarily impacts those who buy individual policies—about 3% of the population–and is often offset by subsidies.
  • Hundreds of thousands of Coloradans lost insurance: Not true.   By “lost insurance”, the Senator is referring to “churn”—transitioning between plans, sometimes a result of canceled plans—and not about a net loss in insured people in Colorado. The rate of churn was not increased by the ACA. Most cancelations—about 250,000—involved limited benefit plans that were not ACA-compliant, and thus had to be either made ACA-compliant or dropped when the ACA was enacted.  Despite the churn, Colorado has nevertheless seen a net increase of over 530,000 in the number insured. In fact, Coloradans continue to enroll in plans in record-breaking numbers, with a 12% increase seen over the same time last year in the 2017 open enrollment period.
  • People can’t see their doctors because of high deductibles. Not true.  The 2015 Colorado Health Access Survey shows that, since the ACA was passed, increased coverage has meant increased access. Coloradans have steadily increased their rate of preventive health visits by 7% (from 61.9% in 2013 to 66.1% in 2015) and have lowered their rate of skipping doctors’ visits due to costs by 15% (from 12.3% in 2013 to 10.4% in 2015).

Although rising premiums, churn and barriers to accessing doctors were not eradicated by the ACA, this review of Colorado data shows that the ACA is part of the solution, not part of the problem. To do what is best for Coloradans, Senator Gardner needs to step away from his false narrative about the ACA and address real issues reflected in Colorado data.  He has taken a first step in raising concerns about threats to Medicaid. There are ways to strengthen the ACA to address some of the real issues in healthcare, but to repeal it based on a false narrative disseminated by Senator Gardner and his Republican colleagues would be a tragedy.  Let’s look at the facts, and work together to make the system better, for Colorado and the rest of America.