Toward an alternative bibliometric

Impact factor

There are a variety of citation-based bibliometrics. The current dominant metric is impact factor. It is highly influential, factoring into decisions on promotion, hiring, tenure, grants and departmental funding (Editors 2006) (Agrawal 2005) (Moustafa 2014). Editors preferentially publish review articles, and push authors to self-cite in pursuit of increased impact factor (Editors 2006) (Agrawal 2005) (Wilhite and Fong 2012). It may be responsible for editorial bias against replications (Neuliep and Crandall 1990) (Brembs, Button, and Munafò 2013). Consequently, academics take impact factor into account throughout the planning, execution and reporting of a study (Editors 2006).

This is Campbell’s law in action. Because average citation count isn’t what we actually value, when it becomes the metric by which decisions are made, it distorts academic research. In the rest of this post, I propose a bibliometric that measures the entropy reduction of the research graph.

On casual futurism

The human race, to which so many of my readers belong, has been playing at children’s games from the beginning…. And one of the games to which it is most attached is called “Keep to-morrow dark,” …. The players listen very carefully and respectfully to all that the clever men have to say about what is to happen in the next generation. The players then wait until all the clever men are dead, and bury them nicely. They then go and do something else. That is all. For a race of simple tastes, however, it is great fun. (Chesterton 1904)

Choose your own exposition

Ordering

When teaching something, is it best to start with concrete and move to the abstract? Or is it best to emphasize the abstract and introduce concrete applications later? Research on this topic is ambivalent (Flores 2009) (De Bock et al. 2011) (Kaminski, Sloutsky, and Heckler 2008) (Peterson, Mercer, and O’Shea 1988). It’s conceivable that the superior approach depends on the student. With one-on-one in-person instruction, this sort of adaptation is possible. With traditional, static text, it’s not. On the web (with computers generally), it is.

A quorum alternative

Motivating examples

1. Pie Club is voting on which pie will be featured at their first August meeting. After tallying the votes, buko pie receives a mean score of $0.69$ and fish pie receives a mean score of $0.18$.¹

   Before the decision is finalized, however, an observant member notices that the meeting is two members short of the 25 required for quorum. Because Pie Club is scrupulously democratic, the vote is annulled. Some members grumble their doubt that the landslide will reverse with two more votes.
2. Pie Club is voting on which pie will take home the title “Pie of the Decade”. Will it be lemon meringue pie or Tarta de Santiago? The results are in—quorum checked in advance this time—and they are… $0.49$ for meringue and $0.48$ for Tarta. While meringue’s devotees celebrate, Tarta’s die-hards feel something has gone wrong. Can such a close result really give them confidence that meringue is the preference of the whole club, including the 12 members who couldn’t make it to meeting? If just one of them had attended and cast a vote favoring Tarta, wouldn’t that have swung the outcome?

About

This is a single-author, half-heartedly pseudonymous (the pseudonym being Cole) blog. The moral imperative and mortal peril of maximizing is the closest you’re likely to get to a statement of purpose.