• ## Toward an alternative bibliometric

Impact factor isn’t great. A bibliometric based on entropy reduction may be promising.

$\newcommand{cond}[3] { #1\mathopen{}\left(#2\mathbin{\big|}#3\right)\mathclose{} }$

### Impact factor

There are a variety of citation-based bibliometrics. The current dominant metric is impact factor. It is highly influential, factoring into decisions on promotion, hiring, tenure, grants and departmental funding (Editors 2006) (Agrawal 2005) (Moustafa 2014). Editors preferentially publish review articles, and push authors to self-cite in pursuit of increased impact factor (Editors 2006) (Agrawal 2005) (Wilhite and Fong 2012). It may be responsible for editorial bias against replications (Neuliep and Crandall 1990) (Brembs, Button, and Munafò 2013). Consequently, academics take impact factor into account throughout the planning, execution and reporting of a study (Editors 2006).

This is Campbell’s law in action. Because average citation count isn’t what we actually value, when it becomes the metric by which decisions are made, it distorts academic research. In the rest of this post, I propose a bibliometric that measures the entropy reduction of the research graph.

• ## On casual futurism

Predicting the future is hard. Research on forecasting techniques is inconclusive, but scenario planning might be the least bad approach.

The human race, to which so many of my readers belong, has been playing at children’s games from the beginning…. And one of the games to which it is most attached is called “Keep to-morrow dark,” …. The players listen very carefully and respectfully to all that the clever men have to say about what is to happen in the next generation. The players then wait until all the clever men are dead, and bury them nicely. They then go and do something else. That is all. For a race of simple tastes, however, it is great fun. (Chesterton 1904)

“How likely are we to have holophonors by 3002?”

“Pretty likely. They’re pretty much just better oboes, right? 80% chance?

“Alright, 70% chance.”

“Also, note that people are pretty bad at predictions. A study by George Wise found that out of 1556 naive medium-term predictions made publicly by Americans between 1890 and 1940, just under 40% had been fulfilled or were in progress by 19761 (Wise 1976).”

“Ah ha ha. You fell right into my trap! Your casual futurism betrays you! Just by imagining that scenario, you think it’s more likely.”

• ## Choose your own exposition

We can use interactivity to augment traditional text. In particular, we allow for choice of ordering and between alternatives.

### Ordering

When teaching something, is it best to start with concrete and move to the abstract? Or is it best to emphasize the abstract and introduce concrete applications later? Research on this topic is ambivalent (Flores 2009) (De Bock et al. 2011) (Kaminski, Sloutsky, and Heckler 2008) (Peterson, Mercer, and O’Shea 1988). It’s conceivable that the superior approach depends on the student. With one-on-one in-person instruction, this sort of adaptation is possible. With traditional, static text, it’s not. On the web (with computers generally), it is.

• ## A quorum alternative

Voting procedures can replace traditional quorum with the use of statistical techniques. If aggregate opinion on the alternatives is “similar”, according to those techniques, we declare a failure of quorum.

### Motivating examples

1. Pie Club is voting on which pie will be featured at their first August meeting. After tallying the votes, buko pie receives a mean score of $$0.69$$ and fish pie receives a mean score of $$0.18$$.1

Before the decision is finalized, however, an observant member notices that the meeting is two members short of the 25 required for quorum. Because Pie Club is scrupulously democratic, the vote is annulled. Some members grumble their doubt that the landslide will reverse with two more votes.
2. Pie Club is voting on which pie will take home the title “Pie of the Decade”. Will it be lemon meringue pie or Tarta de Santiago? The results are in—quorum checked in advance this time—and they are… $$0.49$$ for meringue and $$0.48$$ for Tarta. While meringue’s devotees celebrate, Tarta’s die-hards feel something has gone wrong. Can such a close result really give them confidence that meringue is the preference of the whole club, including the 12 members who couldn’t make it to meeting? If just one of them had attended and cast a vote favoring Tarta, wouldn’t that have swung the outcome?