Interpreting the Hugo Votes, or: Hugo Thunderdome!

For those still tuned into the Hugos and Worldcon, the full results—including counts, 1st/2nd/3rd/4th/5th placement, and such, are here:

http://www.denvention3.org/hugos/Final-Report.pdf

The interesting part for most people starts on page 10, where the results of the “instant-runoff” voting algorithm is posted. Some of that triangle of numbers may be a bit confusing (like “why does Brasyl not place second when it kept ranking second through all those numbers until the end…?”) so here is a very Hugo-specific guide to the voting mechanics.

This may not interest everybody on the front page, so the guide and example analysis is under the cut.


Some information if you’ve never voted in the Hugos: in each category:

  • You rank the nominees according to your preference; it’s not a straight-out “this one wins” vote. “1” is “I like this one the best” down to “5”, “I like this one fifth-best”.

  • You don’t have to fill out every preference—maybe you only read three of the novels, so you just rank those.

  • There’s a “no confidence” box per category that you check if you think none of the candidates should win at all (which is… some amount of hate right there).

    Claudia corrects me below: the “No Award” vote can be ranked among the other candidates as if it were a candidate if you choose; any nominees that you don’t rank above the “No Award” slot you consider not worthy of winning the award at all.

  • And some categories you may not feel you have enough information to have an opinion—so you just don’t vote.

  • And as you can see by the first few pages, quite a few write-in candidates exist. — correction from Simon below: these are nominations for who should be on the final ballot, not write-ins.

As Claudia says, the official Hugo Awards site has a nice summary.

After all the votes are in, the algorithm churns—and it churns to determine each placing after first as well. The best rough explanation is a picture (from the self same Wikipedia page):

IRV_counting_flowchart.png

It looks like “majority” in the Hugos means straight majority, not “more than half” majority.

Correction—that’s a plurality, and the Hugos actually still work on a majority. (It just so rarely happens, so it’s more common to eliminate the candidates down to one.)

So let’s pick the Hugo novel category and see what happened. Here we’re digesting the top triangle of numbers on page 10. Each round below represents one column in the triangle.

Note: the “No Award” interactions have been corrected below, thanks to Claudia, who also has nice examples in her comments below.

Picking 1st Place (the winnah)

Round 1:
The Yiddish Policeman’s Union – 195 people ranked it as #1.
The Last Colony – 158 people ranked it as #1.
Rollback – 152 people ranked it as #1.
Halting State – 115 people ranked it as #1.
Brasyl – 110 people ranked it as #1.
No Award – 15 people ranked the No Award first.

For determining first place, the No Award is a no-confidence vote. Here, 15 people didn’t think any of the nominees should win.

We’ll cover the special No Award test at the end of this section.

The fewest people voted for the No Award, so it’s eliminated like any other candidate. They (obviously) didn’t vote for any other candidate, so there are no votes here to re-distribute.

Round 2:
The Yiddish Policeman’s Union – 195 people ranked it as #1.
The Last Colony – 158 people ranked it as #1.
Rollback – 152 people ranked it as #1.
Halting State – 115 people ranked it as #1.
Brasyl – 110 people ranked it as #1.
No Award

Note that Brasyl got the fewest “#1” votes. It gets eliminated, and votes are redistributed.

Round 3:
The Yiddish Policeman’s Union
        – 195 people ranked it as #1
        – 36 people ranked it as #2
         total: 231
The Last Colony
        – 158 people ranked it as #1
        – 12 people ranked it as #2
         total: 170
Rollback
        – 152 people ranked it as #1
        – 11 people ranked it as #2
         total: 163
Halting State
        – 115 people ranked it as #1
        – 33 people ranked it as #2
         total: 148
Brasyl
No Award

Good attempt at a save, people-who-like-Halting State-second-best, but no dice. Halting State is still lowest in number of votes, so it gets eliminated.

Note that there are 18 votes left over from Brasyl‘s vote redistribution, so that means 18 people marked Brasyl as #1 and left the other nominee boxes blank.

Round 4:
The Yiddish Policeman’s Union
        – 195 people ranked it as #1
        – 36 people ranked it as #2
        – 61 people ranked it as #3
         total: 292
The Last Colony
        – 158 people ranked it as #1
        – 12 people ranked it as #2
        – 49 people ranked it as #3
         total: 219
Rollback
        – 152 people ranked it as #1
        – 11 people ranked it as #2
        – 23 people ranked it as #3
         total: 186
Halting State
Brasyl
No Award

Lots of people placing YPU and TLC; the Scalzi faction is here, folks. Rollback is eliminated next.

By the way, 15 people stopped at #1-Brasyl #2-Halting State.

Round 5:
The Yiddish Policeman’s Union
        – 195 people ranked it as #1
        – 36 people ranked it as #2
        – 61 people ranked it as #3
        – 40 people ranked it as #4
         total: 332
The Last Colony
        – 158 people ranked it as #1
        – 12 people ranked it as #2,
        – 49 people ranked it as #3
        – 104 people ranked it as #4
         total: 323.
Rollback
Halting State
Brasyl
No Award

So it’s YPU by a mere space of just 9 votes—The Last Colony, in other words, showed very strongly in the winning race.

42 people stopped after #1-Brasyl #2-Halting State #3-Rollback.

Now it’s time for the No Award test. All ballots that ranked YPU above “No Award” are counted versus the ballots that rank “No Award” higher than YPU. If the “No Award” wins, then there’s no winner and no award this year; otherwise, there is a winner. In this case, 483 ranked YPU above “No Award”, and only 52 ranked “No Award” above YPU.

So YPU is the winner. Does this ranking above mean that The Last Colony is automatically second place?

Well, there’s a very strong showing, but no—we need to churn through the algorithm again for second place, without YPU this time around.

Ding!

Picking 2nd Place (after eliminating all YPU considerations)

Before we get underway: eliminating one of the candidates (YPU) means that preferential ordering is partly shuffled around; things that were ranked as being after YPU move up one ranking, which has different effects if YPU is first versus last versus in-between. Thus numbers from this section won’t necessarily have much obviously to do with numbers from the previous section.

It’s best to think of numbers in this section as “ranked as #blah after eliminating YPU from all considerations”.

Round 1:
The Last Colony – 198 people ranked it as #1.
Brasyl167 people ranked it as #1.
Rollback – 172 people ranked it as #1.
Halting State – 155 people ranked it as #1.
No Award – 17 people ranked No Award higher than the remaining candidates.

“No Award” is eliminated again for having the least votes.

However, this time two votes do get redistributed—which means that two people explicitly ranked at least one of the remaining candidates below the “No Award”.

Round 2:
The Last Colony
        – 198 people ranked it #1.
        – 1 person ranked it just after the “No Award”.
         total: 199
Brasyl
        – 167 people ranked it #1.
Rollback
        – 172 people ranked it #1.
Halting State
        – 155 people ranked it #1.
        – 1 person ranked it just after the “No Award”.
         total: 156
No Award

After redistributing the 2 votes from above, we still end up with Halting State in last place, as far as second-placing goes, and it’s eliminated and all its votes redistributed.

Round 3:
The Last Colony
        – 198 people ranked it #1.
        – 1 person ranked it just after the “No Award”.
        – 50 people ranked it after Halting State
         total: 249
Brasyl
        – 167 people ranked it #1.
        – 60 people ranked it after Halting State
         total: 227
Rollback
        – 172 people ranked it #1.
        – 28 people ranked it after Halting State
        – total: 200
Halting State
No Award

Goodbye, Rollback. We redistribute your votes now (assuming there are votes after Rollback for those ballots).

Round 4:
The Last Colony
        – 198 people ranked it #1.
        – 1 person ranked it just after the “No Award”.
        – 50 people ranked it after Halting State
        – 119 people ranked it after Rollback
         total: 368
Brasyl
        – 167 people ranked it #1.
        – 60 people ranked it after Halting State
        – 28 people ranked it after Rollback
         total: 255
Rollback
Halting State
No Award

The Last Colony wins second place strongly. I don’t want to get in the way of The Last Colony, I really don’t.

Everything Else

The rest of the narrative from page 10 should be a little clearer now. You can watch Brasyl lose steam, Halting State battle back up, and Rollback hanging on. Nothing too riveting.

Other notable battles from some of the pages after page 10:

Best Novelette

Merchant and the Alchemist’s Gate totally beat up everybody else, which is obvious even without a more detailed narrative: 201 first votes, over 70 past the next guy. The Best Novel category did not see this early drama.

Best Short Story

The first vote battle was even more decisive here; “Tideline” had 213 first votes, over 100 votes past the next guy. Congratulations, Bear—you totally blew everyone else out of the water, and “Tideline” didn’t lose much steam through the rounds!

Best Related Book

This is a particularly interesting battle. The race here is so tight that only the final round was decisive, and then by a narrow margin. You can even see the tightness continue in the next determinations for second/third/etc-placing. Yes, The Arrival finished technically last, but it was a tough battle—people had a tough time choosing, overall.

Many of the battles were pretty tight, actually, but Best Related Book was especially so.

Best Dramatic Presentation, Short Form

Doctor Who fans were out in force, since first votes swept with one Doctor Who (which also shows an extremely strong preference for Blink over Family of Blood), and second votes swept with the other Doctor Who.

Best Fan Writer

Scalzi does a very strong sweep, similar to Bear. Yes, I definitely do not want to get in the way of either Scalzi or Bear fans.

Conclusions

There are a lot of wounds to be licked after the Hugos, as always. But it’s important to remember that every nominee had to claw over many others to get here—and that even the alternative final ballot nominations (a very interesting first few pages of the Hugo final tally document) are excellent. Sometimes we may feel slighted that something didn’t place as well as it should have, but the competition is pretty fierce and preferential voting is non-straightforward (though fair for all that).

I’m not going to say something lame like “we’re all winners!” but I would never say that anybody really lost out.

By the way, I’m kind of sad that Name of the Wind didn’t get nominated… but I’m also a Scalzi fangirl, so I was happy about the Best Fan Writer bit.

Advertisements

13 thoughts on “Interpreting the Hugo Votes, or: Hugo Thunderdome!

  1. Just a note to say that the first few pages aren’t write-ins, but the nominations for who should be on the final ballot. They don’t publish that info when the titles on the ballot are announced, presumably because it might influence how people vote.

  2. I think you’re misinterpreting the No Award option. This is only “No Confidence” if it’s given vote #1. It’s treated as a ‘candidate’ like any other, and is generally the first eliminated, but not necessarily. It just delineates the point at which you think the remaining candidates aren’t prize-worthy.

    So a rabid Doctor Who fan might have voted
    #1 Blink
    #2 the two-parter
    #3 No Award

    Which means they don’t think Razor (for example) should get an award, but not that they think Blink shouldn’t.

    But hey, if they then chose to rank the remaining shows as well, that could conceivably have had an effect on the outcome… once No Award had been eliminated. Suppose the single-party Doctor Who voter had continued the ballot:

    #4 Torchwood
    #5 BSG
    #6 New Voyages

    Then this ballot could contribute to Torchwood winning out over BSG in an alternate universe in which the DW episodes were eliminated first… but the ballot would then be one count *against* Torchwood in the final No Award test (in which more ballots must have placed the winner ahead of No Award than vice versa).

    http://www.thehugoawards.org/index.php?page_id=4 really is quite a good summary.

  3. … also, majority means more than half… I’m not sure what you mean by “straight majority”. Do you mean plurality? Then no. It means “at least 50%” here.

  4. Interesting. It’s clearly a bending-over-backwards-to-be-fair system, but rather hard to follow. It does make me feel better about the raw numbers I saw for Human Nature/The Family of Blood, which at first glance looked to be ranked fourth in the field. Presumably the Doctor Who fans overwhelmingly went with Blink for #1, and the non-Who people chose the Star Trek and BSG entries. I’m glad to know that under the system explained above, Paul Cornell’s episodes did well after all!

  5. Claudia,

    You’re right about the No Award vote. I had that wrong. I knew it was treated like another candidate, from how it was peculating through the rounds. I’ll be correcting that. Thanks for the link to the official Hugo Awards site—I should have been tracking that down myself.

    And your explanation of the interaction of the No Award is very good.

    I did mean plurality—sorry for the sloppy terminology there.

  6. Okay, all corrected, including the No Award interactions. Thank you, Claudia!

    Karen, yeah. And I also felt better about The Arrival losing out for Best Related Book. Although it really should have been in a Graphic Novel category (that doesn’t exist yet, but is proposed for next year), and the lack of a more appropriate category probably didn’t help it….

  7. Nice review of the way it’s done. I just posted something about this on another site, and thought I’d follow up with a couple basic things about the process:

    1. Instant runoff voting simulates what would happen if everyone were in a room and voting in person, going to stand behind their favorite nominee until having to move to their next choice if their favorite loses. The last-place candidate is dropped in each round. You get a winner anytime a nominee has more than half of the “active votes” cast for nominees still in the running. The winner typically will be the nominee that gets the most first choices, but not always — it can change if the majority vote is “split” among other nominees.

    2. To get the second place finisher, you don’t just take the nominee who finishes second in the count, as that nominee doesn’t have ANY of the votes cast for the winner. It’s possible that one of the earlier nominees eliminated is the second choice of lot of backers of the winner. So instead, you start over and pretend the winner isn’t there – -again, getting the result you would get if everyone were in the same room casting the same rankings as they did in the first vote.

    This system of voting has worked in a lot of governmental elections over the years — nice way of handling voters having more choices.

  8. A nitpick of Jack Boyd’s point 2: you start over, in the case of the Hugo balloting at least, _without the ballots that cast first-place votes for the first-place winner_. This is sort of arbitrary in the case of ranked results like these, but when it is used for voting in a political context, say five slots from your district in a legislative body, it means that everybody’s vote was used to elect somebody (until you get past the threshold where there aren’t enough votes to elect another person), and is intended to encourage a diversity of representation. (In that case, you remove the number of ballots needed to elect one representative, rather than all the first place votes as the Hugo does.)

    Pedant by nature,
    Claudia

  9. Claudia — I’m pretty sure that all the ballots that elected the first-place winner are used again when determining second-place and the rest of the finishers.

    For example, suppose a very unlikely scenario. 55% of Hugo voters rank all five nominees in one category in the exact same order and the remaining 45% of voters rank all five nominees in the exact reverse order. The 55% majority block would elect their first choice as the first-place winner on the first count, with one other nominee winning 45%. In determining second place, the nominee who was listed second on all those ballots would win 55% against 45% for that same other nominee, even though the winner receiving 0% of first-choices in the vote for first place. In determining third place, the nominee who was listed third on all those ballots would win 55%, and so on.

    This verion of instant runoff voting can be used to elect five seats in a legislative race, but it is very “majoritarian.” What is typically done — Ireland’s parliament is a good example — is to use the “single transferable vote”, which provides proportional representation. See:
    http://en.wikipedia.org/wiki/Single_Transferable_Vote

  10. Jack, what I am saying is that the Hugo awards do in fact use a variation of STV. You can see this on the summary page at the official Hugos site, in the last paragraphs, headed “Lower placings”, or by inspecting the PDF of the full tallies and seeing how many ballots are *not* involved in determining second and later places in each race.

    It’s a variation because this is to produce ranked results, not 5 legislators or whatever, and because they don’t remove a fixed quota of ballots, but all first-place ballots.

  11. Claudia — Not to be too nerdy about it, but….I think the summary page at the Hugo sites would suggest you’re right, but the results suggest that description is inaccurate. If you really removed the votes for the winner, the number of of votes used to determine the second winner would be about half the original count number and so on. Instead, the number of ballots in each round are about the same.

    That combined by an exchange at another site suggests that all ballots are examined in every round of counting, starting each election after removing previous winners as options. The total number of ballots goes slightly down just because not everyone chooses to rank all the nominees. If everyone ranked every nominee, the number of ballots would stay constant — but Hugo gives people the option to abstain.

Comments are closed.