Sunday, January 8, 2017

Testing Dice for Fairness: Part II

The power of the fair die fair test.

Previously a method was outlined for testing a die for fairness.

The test calls for rolling the die a large number of times, computing a p-value, and classifying the die as unfair if the p-value is below a significance threshold.

The example uses 100 trials and a significance threshold of .05. It doesn't mention why these numbers are chosen.

The Two Types of Error

To intelligently choose the numbers, we have to consider two types of mistakes. We might reject a fair die, or we might accept an unfair die.

The chance of making the first error—rejecting a fair die—is our significance threshold. To reduce the chance of making the first error we make the significance threshold low. A threshold of .05 is a bit high; our test has a 1 in 20 chance of rejecting a fair die!

The Power of a Test

The power of a test is the chance of not making the second error.  That is, the power is the chance of rejecting an unfair die.

The power of a test usually depends on how unfair the die is. The more lopsided the chances, the easier it is to detect the unfairness.

An Approximation

Because of the dependence on the amount of unfairness, we can't represent the power as a single percentage. It is natural to characterize an unfair die as a categorical distribution with n - 1 free parameters where n is the number of faces. Even in the case of a d6 we can't plot the power as a function of the free parameters: there are too many.

For this reason let's only consider unfair dice characterized by a single parameter which is the probability of rolling the most likely face. All other faces are assumed to be equally likely.

The Power of the .05 Significance Test

Now we can plot the power of a test, given the probability of the mostly likely face.
If we increase the number of trials, the chance of detecting an unfair die goes up. Even if we only do 100 trials, the chance of detecting an unfair die where one of the faces is twice as likely as expected is about 90%.

I would like the percentage to be higher, but when I think the about the effort of making 200 trials instead, the percentage seems acceptable.

The code used for calculating the power of a test is online. It uses the Monte Carlo method as discussed previously. The number of Monte Carlo trials was 10,000 which gives a 95% confidence our estimate is within 1%.

The Power of the .01 Significance Test

What if we decide to use a .01 significance level instead?
The chart doesn't change much. The chance of detecting the die where one of the faces is twice as likely as expected is now 77%.

Sunday, January 1, 2017

When Did RPG Become Collectible

Three events point to 1991 as the year your RPG crap became collectible.

Yes, there were a few people collecting before 1991.  I don't mean players, of which there were millions. I mean people buying stuff when it was still in print and leaving it in the original shrink wrap. People who maybe had uncommon foresight. Or maybe a type of compulsive hoarding disorder.

Ignoring such people, 1991 was the year the collecting started. I couldn't say what it was about 1991, exactly. The Mentzer box sets, also known as the BECMI rules, went out of print.  Their passing made some people wistful. They introduced a lot of people to the game.

My argument that 1991 was the year the collecting started rests on three pieces of evidence.

Exhibit 1: Heroic Worlds

Lawrence Shick finished Heroic Worlds in 1991. It lists nearly every RPG item published before 1991! It presents over 3000 items organized by genre.

Heroic Worlds told collectors what was out there to be collected. Here is what it says about the original set:
230-010.1/900-74. Digest-sized, boxed: three digest-sized books ("Men & Magic," 36 pp.; "Monsters & Treasure," 40 pp.; "The Underworld & Wilderness Adventures," 36 pp.), six reference sheets of tables and charts. Tactical Studies Rules (TSR), 1974 (1st ed., brown wood-grain box, white sticker cover). 1st and 2nd pr. have drawing of mounted warrior on "Men & Magic" and box cover. 2nd pr. says so inside "Men and Magic."
230-010.2/900-76. As above, but all-white box. "Men and Magic" has fighter with sword on cover. Early printings refer to the J.R.R. Tolkien creatures "hobbits" and "ents"; after legal hassles, these were changed in later printings to "halflings" and "treants." Tactical Studies Rules (TSR), 1976.
The description identifies at least four printings of the original edition. Collectors care about printings.

Today we know there were in fact three printings of the wood-grain box and three printings of the white box.

The original set isn't the oldest item in Heroic Worlds. Here is the description for Chainmail:
Medieval miniatures rules. The 2nd and 3rd eds. contain the "Fantasy Supplement" section that directly contributed to the development of D&D. The supplement includes rules for individual wizard, hero, and super-hero characters, plus hobbits, dwarves, elves, goblins, orcs, trolls, dragons, elementals, balrogs, and ents. Wizards can throw fireballs, lightning bolts, and eight other types of spells; hero-types can use magic weapons.
230-005.1/379-71. Digest-sized, 36 pp. Guidon Games, 1971 (no Fantasy Supplement).
230-005.2/379-72. Digest-sized, 44 pp. Guidon Games, 1972 (2nd ed.).
230-005.3/900-75. Digest-sized, ring-bound, 44 pp. TSR, 1975 (3rd ed.).
The description makes a case for the historical importance of Chainmail. Collectors care about historical imporance.

Schick must not have had access to the 1st printing of Chainmail, since it does in fact contain the Fantasy Supplement!

Next let's look at the description of Deities & Demigods:
Supplement describing deities of 17 different mythoi, with rules on how to incorporate them into AD&D. Unfortunately, the book is usually used merely as a sort of Monster Manual that describes very high-powered monsters. This usage is encouraged by the book's format, which emphasizes the gods' physical abilities over their religious significance. A descendent of the Original D&D supplement Gods, Demigods, and Heroes, the book covers 12 pantheons of gods from myth and folklore (Egyptian, Norse, etc.), plus gods for various AD&D nonhumans, and four fictional groups: Arthurian heroes, Fritz Leiber's "Newhon Mthos," Michael Moorcock's "Melnibonéan Mythos," and H.P. Lovecraft's "Cthulhu Mythos," (The latter two mythoi were removed from the 2nd ed., thus giving the 1st ed. a high collector's value.) The 3rd ed. was repackaged and retitled Legends and Lore (see listing).
004-228.1/900-80. Hardcover, 144 pp. TSR, 1980.
004-228.2/900-81. Hardcover, 128 pp. TSR, 1981.
Ignore the part about gods having stats blocks. That's Schick telling you how to play the game. Focus on the comment about the 1st edition. That's Schick telling you what to collect.

Another cue for collectors is found in the description of B3: Palace of the Silver Princess, where Schick alerts us to the existence of the recalled edition:
Scenario for character levels 1–3 describing a ruined palace and the evil creatures that have taken it over. The 1st ed. is similar to B1 in that monster and treasure listings are left open for the GM to decide. The 1st ed. was released, then quickly recalled and destroyed due to several illustrations that were deemed to be in poor taste. (Look for the captured princess tied up with her own hair!) There may be fewer than 300 copies in existence; it is highly collectible. The 2nd ed. was entirely rewritten and largely reillustrated. It is a more standard dungeon, without fill-in encounters.
230-204.1/900-81. 32 pp., outer folder. TSR, 1981 (1st ed., tan border on cover).
230-204.2/900-81. 32 pp., outer folder. TSR, 1981 (2nd ed., dark green border on cover).

Exhibit 2: Gen Con Auctions

Consider this advertisement for Gen Con 91 from the June 1991 Dragon:

It mentions the opportunity to "bid on treasures in games and art auctions". In 1991 Gen Con started catering to collectors.

In truth, I'm not sure when auctions started at Gen Con. The ad from 1991 is the first mention I've found. There were auctions at Gen Con 92 as well; they became a regular feature.

Exhibit 3: Zocchi's TSR Collector Treasures Sale

The last exhibit contains the full page ads Lou Zocchi took out in the Dragon to sell used TSR products. Zocchi placed 10 of these ads between May 1991 and December 1996. Here is the ad from October 1991:

A PDF of all of the ads is available here.

Like the Gen Con auctions, the Zocchi ads cater to collectors. Collectors are mentioned explicitly in the title of most of the ads!

I take prices which are several times the original list price as evidence of collector interest. There are a few such prices in the original ad: the "Chainmail Rules" for \$30, a "Fine 1st Printing White Box D&D" for \$80, and "Deities/Demigods 1st Ed. damaged" for \$50.

If we compare ads we see items appreciate. In October 1993 the "D&D White Box Set" is \$110 and the "Deities & Demigods, 1st Ed." is \$130. The first issue of Dragon becomes available for a record price of \$230. That's \$384.16 in 2016 dollars!

The final ad in December 1996 has an even more expensive item: a "B3: Palace of Silver Princess (Brown cover ed.)" for \$250.

By the way, are Zocchi and Schick both color blind? The recalled edition of B3 had an orange cover.

Appendix A: The Dragon Shuts Down

After the December 1996 issue came out, the Dragon shut down. TSR had gone bankrupt. When the Dragon resumed in August 1997 it was owned by Wizards of the Coast.

Zocchi had been a monthly advertiser in the Dragon for years. The TSR collector treasures sale was just one of the angles he was working. Most months his ads were for Gamescience dice or Zocchi's own RPG products. He owned the rights to some Judges Guild titles and he sold a game designed by Jeff Dee called TWERPS.

For whatever reason, Zocchi didn't place any ads in the Dragon when it got back on its feet.

For collectors it was just as well. They were discovering AuctionWeb, which changed its name to eBay in September 1997.


I've since found Gen Con advertisements in the Dragon mentioning auctions before 1991.