Set Scores 3 3

[Magmar]

Anybody else think its wrong that  someone with a single 9.4 first edition TNMT  has a score of 41000   (worth $30K-$40K)

and someone who has collected 93% of the first edition in 9.8 has a score of 14000  ?    (set worth currently $50K - $60K)

 

I am not a fan of putting first edition first print worth 120000 points  and the rest of the set worth a grand total of far less than that

 

 

 

[Figment]

I think one way to reward members for set completion would be to have a "Completion Bonus" score.  The bonus score would be the simple calculation of (Score x %Complete).  Example:

Rank   Owner     Set Type   Set Name       Score   %Complete   Complete Bonus   Total Score

4      Figment   MSH        Many Stories   9300    8           744              10044

In this manner a set that was 100% complete would have a total score twice their normal score.  The calculation of the normal score would remain the same using whatever points are assigned to the issue grades.   In this manner quality continues to be awarded for grade, but set completion would also be factored in when determining rank.

Thoughts?

[Icculus308win]

A bonus score of some sort is neat and good idea imho

[Sauce Dog]

I am fully on board for having some sort of % multiplier or completion bonus (a set 100% bonus would be easier to implement I imagine, perhaps the bonus is a set amount or just equal to the total number of books in the set - so 1 point extra each), as I'm seeing a good chunk of top spots in sets taken by only a single comic (albeit one at super high grade) with no other entries in it.

Of course I would also want some sort of Pedigree bonus for issues as well, especially after all this excitement for the Promise collection. A 9.8 Pedigree should be worth slightly more than a 9.8 universal.

[Icculus308win]

…..kind of what they’ve done with SS books, getting a bonus

[Qalyar]

Playing devil's advocate here, completion bonuses would be... erm, challenging to maintain for some of the broadly inclusive categories, and would make adding oddball books a lot more contentious than it already is.

[Sauce Dog]

19 hours ago, Qalyar said:

Playing devil's advocate here, completion bonuses would be... erm, challenging to maintain for some of the broadly inclusive categories, and would make adding oddball books a lot more contentious than it already is.

Nobody is forcing anyone to buy random filler books though, and it would still be possible to hold top registry positions without a full set - I just think having MORE point paths available to collectors to compete for top spots the better. Maybe some like to collect small amounts of high grade issues from a run, maybe others only want pedigree books, and others might like to aim for collecting them all but in lower grade. Could be much more fun strategizing the best way to compete and not always just boil down to who has the single highest graded key book from the run and nothing else.

Edited July 9, 2021 by Sauce Dog

[Figment]

The completion bonus would be simple.  setScore x completionPercent = completionBonus

The total score would be equally as simple. setScore + completionBonus = totalScore

It would have nothing to do with the grade scores of any individual book.  It would have no bearing on what books are included in what sets.

It would go a LONG way towards encouraging people to build more complete sets, and it would help alleviate the very real frustration that Magmar addresses in the initial post in the thread.  A single FF48 in 9.8 would be sufficient capture the top spot in the Galactus Master Set (and probably a few others) so I will again echo the sentiment that a single issue does not a set make.  And yes, I have an entry in that set so I like the person currently in the #1 position would be quite frustrated by someone with one of those issues reading this and taking that top spot with that single issue.

If the maintainers of this registry truly wish to make the playing field a bit more competitive they'll do something, even if it isn't this.  Just my two cents.

[ThatNerdyCanuck]

On 8/11/2021 at 3:06 PM, Figment said:

The completion bonus would be simple.  setScore x completionPercent = completionBonus

The total score would be equally as simple. setScore + completionBonus = totalScore

It would have nothing to do with the grade scores of any individual book.  It would have no bearing on what books are included in what sets.

It would go a LONG way towards encouraging people to build more complete sets, and it would help alleviate the very real frustration that Magmar addresses in the initial post in the thread.  A single FF48 in 9.8 would be sufficient capture the top spot in the Galactus Master Set (and probably a few others) so I will again echo the sentiment that a single issue does not a set make.  And yes, I have an entry in that set so I like the person currently in the #1 position would be quite frustrated by someone with one of those issues reading this and taking that top spot with that single issue.

If the maintainers of this registry truly wish to make the playing field a bit more competitive they'll do something, even if it isn't this.  Just my two cents.

I think this is a great idea. It is pretty frustrating seeing some scores where someone has one or two high grade books and they're at the top of a list with hardly owning any issues in a set!

[wytshus]

Great points made by everyone in this thread.  I agree that 1 book should not shoot someone to the top.  I assume the original poster is referring to TMNT 47?

I will bring this up with the powers that be at our next meeting.

However, @Qalyar is right,  we also have to consider things from a database perspective.  A completion bonus is a great idea, in theory.  Implementation is another story altogether....

I would appreciate more input on this topic, so I can bring a strong case to my boss.

 

Thanks!

 

-Brian

[Qalyar]

On 8/19/2021 at 10:34 AM, wytshus said:

Great points made by everyone in this thread.  I agree that 1 book should not shoot someone to the top.  I assume the original poster is referring to TMNT 47?

I will bring this up with the powers that be at our next meeting.

However, @Qalyar is right,  we also have to consider things from a database perspective.  A completion bonus is a great idea, in theory.  Implementation is another story altogether....

I would appreciate more input on this topic, so I can bring a strong case to my boss.

 

Thanks!

 

-Brian

Okay, let's try to take this seriously with an eye toward implementation. If there are going to be set completion points added to sets, then we have several big questions, and a few edge issues to address. Warning: post is long.

I think it's obvious that any completion bonus has to be determined algorithmically. It cannot be a manually-determined value because there are eleventy zillion sets, and asking Brian to manually create a set value for everyone one of them will cause him to come murder us in our sleep. So that gives us, broadly, two options:

• Base the score entirely off the books you have and your completion percentage.
• Base the score off your completion percentage and the total potential value of the set, in some manner.

Let's look at a set where a single book dominates. TMNT is probably the better example here, but that set is huge and I'm lazy. Let's look instead at Albedo. That's not a perfect example, not the least of which is because not everything that ought to be in that set has even been graded (the 1st series of Albedo goes to #14), but it is a good example of many of the obstacles this process will face. It's a set with a few mid-value books, several books with minimal-to-zero FMV, and one of the biggest modern keys.

For the sake of argument, let's assume our aspiring completionist gets Albedo #14 slabbed (it hasn't been yet) and gets that one and the rest of the first series added to the set (the others exist but aren't in the set yet). When it matters, we'll also assume that Albedo #9-14 all have the same minimal point value that #8 does. That would give us a set with 16 slots (#1-14, #0, and the #1 2nd printing). Currently, all printings of #0 share the same slot. For the moment, we'll retain that because it's important to discuss in the context of some implementations.

• A collector who accomplishes the Herculean task of owning 9.8 Universal graded copies of almost every book in the set (including the #0 1st printing, because that's the highest scoring option in the slot) but completely lacks any copy of the first appearance of Usagi Yojimbo in #2 has a set worth 1400 points. Without any modifiers, a single copy of Albedo #2, even in a 9.4, beats that entire collection.
• If we multiply effective scores by (1 + completion percentage), then the nearly-complete set is almost doubled (1 + (15/16) = 1.9375). Its new score is 2712.5. A set that consists solely of Albedo #2 is multiplied by 1.0625 (for its 1/16th completion ratio). But the score differential means not much changes. The nearly-full set now beats a lone copy of #2 in 9.4, but still loses out to a lone copy of #2 in 9.6, with a modified set value of 2921.875.

What if the set was larger? Technically, the Albedo set could include the later series also, if anyone cared enough to collect and encapsulate them. The 2nd series has 10 issues. The 3rd series has 4. There are 2 issues each in the 4th and 5th series, plus the Albedo Color Special that reprints part of the original #1. That's 19 additional books. We'll score them all like Albedo #8. The set is now 35 books long.

• Unmodified, our near-completionist now has 2312 points. His 34-book collection beats out a 9.4 copy of Albedo #2, and loses to one in 9.6.
• Multiplied by completion percentages, that rockets him up to 4557. Finally, just barely, he squeaks past a 9.8 copy of Albedo #2 (which, counting the 1/35th bonus, is now worth 4525). Although, playing devil's advocate here, one of the 3 9.8 Signature Series copies of #2 still blows past the other 34 books combined.

What does this mean? It means when a single book dominates a long run in terms of market value, it takes a lot of other books to catch up, even if you're spotting them a completion bonus. The bigger the gap between a single dominant book and the field, the more books it will take to "catch up". Sometimes -- perhaps often -- there's no way for the filler issues to catch up at all. I said I wasn't going to look at TMNT, but... let's look at Teenage Mutant Ninja Turtles (1st series). I'm still too lazy to sum up the 9.8U point value of all the books, but let's look at Icculus308win's ridiculously amazing complete 9.8 set. Now, some of those non-#1 books are Signature Series, so just using his scores actually makes things look a little better for the run than it would be otherwise. Close enough.

• His entire set is worth 141317 points. But a whopping 112000 of those are the grail-tier 9.8 #1 first printing, leaving a mere 29,317 for the rest of the books. You see where this is going. I believe I count 90 books in the set. A collector with 89/90 in... whatever version of 9.8 that set has, but who owns no copy of the #1 first printing, would see their point total rise to 58308 with a set completion bonus, which still loses (badly) to a single key issue in 9.6 or higher. When one and only one book is worth that much more than the rest of the series, there's no realistic way for the run to compete.

More complex options are theoretically possible. We could derive some value for the total set. Let's say, for sake of argument, 10% of the maximum Universal score possible. That's the equivalent to having the full run of a series in 9.2 based on the way scores work. This is probably possible to do from a database standpoint depending on how CGC built the registry to begin with. In this hypothetical, instead of multiplying your completion percentage by your own score, you multiply it by that derived value, and add it to your books' individual scores.

• Going back to Albedo, the full set in 10.0 is 10875 points, so the completion bonus in this case would be (percentage * 1087). Our original scenario, where we only care about the 1st series, still has the #2-less set in 9.8 worth 1400 base points. But now, it gets (1087 * 1.9375 = 2106) bonus points, for a new score of 3506. A lone copy of Albedo #2 earns 68 bonus points, which doesn't really affect much. You now need a 9.8 copy for the single key to beat the rest of the set combined. But for longer sets, like a hypothetical Albedo (Complete), or the 1st series TMNT, a run collector can actually pass the one-off key book.
• But is this possible? Remember I said I'd mention that Albedo #0 entry? That's not the only case where there's a single slot that can be filled with multiple different books. Depending on the database structure for the registry, it may not be easy to select the highest point value option there.

Basically, how to compute the set completion bonus, if implemented, depends in a large part on how aggressively CGC and the community want to incentivize run collecting over key issue collecting. Plus data structure limitations.

Regardless of how a set bonus is implemented, there will be extra work for the Registry Staff (hi again, Brian!). Suddenly, point values become more important, not just for the top-dollar books, but for the supporting issues in the run. Right now, as I understand, score updates are a purely manual process in which a CGC employee must attempt to evaluate FMV based on recent confirmed sales in order to generate a point score that is intended to (but doesn't always) represent a fixed-ish point-to-dollar ratio. How "filler" books are scored in the Registry has varied over time. Is the 10.0 reference score for a book with essentially null FMV 90? 75? 60? 45? In some cases, this will make a lot of difference, so that score adjustment thread would suddenly be the happening place to be.

• Personally, I'd actually suggest discarding the idea that points are supposed to have a dollar-equivalent value. Create something like 8 to 10 tiers of "value" for books, reassign everything to a tier based on its existing value (which can be done algorithmically) and then books probably only need adjustment if something substantial changes. This would, however, cause a great deal of drama and is more or less outside the scope of this discussion.

Finally, creating any sort of completion bonus would essentially mandate more stringent definitions on what is or is not included in a set.

• The big challenge is going to be the conflict between title sets and character sets when those look like the same thing. For example, the character Spawn is the star of the comic titled Spawn. The registry set called "Spawn" is a title set; it includes exclusively the books in that series (and doesn't include variants, because of the category type). There is also a registry set called "Spawn (Complete with Variants)", but that set is a character set. It does not just include all variant printings of Spawn, it includes all the printings of all the books in which Spawn appears. There is no set -- and I bet such a request would currently be denied -- that includes only the title Spawn, but does include variants; nor is there a set for all books in which Spawn appears (whether or not they are Spawn) but not variant covers and printings thereof. Whew!
• Also, pre-first-appearances. Hoo boy. For example, the Spawn (Complete with Variants) set includes Malibu Sun #13, Malibu's house advertising periodical, presumably because of the Spawn cover appearance that predates Spawn. Is that different than, say, Robotech II: the Sentinels #13 or Torg #2, which are even earlier than Malibu Sun #13 and feature a two-page centerfold advertisement for Spawn that includes a more-than-full-page image of Spawn? What about Entertainment This Month #31, which mailed the same month as Malibu Sun #13 and also has a Spawn cover?
• As a somewhat lesser concern, there are "non-competitive" slots in quite a few sets. Some of those are grandfathered TPB slots, but there are also a few oddities like Wizard Authentic Editions that are no longer considered to be competitive set elements. Based on my educated guesses about the database structure here, there's no (easy) way to automatically flag those, and they would still be a "slot" for the purposes of slot counting, which suddenly makes them no longer "non-competitive". The current policy is also to have foreign republications shunted to their own sets, but there's a lot of those grandfathered in as well. Sticking with the theme, Spawn (Complete with Variants) includes the 1996 Japanese edition and the Les Chroniques de Spawn #27 variant. A policy would need to be set for the handling of all these entries (and likely removal from sets).

[wytshus]

So, we have our bi weekly registry meeting this Thursday.

Thank you @Qalyarfor your input, I need to read it at least a few more times to fully digest it.  And I promise I would never kill anyone! 

(not in their sleep, at least.....)

I have been thinking about this all weekend, and I have a couple ideas, not sure how the higher ups would feel about them, but I wanted to run it past you guys first.

1.  "Simply" take the ridiculously high scored issues out of competition.

Probably the most simple solution, not sure how to implement it though:  I noticed a few of the vintage keys have been removed from the competitive sets, and I suspect its for this exact scenario.  We would have to come up with a max cut off, then just run a query to pull all of the books that exceed it.  (I am kinda curious about how many books in the registry are over 5000 anyways.) 

2. Establish a maximum score across the board.

This would be more complex, and I'm not sure about implementation at all.  I actually like this one the best.  These keys are worth what ever someone is willing to pay for them.  It really doesn't matter what CGC scores them.  If someone really wants a particular issue, they will buy it, regardless.

 

If anyone else has any other ideas, let me know asap.  Even if you just wanna nit pick our ideas, that's fine.  I'm sure my bosses will pic plenty of nits when i bring this up!

 

-Brian

 

 

 

[Qalyar]

So, ideally, I would have gotten back to this before Thursday. Sorry about that, Brian.

1. I for one don't like the idea of excluding books because they're too valuable. Fundamentally, the Registry encourages run collecting, and sharing collections with the community. Those goals aren't well served by excluding books because they become too rare or valuable (and what a nightmare that would be (a book has a record sale at Heritage and now it falls out of competitive sets? yuck).
2. Of these, this is my preferred solution, but see below.

What I think is probably the best solution would require a lot more work. In short, stop trying to match point values directly to market prices. Create some arbitrary number of "point levels", each of which represents an approximate tier of value. The top level is open-ended at the top in terms of FMV value (maybe benchmarked at roughly $10k+, I don't know). Condense all existing score options into those preset point levels. This can probably be done algorithmically if the rules are well-defined.

You'll still need the "score needs adjustment" thread, but it shouldn't have movement quite as often because books have to cross tier bands to get values changed. Maybe require that their FMV shifts to some percentage of the way through the band before score updates, to prevent a comic wobbling around the breakpoint from needing frequent adjustment.

[wytshus]

Great points @Qalyar I really appreciate your understanding of the registry, and your willingness to share your thoughts.

Most of what you said above was brought up at our meeting yesterday.  My boss wants more examples of grails/keys throwing the set scores out of whack.  I got a lot on my plate right now, and I am in the process of relocating, so it probably wont be until late next week at the earliest.

There is an algorithm that flattens the curve on the coin side(NGC), it was explained to me thusly:

A $1,000 coin gets 1707 points, then a $10,000 coin gets 5,158 points, then a million-dollar coin gets 24,570 points. So in the registry, someone with about 5 $10k coins has as many points as the guy who just has the million-dollar coin. And the person that has 14 $1,000 coins has the same score as the other two.

We also discussed what, exactly, makes these issues "pop" like they do, and if the value holds over time.  Personally, I have a hard time wrapping my head around scoring a book for thousands of dollars when the cover price is $3....o_O

But, that's just me. I understand the vintage stuff, they're rare, and finding a 70 year old book in near mint condition is next to impossible.  It's the modern stuff that boggles my mind.  I understand if an artist passes away, or if a movie renews interest in a particular series, but these pops seem to happen a long time after the book is published.  It's story or cover is known at the time, and other than the scenarios above, I don't get how the value increases exponentially for no apparent reason....

[Qalyar]

On 8/27/2021 at 2:41 PM, wytshus said:

My boss wants more examples of grails/keys throwing the set scores out of whack

Let's see what I can do here:

• Albedo. There are actually a couple other rare books in this run (#0, #1), but FMV for #2 is so crazy this set is basically just a referendum on that book.
• Incredible Hulk (1968) #102-#474. So, here's a good example of what REALLY expensive books can do to even a long set, even one with other pricey issues. The first place collection is 29% complete. In the #2 slot is a collection that's a whopping 63% complete. Is the #2 set lower grade across the board? Not really. But the current top of the category has IH181 in 9.8 for 50000 points, while the set with over twice as many slabs "only" has a 9.6 copy, which drops its score all the way down to 31250. Dozen more books worth 50 (or less) points even in 9.8 don't make up the difference.
• Marvel Spotlight (1971). This is a tough run to assemble, with a couple big books. #2 isn't cheap, but hardly matters, because the first appearance of Ghost Rider in #5 is still nearly half the potential score. A 9.6 copy of #5 plus a 9.4 #2 is good enough for 6th place at the moment, over some much more comprehensive collections.
• Marvel Super Heroes Secret Wars. Short sets with a random key really highlight the problem. A 9.8 copy of #8 is more points than the rest of the set in 9.8. It doesn't help that there's an unusual number of copies of the high-dollar book in 10.0...
• TMNT (Volume 1). It doesn't look as crazy here because most of the people who own #1 1st printings in high grade also own a ton of the other books in this long run, but its point value is miles higher than anything else in the set.

To some extent, any long series of filler with one or more expensive books is weighted very heavily toward the keys. Danger Girl (Complete) is probably a good example. A solo 9.8 copy of #2 Ruby Red Variant would "only" be good enough for 11th place right now, but it's outweighs big stacks of the lesser books. To an appropriate degree? I'm uncertain. In any case, this is also a good example of the random values of fill books with minimal FMV outside of the inherent slab costs. Are those sorts of books worth 90 in 10.0? 75? 60? 45? Werewolf By Night is probably another set to look at in this regard, with #32 dominating value even over several other significant books.

Sometimes its the scale that matters. In contrast to the sets above, Superman's Pal Jimmy Olsen has a big key, #134 -- the first appearance of Darkseid. But the filler isn't filler score-level filler, and catching up to a 3-4000 point book 400 points at a time is far more realistic than catching up to one 50 points at a time.

[wytshus]

Good stuff @Qalyar, thank you!

I'm still trying to find my footing in this job, and your knowledge is really helping me understand what exactly is going on.

I am familiar with IH181, first appearance of Wolverine. Pretty much defines the term "grail".

$50,000 though?  To me, that is just nuts!  

I am going to forward your entire post to my boss.  I agree that scores shouldn't be tied to value, but they are.  Revamping the entire scoring system isn't an option, and implementing a completion modifier would require far more resources than we have.  I am leaning heavily towards capping the scores, I think this would be the most fair, and "easiest" solution (although implementation would still be problematic).  I will let you all know what my boss says about this.

In the meantime, more input, please!  

[Figment]

Regardless of what is decided or implemented, thank you to the admins and the board for considering the idea, and thanks all for their input.

I do think the simpler the algorithm, the simpler it will be to understand and accept.  No solution will satisfy everyone, and complexity in this case may cause confusion.  I do think that points should roughly reflect value, but I also think that a way of the ignoring temporary spikes of "hot, then not" may need to be considered.  I'm also not sure I like the idea of capping the scores.  I admit to having lots of books that hold relatively low point scores and might benefit if somebody else's were capped, but I also have some of those "grails" and would most likely cry foul if someone tried to have them clipped.

The fact that #1's and other key issues are priced, and have point values, beyond what seems reasonable or realistic is just the way it is.  We all have to live with the supply and demand aspect of the hobby.  

From a technical perspective it strikes me as relatively easy to add the additional columns and calculations, but I fully admit I have no clue as to the implementation details or server environments behind it all.

Another idea would be to simply say that a set must be at least x% complete before being considered for registry award ranking.  I'm not sure what value is appropriate for "x", but something that would generally eliminate sets of "one or two".

[dragonmgmt]

On 9/17/2021 at 3:37 PM, Figment said:

Regardless of what is decided or implemented, thank you to the admins and the board for considering the idea, and thanks all for their input.

I do think the simpler the algorithm, the simpler it will be to understand and accept.  No solution will satisfy everyone, and complexity in this case may cause confusion.  I do think that points should roughly reflect value, but I also think that a way of the ignoring temporary spikes of "hot, then not" may need to be considered.  I'm also not sure I like the idea of capping the scores.  I admit to having lots of books that hold relatively low point scores and might benefit if somebody else's were capped, but I also have some of those "grails" and would most likely cry foul if someone tried to have them clipped.

The fact that #1's and other key issues are priced, and have point values, beyond what seems reasonable or realistic is just the way it is.  We all have to live with the supply and demand aspect of the hobby.  

From a technical perspective it strikes me as relatively easy to add the additional columns and calculations, but I fully admit I have no clue as to the implementation details or server environments behind it all.

Another idea would be to simply say that a set must be at least x% complete before being considered for registry award ranking.  I'm not sure what value is appropriate for "x", but something that would generally eliminate sets of "one or two".

Figment's idea of having set be x% complete before it is considered for registry award ranking, seems to me, to eliminate almost all of the abovementioned issues of points for keys and fillers.

I would humbly nominate this as the best solution thus far.

[wytshus]

On 9/17/2021 at 3:37 PM, Figment said:

 I do think that points should roughly reflect value, but I also think that a way of the ignoring temporary spikes of "hot, then not" may need to be considered.

This is turning out to be the most challenging part of this job.  What I was told to do is go by Overstreet, if it's not in there, then we average the sales from the past 3 or so years.  I think this is the most fair way to score the books that "pop".  I don't think it's fair to adjust the score for a comic that had one big sale.  It could very well be a money laundering scam.  I do as much research as I can, but sometimes there isn't a logical explanation for why a $4 comic is suddenly worth $10k +

I will follow up with my boss this Thursday, and I will let you all know what was said.

 

Thanks!

[Figment]

On 9/20/2021 at 12:59 PM, wytshus said:

I don't think it's fair to adjust the score for a comic that had one big sale.

This is a tough one also, especially for the GA books which don't have many sales in general, and even fewer that get recorded.  Part of the problem is that GPA and GoCollect automatically update everything they're aware of in what is fairly close to real time, while Overstreet editors must manually try to do the same in a once yearly cycle.  I don't envy you (or the Overstreet guys) the task, this is a job for the new Amazon superhero called Analytics-Man.