Four Scores and 1.5 years Ago...(Registry Scoring Discussion Thread) 5 5

[ADAMANTIUM]

On 1/18/2023 at 12:54 PM, wytshus said:

Please see this post regarding my thoughts on scoring in the Registry:

 

After this Year's Awards, I plan on implementing major changes to the Registry structure.

Some of the changes I am planning are not going to be popular, but I feel they are necessary.

My intention is to make set scoring more dependent on completion percentage, set detail, and photographs, as opposed to having 2 or 3 keys/grails.

 

How are these changes to affect the over all score? Like if we have a bunch of sets, but only halfway completed half of the sets we create?

As has been stated in the thread, some sets have tons of books! Some collectors although its "not the census", want books in the registry cause they may be of same character and type. Rather than a custom set or creating a bunch of sets, I like to obtain "keys to me." How will the over all picture look?

It was mentioned TMNT that all but number 1 obtained in 9.8 and they're ranked #15-18. How in the world will a completion percentage affect them to still win an award? I'm not familiar enough but I'm guessing that #1 in 9.8 is just that many points? Yet imo I haven't checked, but with TMNT especially and there being a LOT of completionists in that set, assumed, I'm not sure the "fix is in?"

Edited January 19, 2023 by ADAMANTIUM

[wytshus]

On 1/19/2023 at 4:29 PM, ADAMANTIUM said:

How are these changes to affect the over all score? Like if we have a bunch of sets, but only halfway completed half of the sets we create?

 

Not sure yet, my original thought was to set the 9.0 Universal score at 4 for all comics, then let the algorithm determine the weighted score. 

But that would make base scores for 7.5 and below  - 1. 

This isn't optimal, and it would also make the base score for 9.8 - 32.

 

What I am sure about is that registry scores should not be based on open market value.  How to get away from that is a very complicated question.

 

 

[ADAMANTIUM]

On 1/19/2023 at 4:09 PM, wytshus said:

Not sure yet, my original thought was to set the 9.0 Universal score at 4 for all comics, then let the algorithm determine the weighted score. 

But that would make base scores for 7.5 and below  - 1. 

This isn't optimal, and it would also make the base score for 9.8 - 32.

 

What I am sure about is that registry scores should not be based on open market value.  How to get away from that is a very complicated question.

 

 

I was remembering some specific issues where the scores are out of whack, but not knowing all the importance of certain issues and there scores, I'm wondering if certain books value will change?

will they change or is it just the way all books are scored might be tallied different, ie completion, pics, description?

There are a few books I'd be interested in purchasing, but I saved them for last as they're worth a "lot of points."

It's time when I need to purchase them, but I'm still hesitant if I should get them before others because it may look like point chasing? Even though I own other similar or in the run, because the points for the books themselves might change?

If you lowered everything to the "9.0 mark", as you said, can you give us an example of a book and the affect on it?

[wytshus]

Keep in mind that points in the registry are for entertainment purposes only.

They do not, and will never, reflect market value.

 

I want to get away from points being the determinate factor in top sets. If I had my way, sets would be ranked by:

 

Set Completion Percentage

Percentage of comics with images

Percentage of comics with descriptions

(BTW, this is how we determine Award Winners)

How to accomplish this goal, without major coding changes, remains a significant challenge.  I'm not sure that it can be done in the Registry's current configuration.

But, a boy can dream....

 

I can't give specific examples at this time, and I may have to scrap the whole thing eventually.

 

 

 

 

 

 

[Yorick]

It would really mess with people if a book's score factored the census numbers.  You would see a Hulk 181 drop to the bottom while another non-first-appearance Hulk issue with no submissions rocket to the top. 

That would be cool.  I like scarce books.

[Off Panel]

On 1/20/2023 at 11:36 AM, Yorick said:

It would really mess with people if a book's score factored the census numbers.  You would see a Hulk 181 drop to the bottom while another non-first-appearance Hulk issue with no submissions rocket to the top. 

That would be true if scarcity in the census were the ONLY factor used to assign points. By the same token, if scarcity in the census were the only factor used to price books, Hulk 181 would probably be worth about $25 dollars (and Amazing Spider-man 300 would come free with your Happy Meal).

We can assign "value" to a book using either points or dollars, but I think the basic criteria in either case would be the same: Supply (number on the census; number in grade; number of a particular price variant, double cover, pedigree, etc) and Demand (desire for the book; historical significance of the book; aesthetics of the book; desire for a particular price variant, double cover, pedigree, etc.).

People argue that the price of a book shouldn't determine the points for a book, and I agree, but I don't think that's what is happening. Rather, I think the same things that make a book worth a lot of dollars also makes it worth a lot of points and vice versa.

I would love to see census numbers factored into the registry score calculations because I'm a run collector and I end up buying high grade copies of a lot of non-key books that "no one cares about." A lot of times the prices are crazy because there are less than ten copies in grade on the census. In that case, you don't need a thousand people clamoring for the book to drive the price up; you only need eleven people.

The two biggest problems I see with using census numbers in figuring points for a book are:

1. I don't really want to manage my points eroding over time as census numbers for each book slowly creep up, and
2. We all know the census numbers are only an approximation of reality at this point.
   • How many 9.6s have been cracked, pressed, and submitted and are now 9.8s? (Or CGC had a bad day and now they are 7.5s?) Either way, they are ALSO 9.6s in the census right now.
   • How many 9.8s were destroyed in Hurricane Katrina? Zero? One thousand? The world will never know...

Having said that, if CGC found a way to incorporate census scarcity into the points algorithm (maybe as a percentage of the average CGC Census book population), I'd be in favor of it. However, this would have to be one factor of the equation; not the only factor.

[Gotham Kid]

On 1/22/2023 at 5:51 PM, Off Panel said:

 if scarcity in the census were the only factor used to price books, Hulk 181 would probably be worth about $25 dollars (and Amazing Spider-man 300 would come free with your Happy Meal).

 

[Gotham Kid]

On 1/22/2023 at 5:51 PM, Off Panel said:

People argue that the price of a book shouldn't determine the points for a book, and I agree, but I don't think that's what is happening. Rather, I think the same things that make a book worth a lot of dollars also makes it worth a lot of points and vice versa.

[Off Panel]

On 1/20/2023 at 11:25 AM, wytshus said:

I want to get away from points being the determinate factor in top sets. If I had my way, sets would be ranked by:

 

Set Completion Percentage

Percentage of comics with images

Percentage of comics with descriptions

(BTW, this is how we determine Award Winners)

 

I think there is a simple solution to your problem: 

There can be two different types of "top sets."

There can be a top point-scoring set, which is just a factual list of 'this member has these books in these conditions and they are worth this many points.'

There can also be a "best set" which speaks to the care the member has taken in cataloging and presenting a set (% of comics with images, % of comics with descriptions, related journal entries -- go wild).

'Set completion percentage' would be a factor in either type of set.

Does the top point-scoring set have to be the Best set? Nope. Does the Best set have to be the top point-scoring set? Nope.

As a member who is unlikely to find myself in either position, I would be interested in viewing BOTH types of set. I like to know who is ahead of me in points and what books they've got and in what conditions. (I'm just competitive enough that it will push me to acquire that next book.) However, if someone has a well-curated set with pictures, recollections, and reflections, I will read the spoon out of that. (I love Thomas' You look MAR-VELL-ous set and I would love it even if it didn't score the most points.)

To a degree, you've already got this dynamic in place with the Awards, but you need to let go of the idea that only points can determine the best set. (That thinking causes you to keep futzing with the point system until the "best" set "wins." At some point a user with fewer points is going to create a set where every book has its own haiku and it will be so beautiful that it will make the boards collectively weep. When that happens, don't rewrite the points system again to add "% of haikus"; just announce that member PointsGoblin has the Top-Scoring set and member Haiku4U has the Best Set. EasyPeasy.)

 

 

[Off Panel]

On 1/20/2023 at 11:25 AM, wytshus said:

How to accomplish this goal, without major coding changes, remains a significant challenge.  I'm not sure that it can be done in the Registry's current configuration.

 

I can't give specific examples at this time, and I may have to scrap the whole thing eventually.

There is a change in thinking that needs to happen at CGC (and CGC isn't alone -- A LOT of companies need to make this change).

Stop approaching problems from the perspective of "how can we make this less work?" and start approaching them from the perspective of "what do our users really want?"

Every single problem CGC has right now is some variation of cutting corners: Quality Control issues, grading inconsistencies, PR disasters, etc. Every instance started with somebody at CGC saying "How can I take a shortcut here?" 

First, figure out what the people who pay for your services want most and then think about how you can get it to them. You have, in these boards, a resource that most companies would kill for. Yes, we are tremendous pains in the butt -- no argument there -- but every day we communicate clearly what you're doing well as a company and what you're messing up. For a company that listens, that is absolute gold!

Once you have a list of what your clients want (not what the suits want, not what the influencers want, not what some sketchy company will pay you to do -- what your clients want), create a roadmap to get there. It may take a few iterations. That's okay. That's how successful companies build things. (The first iPhone couldn't even copy and paste.)

If you don't have the skills to get there, hire the people with the skills. If you need to license a data feed, license a data feed. If you have a bunch of employees that don't want to change or do the work, set them free to follow their dreams at the DMV. 

You may say, "But it's not as easy as that."

It isn't easy at all. Go back and start reading from the top. The theme here is "stop trying to take the easy way."

Ironically, once you start working to do the right thing, rather than the expedient thing, it eventually does start getting really easy. Because you are fixing problems. Because your quality control isn't a crisis and your grading isn't a crisis and your PR isn't a crisis. Look how much time you have now to create something cool and new and amazing.

None of this is targeted at you, Wytshus. This is an unfortunate theme I see at CGC, which is a good company, but several attitude adjustments away from being a great one.

 

Edited January 22, 2023 by Off Panel
More specificity.

[Yorick]

@Off Panel You're not a consultant for CGC?

I too pay top dollar (aka: stupid money) for non-key books in order to complete CGC sets.   I'm now feeling a little sheep-ish.

[Off Panel]

On 1/22/2023 at 3:04 PM, Yorick said:

@Off Panel You're not a consultant for CGC?

 

Howdy, @Yorick. I'm what is known as an "unpaid consultant." (In other words, a busybody.)

On 1/22/2023 at 3:04 PM, Yorick said:

I too pay top dollar (aka: stupid money) for non-key books in order to complete CGC sets.   I'm now feeling a little sheep-ish.

I feel your pain, buddy. I truly do. Still, nothing beats the kick of seeing a nice run of high-grade books in a set with no open slots...

[chaddesden33]

On 1/20/2023 at 4:25 PM, wytshus said:

Keep in mind that points in the registry are for entertainment purposes only.

They do not, and will never, reflect market value.

 

I want to get away from points being the determinate factor in top sets. If I had my way, sets would be ranked by:

 

Set Completion Percentage

Percentage of comics with images

Percentage of comics with descriptions

(BTW, this is how we determine Award Winners)

How to accomplish this goal, without major coding changes, remains a significant challenge.  I'm not sure that it can be done in the Registry's current configuration.

But, a boy can dream....

 

I can't give specific examples at this time, and I may have to scrap the whole thing eventually.

 

 

 

 

 

 

Brian  - this doesn't seem terribly optimistic considering this topic started over three months ago and the This Score Needs Fixin' thread was locked down last September.

Many good and thoughtful ideas have been put forward since then so could you let us know if you have any timescales in mind to complete this review and institute a new Registry scoring system?

[wytshus]

On 1/27/2023 at 12:26 PM, chaddesden33 said:

Brian  - this doesn't seem terribly optimistic considering this topic started over three months ago and the This Score Needs Fixin' thread was locked down last September.

Many good and thoughtful ideas have been put forward since then so could you let us know if you have any timescales in mind to complete this review and institute a new Registry scoring system?

With the new company wide ERP migration , there will be no coding changes to Collector's Society.

We are waiting for approval to make scoring changes twice a year, as opposed to doing it on the fly.

The scoring adjustment period will probably be April and October. I will make an announcement on the boards once it is approved.

[chaddesden33]

Thanks Brian, much appreciated.

Look forward to further announcements as and when you have them, including criteria for making biannual adjustments to scores and what level of participation will be allowed for collectors.

[Rosland]

If we're part of the problem (by requesting scoring changes) we can also be part of the solution (by providing justification in detail)

[Steppenwolfscomics]

On 1/27/2023 at 1:48 PM, wytshus said:

With the new company wide ERP migration , there will be no coding changes to Collector's Society.

We are waiting for approval to make scoring changes twice a year, as opposed to doing it on the fly.

The scoring adjustment period will probably be April and October. I will make an announcement on the boards once it is approved.

I think this is a good idea. Hopefully the process of adjusting scores isn’t too arduous. 

I think for most people to be happy the scores don’t have to be dead-on the market, but rather somewhere close to reasonable. 

On a personal level, I have 5 Golden Age sets and I’d like to see the scores adjusted throughout those sets. I don’t mind when a $1k book gets 600 points but it’s bothersome when a book costs hundreds of dollars and gets 10 points or costs thousands of dollars and gets 80 or 90 points. 

Come April, if I’m allowed to request a revision of points to my sets then I’m happy 👍 I suspect there are others in the same boat. 

[Crushalot]

Registry points should absolutely be assigned based upon market value. It is the only fair way. It doesn't have to be dollar for dollar so maybe 60% of 3 year average price to keep a big buffer for market gyrations.

The purpose of a dollar is to be a universal unit of exchange. This translates to almost every part of our daily existence and this should extend to the registry.

However, I do also heartily support points being given to sets on top of the individual comics for % completed, % with pictures and % with descriptions.

That is how you do it

[Sauce Dog]

On 2/5/2023 at 3:56 PM, Crushalot said:

Registry points should absolutely be assigned based upon market value.

Hard no. This would result in either requiring a paid API to another price guide for data (which brings a host of issues, including but not limited to; questionable accuracy if the source mixes up variants/newstand values, outright lack of data points for things like GA books, allowing people to have direct access to gaming the registry values via fraud on books with little to no sale history, ), or require so much constant maintenance by a limited team of CGC employees to update that it becomes unfeasible.

It makes more sense to just have a standard point structure (which can differ across each era of comics, so Golden Age books feel more 'valuable' to obtain compared to any Modern age book), then introduce bonuses such as; completion % points, image bonus, and perhaps introduce a KEY BOOK bonus which is assigned to some of the most obvious books out there that have great significance (such as Hulk 181, AF15, a slew of DC GA first appearances) and makes those books stand out from others (this is where the community can suggest new books that deserve the tag & key bonus, rather than us constantly suggesting registry point corrections every week)

 

[wytshus]

As I mentioned earlier, we are going to adjust individual scores twice a year.  

Adjustments will be done in April and October.  I will create an official announcement on the boards, and the main website soon.

I will be using Overstreet to determine scores for individual books.  Keep in mind that the census algorithm will still weigh scores like before. 

ONLY UNIVERSAL SCORES WILL BE ADJUSTED.

I do not have access to the algorithm, or have any control over weighted scoring.

 

I will fulfill the existing requests in the old thread first, and create a new thread in April for new requests.