Caedes

Non-art Website Issues

Discussion Board -> Non-art Website Issues -> Thinking about fairness of c-index...

Thinking about fairness of c-index...

NzglKing
12/13/05 4:40 PM GMT
I've been thinking about the fairness of the c-index system. according to the FAQs "The c-index is a number between zero and 100 that aims to give a rough idea of the quality of an image. The primary use of the c-index is to enable users to sort galleries in order to bring the best images to the top." Thinking about the system and how it works, i believe it actual may not be as fair as it should be. Allow me to explain: The c-index is based on the votes it gets (needing at least 5), views and downloads, correct? I find problems in the way these individual rankings work together to create an overall rating.
First-off, there is a lot of grey area in the area of voting. obviously a 10 is 'perfect' but what exactly is a perfect image? what does a 5 mean? one might accept it to mean that it has as many 'good' aspects as 'bad'? however, therein lies a problem. for one, there is a lot of it has to do with preferences. for example, if i prefer fractal images and another person prefers photography, we might give different ratings. i might prefer fractals because i mostly do fractals, as a result, i might know what i should be 'looking for' as a bias which to rate the image, while another person who isn't as familiar might not be basing his rating on the 'correct' priciples. to sum it up, preference of image 'genre' plays a large part. also, what the numbers 1-10 mean. does a 9 mean perfect except for maybe a little fuzzy/blurred section? is an 8 fuzzy and excess noise? i believe the fact that a number scale is used to grade, creates a canyon of grey area, an image that is an 8 to me might be a 10 to someone else.
I also find issues within using downloads and views as a factor. I may need to view the image to vote on it, but just because i think its a good image, doesn't mean i want to download it. maybe i never download images, i just rate them. however, i am hurting their rating even though i think it is good because i didn't download it. the only people who download the image are the people who might want to use it as a desktop background or rework it possibly. hbut not every person who look at my image wants it on their desktop. conclusively, using the 'ratio' of downloads versus views can ultimately be an image rating killer.

I propose several possible solutions:
1) users have a prefence profile. possible bases for prefernces: listing image genres from most favorite to 'least'; preference of image dynamics (ex: i like darker colored images over brighter colored images). these are just some examples of preference possiblilties.
2) 'weighted' impact of rating. if i am very skilled at making fractal images and normally upload them, then maybe my vote on an image should 'count more' than someone who is more involved in photography. this is simply for some above issues i pointed out (knowing what to look for, experience in the genre). at the moment i am not sure how that would work out because of the random voting booth and such, that would be something that the specifics would have to be discussed. but another benefit is though that the caedes member who have been here longer have more of an image than the newer members. currently with the way the system works, a new member has as much 'say' in the rating as the oldest, which is sometimes a problem.

people rate images sometimes because of quality. othe rtimes, because someone really likes the image, they give it a ten, just because they like it. as i quoted above: "The c-index...aims to give a rough idea of the quality of an image..." just because a bunch of people like an image and give it a ten, doesn't mean is a good 'quality' image.

there rests my case.
0∈ [?]
I am able to BE, to BEAR, to DO, or DARE, all things through Christ who strengthens me, every day of my life.

Comments

Post a Comment  -  Subscribe to this discussion
NzglKing
12/13/05 4:45 PM GMT
update:
additional idea: using a user's karma to affect the person's impact of their rating. higher karma means they comment and rate a lot of images, so they have to have been here longer and have more 'experience.' this works in conjunction with the speculation of impact of ratings by new members versus older members
0∈ [?]
I am able to BE, to BEAR, to DO, or DARE, all things through Christ who strengthens me, every day of my life.
*caedes
12/13/05 6:06 PM GMT
1. The c-index doesn't include views or downloads at all so that discussion is moot.

2. The preferences idea has been discussed, and if I added it I would only give a choice between the four or five major genres of art here. Anything more would introduce too much bias in the sample.

3. The votes are currently weighted, but instead of picking some outside metric I weight them based on the historical accuracy of the voters' votes (how close they were to the average vote for each image they have voted on). We also do this based on the genre, so people who are not good at voting on fractals don't have their vote counted highly.

4. Even though everyone has a different 'average' that they assign to images, this is not a problem. I can easily enough calculate the average for each voter and use that to make each voter's average the same, so they are all speaking the same language.
0∈ [?]
-caedes
.animaniactoo
12/13/05 7:09 PM GMT
Don't you hate it when all these new people come out of the woodwork w/suggestions & questions that have already been debated and decided?

Does it make you want to bang your head against a wall? *grin*
0∈ [?]
One man sees things and says "why?", but I dream things that never were and say "why not?"
.NzglKing
12/13/05 7:55 PM GMT
O....K lol i didn't know all that. i never found any of that information in the FRQ or anything. sorry about that.
0∈ [?]
I am able to BE, to BEAR, to DO, or DARE, all things through Christ who strengthens me, every day of my life.
.animaniactoo
12/13/05 8:03 PM GMT
(I'm a newbie who just posted an apparently already discussed issue too, I'm pokin fun @ me just as much here)
0∈ [?]
One man sees things and says "why?", but I dream things that never were and say "why not?"
.KEIFER
12/13/05 9:37 PM GMT
Hey Hey Hey .. none of that ... newbies are not allowed to talk amongst themselves

"Jailor" .. he barks .. "take them away .. to separate cells, this time .. and only half rations of gruel .. perhaps if they be famished, it'll quiet their wagging tongue"

0∈ [?]
wookin pa nub
.animaniactoo
12/13/05 9:41 PM GMT
*hangs head in shame*

I'm sorry, I'm just so bored waiting for my 300 mb file to save YET AGAIN. *sigh*

I'll behave now, I promise *crosses fingers behind back* 8•P
0∈ [?]
One man sees things and says "why?", but I dream things that never were and say "why not?"
*caedes
12/14/05 12:58 AM GMT

One way to think of the voting that may help you is to understand that the ultimate purpose of it is to help visitors to the site find images that they will like. One way we can do that is to give them a list of images that other people liked. We determine this by showing people images and asking "Do you like this?"

0∈ [?]
-caedes
.NzglKing
12/14/05 1:37 AM GMT
i think i understand now. sorry about my little 'outburst' (as you could maybe call it) it was something i kinda did say anything about for a while and finally spoke up. i didn't mean any slander to this site or how its run. i think this site and teh idea of pooling works of art and photography to get people all over of different hobbies, ages, cultures, etc. to tell each other how they did, how to do better, a giant supportive community. i love it. again, sorry if i offended any part of teh team or seemed like i was bitter towards the site.
0∈ [?]
I am able to BE, to BEAR, to DO, or DARE, all things through Christ who strengthens me, every day of my life.
::regmar
12/15/05 2:17 PM GMT
Don't be sorry, Kevin. All opinions are welcome here at Caedes except those that intentionally hurt others. Caedes answer to your question revealed something I didn't know about the rating system - that views / downloads are no longer factored into the c-index. Thanks for bringing it up.
0∈ [?]
ж Regmar ж

Leave a comment (registration required):

Subject: