Rating groups an the quality of their work

[littleoni]

I have tried to find a thread about this issue but I have found none, if there is one feel free to inform me and I will take this down immediately.
With the immense ammount of new translators joining the ranks in the last year more or less, there has been a huge decrease in quality. Also the current system encourages being the first and the fastest, and completely disregards being the best at something. At the point where we are at, I'm surprised if I'm to read a chapter in a work that has two or three chapters translated and the language looks something like proper English, the pixels are not bigger than the average chicken, the font used is not Arial, etc.
I have no tools to inform other readers, and other potential translators of this. If I were to start translating (and with this I refer to the whole process: translating, cleaning, redrawing, typesetting, etc) I would surely try to "fix" some of the series I actually liked but were first butchered by someone without the skills or the knowledge to do a proper job. But I can also understand that if a translator sees a series is being translated already, they would prefer to work on something else.
I think it's only fair to be able to rate someone's work as a consumer. And it may be worth to someone to be able to avoid suffering through machine translation.
If I were to do it, I would probably allow rating the group's work on a per series per position work, since they may have done a good job with some of their works, but thrashed others; and it's probably not everything horrible or perfect about their work.
I don't know the exact effort needed to implement such a system. I know it's not easy at least, but I truly think of it as necessary. I have sometimes pretty much given up on trying to find something to read due to everything being of such low quality.
This would also stop some of the BS "wars" going on between translation groups because "someone takes over the work entitled to others" because they were faster or something: faster groups would have their impatient audience, better groups would have their picky audience.

 

[ikemen]

Very relevant. Would like to add that we would benefit from having dedicated manga quality reviewer as a role. Similar to what they have at rottentomatoes or metacritic: one part of page is for normal user reviews, the other is for the actual critics. Clicking on any reviewer should show their past reviews.
There also should be rating charts of groups sorted by their overall quality.

 

[Tikibo]

Imagine the drama that would happen when both of the groups think that they do an awesome job but one is rated lower than the other. It would be on par with a comments rating system. Even if this seems like a good idea it would degenerate into a mess. Fake ratings would be a normal occurrence, and using Bayesian correction would make it unusable as the current manga rating system in which no matter how trashy manga is, it shows together with mediocre and above average ones.

BTW if you don't pay the group, you are not a consumer, but a recipient. So, that argument is invalid. And if you donate to a group with low quality... well, it's your problem.

Currently commenting about the bad quality of translation under chapter or even in a manga comment section is good enough to inform others about the quality of scanlation.

But yeah, I think it would be useful if done correctly.

 

[ikemen]

BTW if you don't pay the group, you are not a consumer, but a recipient. So, that argument is invalid. And if you donate to a group with low quality... well, it's your problem.

So, by the same logic, since mangadex is not a paid service, nobody here is a consumer and therefore, asking for its improvement(which is done in this forum) is verboten? Time and attention isn't free, you know. Out of those groups who don't ask for donations, would there be anyone scanlating if they knew their work would never be seen by anyone? Really doubt it. Therefore, they do it for the people. I know, I do.

 

[Zephyrus]

Staff has discussed this but we haven't come to any kind of conclusion. Most like the idea. If implemented correctly, it could encourage competition among groups and have them stride towards faster and higher quality releases.

However, the system would need to be implemented in such a way that users cannot abuse the system and downvote a group for fun or for reasons that don't involve the quality of the release.

 

[Kendama]

@Zephyrus

I'm just out of a long discussion on the matter with fellow translators in Death Toll. Personally, I think if a system like that is implemented, all hell will break loose. Speedscanners ragequitting, people changing nicks frequently, mutual accusations etc etc.

Let me also point out that, for translation, this isn't a new idea. Mangahelpers, a site that still exists if mostly forgotten, has a translator ranking system, but it's curated, meaning that TLors attain their ranks not by reader scoring, but because someone took a look at their output and judged its quality. The result? Most TLors are listed as "unranked" because TLors have better things to do with their hobby time than assessing their fellows' work.

Basically, the vast majority of readers are unqualified to pass judgement on the quality of translator's work. A TLor can write the darndest things in a way that makes sense and if the English looks good (mostly the job of the proofreader), only other translators who bother to check the originals will know what's wrong. A person in our discussion brought up a recent example of a translation that was in good, even poetic English, but was a gross deviation of the original.

As I said, I'm against it, but if you really insist in creating such a system, you'd have to rank the graders, too.

i. Give all MDex members who are also scanlators the option of self-identifying as one of the main scanlating roles: TLor, CLer, RDer, TSer, PRer, and indicate which group they belong to. If they are leaders, the self-appointed role is immediately validated. If not, the leader is pinged to counter-validate the role of the member. Multiple roles should be possible.

ii. Then give each chapter four parameters for grading: TSing, TL, quality of the translated text (not the same as TL, since it reflects the role of the PRer, too) and picture quality (CL and RD).

iii. That being done, grading should be weighed. A TLor vote on TL should count MUCH more than a non-TLor; ditto for a TSer on TS and so on. Give the peasants a minute contribution, perhaps 1/100 of a scanlator's per vote.

Voting should be per release, not series.

Then again, I still think all hell would break loose. Or not, to be fair the system has to be rigorous. And being rigorous, most people won't bother voting. But as I see it, an equal-weight, one-score per chapter system would be uninformative, abuse-prone and drama-ridden.

Better not do it. Just saying.

 

[QSS]

This is a dumb idea. No one's going to rate anything accurately. Readers don't know shit about whether a scanlation of anything is good (see: those who defend really awful releases) and translators (because editors can't truly judge whether a scanlation is good) are a bunch of egoists (on a worse level than your average editor) who would rate a series 0/10 if they saw even one typo. The only way to have somewhat consistent and reliable rating would be to have a mdex controlled panel of judges. And clearly mdex staff have no expertise or any idea wtf they're doing (see: the fact that they considered this) and no skilled scanlator actually has the time to do nothing but judge other people's TLs, so as said before, there will be no good ratings.

Then, on top of that, assume the scanlations ratings are actually accurate somehow. No one's not gonna read something because it has low ratings. No one's gonna rescanlate anything because it has low ratings. And no one's not gonna speedscan something because it has high ratings. So what's the point?

One more thing. You'd have to be some combination of legally blind, legally stupid, and legally deaf to click on a chapter of a series and not immediately realize when the quality is legitimately awful.

 

[Duralumin]

Don't see a way this system could possibly be implemented and result in a reliable score. The online audience writ large has no standards of quality and abuse would be rampant.

 

[DrWhoCares]

I fought this good fight with Potion Loli. It's a hard fight tho when you don't have the manpower or the motivation to work on horrible series.

EDIT: oh yeah I never actually addressed the topic though. I too think it's a pretty awful idea. Because it can't actually work. Readers' opinions are awful.

 

[motokare]

oof, all my fellow yaoi tl that are rated unfairly here (those ninja 1 rating from some weird homophobic sjw) and whatever that's not 'pc' enough for this site are going to be rated unfairly... i can just use the recent one shot i released as example... thr amount of '1' rating lol and the cesspool comments...

i throw my vote in to say this is a bad idea. the vast majority of readers aren't going to appreciate. maybe you limit this to only scanlator rank

 

[Trodsen]

if I had a suggestion to this, I'd say the bad english tag like the one on e-hentai/exhentai sounds like a much better idea, a scanlation's quality has nothing to do with it's release speed, and as it has been mentioned above, only someone with proper japanese knowledge would be able to judge if a translation is truly depicting what's in the original work or not. Being able to mark chapters with blatant machine TL or incredibly bad grammar would probably work a lot better than a rating system. And of course that shouldn't include translations that missed a few typos or something like that.

 

[Aoitenshi]

It actually brings to the question of 'how' would you rate a scanlator. Manga is subjective, so its rating reflects whether or not it is well-liked by everyone. Are we going to use the same 'subjectivity' to rate the scanlators, too?

- Speed?
The line between 'good' and 'bad' speed would be very hard to draw. For up-to-date releases, how many days would a group need to release the latest chapter before the get marked as 'slow'? For others, how many chapters a day would be deemed 'good'? What if one group works on 10 different projects and releases each series alternately every 10 or maybe 20 days?

- Translation quality?
There are two ways to approach this in general: the way the target language is written, and whether or not it's accurate to the source language. Let's use a JP > EN manga as an example. Only those well-verse in Japanese would be able to check if the translation is accurate. Furthermore, some may favor localization and others may say otherwise. On the other hand, perhaps those with English knowledge, or perhaps native would be suited to check for the English translation (the grammar and such).

- Redraw & Typeset quality?
At least this one is easier to handle, I guess. Although when I think more about it, will distracting TL notes and watermarks become a valid reason to reduce their ratings? And, would helpful sfx translations and end notes (like how EHS is doing Golden Kamuy) contribute to better ratings?

I'm really worried if these ratings will get abused and groups start falling apart because of it.

 

[Teasday]

Even if we implemented something it would most likely not be actually rating-based.

We have like buttons for groups, but their visibility and accessibility are so bad I completely forgot they exist. Should probably do something about that.

 

[Pika]

Wouldn't it actually shoot down the new and still rising scanlators? I mean not everyone starts out as a good translator/editor/proofreader/....
I mean this system might discourage newbies more than the comments

 

[DANDAN_THE_DANDAN]

Oh, I got an idea.

With Disqus, there's this "thank you" button (I forgot the name).

Why not just have an upvote system with no downvotes? Then we can have it so that the group will have their average number of "thank you"s per chapter shown on the manga page.

1. Everyone, from readers to staffs to fellow scanlators, can press the "thank you" button at each chapter.

2. Impossible to abuse, because the only way you can send a hate speech is by not pressing the button.

3. Better scanlators will receive more "thank you" on average when compared to worst ones

4. A long-running and popular series would also get more "thank you" because it has more fans. It will also motivate the scanlators to keep going, if they for some reason hesitate to continue.

5. Unpopular series wouldn't go to downvote hell since there's no downvote system

6. If a series has two competing scanlators, it should be easier for the newcomers to decide on which group to choose reading, based on the number of "thank you"s they receive for that series per chapter.

7. Snipers are similar. If they snipe a bad scanlator with better quality, they will get lots of "thank you"s. If they're a bad quality sniper who snipes a good quality, they won't get as much.

8. Dramas should be limited. YouTube comment system also has a dislike button that does absolutely nothing and I see no drama from YouTube comments
(except for the times when someone copypasted a popular comment and the copypasted blew up with likes while the original is lost in the sea of comments)

9. Since this is an upvote ranking based per series and not per group, if a popular scanlator group decided to release a bad series with good translations, it won't impact on their overall score and only the series score.

I can't draw or edit images and upload them so here's a very vague illustration

[Manga image]

[Manga descriptions, etc]

[Actions panel]

Scanlator team 1: 20.1 Thx per chapter
(Total chapters released = 10)

Scanlator team 2: 16.5 Thx per chapter
(Total chapters released = 9)

Scanlator team 3: 2 Thx per chapter
(Total chapters released = 1)

[Chapters stuff]


Oh, and in case a scanlator doesn't have a group then, by default, it goes like this:

No group; Username123: 5 Thx per chapter
(Total chapters released = 2)

 

[Retro68]

Actually, I have a suggestion since I agree that a "ratings" system would be abused. I believe that this system would not only benefit the reader as it should show what to expect, but benefit the scanalation groups as well. I suggest adding perhaps a "poll" system either per manga, or per chapter. The following could be an example of the questions to be asked

How was the spelling/grammar for this chapter?
o Good
o Needs Work
Please describe what needed improvement ___________________________________________________________________________

The results could be shown below or beside the poll from previous votes or vevn via another link, thus showing people what they can potentially expect to see, as well as providing (hopefully) constructive criticism to the respective person involved. I believe that it would be less able to be abused versus a straight up ratings setup, plus it would incorporate info for each section (TS, RD, etc) of the chapters processing. Anywho, thats jusy my 2c worth, good luck.

P.S. The reason I suggest this on a chapter to chapter basis rather than by manga is for the situation where you have more than one group uploading chapters in the series.

 

[Bainhardt]

Um.... I don't think there's any point to having "reviews" for scanlators or individual scanlation projects (be they groups of chapters, one shots, or entire manga series).

First, someone who goes to read a manga will decide pretty quickly for themself if they can tolerate however low quality it may be. I've read some real awful ones before, usually because there was nothing else available (like pre-_anq Anjou-san... those were dark days). Are these reviews supposed to provide advance notice that something is going to be a rough read? How much advance notice does someone really need when looking at 2 or 3 pages in a matter of seconds will let them decide the quality?

Second, are these reviews or ratings supposed to encourage other groups to pick up the slack? Is it supposed to serve as some kind of metric by which to judge how much "attention" a manga needs? If one is being handled only by very unskilled scanlators, are the reviews supposed to call in attention from more skilled groups so they can save the day? And conversely, are good reviews supposed to basically ward off anyone else from coming in and picking up a project? "It's already being taken care of, so don't bother." I feel immediately that this simply wouldn't work; scanlation groups work on whatever manga they are interested in doing, and that's really the end of it. Also, I'm not even going to get into the fact that there could be two separate groups putting out virtually identical-quality scanlations (just look at Nagatoro, lol).

Third, I think comments serve all the functionality these reviews could possibly cover and way more as they exist now. I already occasionally receive comments on my works along the lines of "I like the scanlation quality" or discussions about possible translation mistakes. These are active and flexible interactions because of the nature of what comments are. Reviews would be one-sided in addition to all of the many ways they could be abused that others have already discussed.

Finally, I will always think any effort spent on implementing a review system would be better spent on education instead. Rather than making a tool for people to use to belittle others' hard work that most people are doing totally for free, I would prefer people consider instead making tutorials and guides on how to improve the quality of scanlation work. I learned what little I am capable of currently from people who were gracious enough to teach me, and I'll always be thankful that I didn't have to dive in head-first with no idea of what passes as "good work." Perhaps a lot of these people you'd be criticizing into the ground just need some information and guided practice in skills they can use to meet higher standards of quality?

 

[ununseti]

_{@Retro68 said:}

How was the spelling/grammar for this chapter?
o Good
o Needs Work
Please describe what needed improvement ___________________________________________________________________________

Some variation of this could be a cool idea. Sometimes I spot a typo/other minor mistakes, but it's not a really big deal so I wouldn't want to make a comment drawing attention to it.

Alternatively, maybe there could be a link someone on the chapter that links to the release post for that particular chapter (for the groups who have their own websites), and feedback could be commented there instead, since it'll be more discreet / out of the way that way. This way, the scanlators are more likely to actually see the feedback.

@DANDAN_THE_DANDAN I think if you add any kind of voting mechanism to chapters, lots of readers will mistake it as a vote on the chapter content itself, not the scanlation quality. (Chapter voting might be a interesting idea in its own right... though chapter comment count already kinda serves this purpose. e.g. scroll down to the bottom of this manga (NSFW) and see if you can find the popular chapter ( ͡° ͜ʖ ͡°))

Twitter also has a "no downvote" philosophy: You can only like, retweet, or comment on a Tweet. This gives rise to things like "the ratio". Could be interesting.

As @Teasday mentioned, group likes (and also group follows) are already a thing. Maybe a solution could be to just make these stats more visible on the manga pages so that more people vote on the group. Even just showing the groups at the top of the manga page more prominently could increase click-through rates for the group pages themselves, where the voting/following buttons are. It could be cool to have a high-level overview of how many chapters each group contributed to a certain manga. It could be sort of like the contribution breakdowns for GitHub repositories (example).

Though these group like counts etc. will be skewed depending on which manga the group chooses to work on. Obviously, groups that do popular manga will get more attention.

 

[DANDAN_THE_DANDAN]

@ununseti Actually, my proposal is more like a "do you like it?" counter. Ultimately, it doesn't matter if the readers click "thank you" for the chapter or for the translation because, at the end of the day, a good translation means a good chapter and vice versa; same holds for the bad counterparts only this time the effect is much less Thxs. In the case of a good chapter with bad translations, it will reflect on less Thxs so the counter is still useful.

And like you said, this serves to be a "ratio". I believe that the Thank You Counter (I made this up on the spot) will be most useful in series with two competing groups scanlating, which solves the initial problem. This is also a form of "reader's gratitude".

@Retro68 The problem with your system is that it's too rigid.

-The options are basically upvotes and downvotes, except they don't vote for anything but rather give a static result.
-"Needs work" is an umbrella term which, in most cases, is sufficient but I believe that it is not in this case. Why?
-It takes too much work for the readers to point out. "Needs work" can be: a typo, multiple typos, wrong grammar, or even utterly garbage. In order to describe these things, the readers must type it out, which most are too lazy to do. If they absolutely care about the series, they would point it out in the comment section which already exists.
-Plus, it takes two clicks to submit your vote which some would just ignore since they're lazy.
-It can also be abused because haters can send hate messages directly to the translators through the feedback system.
-MD staffs would also have to implement an entirely different system for translators to view their feedbacks in private.

 

[littleoni]

BTW if you don't pay the group, you are not a consumer, but a recipient. So, that argument is invalid. And if you donate to a group with low quality... well, it's your problem

I won't go too deep in why this is completely wrong, but it is. I'm not giving direct monetary compensation, but I'm providing the group with a series of privileges they would not have if they didn't publish their work, and that makes me a consumer of the work even if it was given for free. Also donating does never make me a consumer by the definition of the word donation. Also making any kind of money out of scanlation is illegal (I'm not against people making money out of it, but money should never be mentioned, for when it is there comes the remuneration, liscenses, taxes and other problems of the real world)

Now after the innecessary, angry, hateful rant I'm quite tired of repeating, to the issue at hand.
A bad job at either translating or presenting the work hurts the work and while it technically increases aviability (more low quality tranlsations is very technically better than fewer higher quality ones), it prevents someone with the actual skills for the job to pick it up.
The idea proposed by @DANDAN_THE_DANDAN is decent, but it does not solve the issue at all. The point would be to punish groups doing a horrible job. Now come and scream at me "what right does anyone have to punish someone doing voluntary work!". Well, everyone does. If you sign up as a volunteer for disaster relief, for example, but then you fuck it up you can be sued. Upon taking wolunteer work you take the responsibility of doing it right, if you are not willing to even try to do it right, then don't do it and save yourself some face.
I have no idea of a good way of implementing the rating system. While it's true that the average Joe can't objectively judge the quality of the work so their oppinion will be biased, a majority of biased bad oppinions should say something to the ones doing the job. It's also true that trolling and self rating and whatnot would be an issue. Maybe some system like that of Stack Overflow? Something that rates the voter's vote based on their previous votes (a voter with many one-sided votes will have less weight, a voter who usually disagrees with the majority will have less weight. I don't know the exact algorythm). Or maybe limiting the voting power of the reder per period of time, so you only vote when you really like or dislike something. I don't know.
Then again, a system of this nature is not actually needed, we can survive without it. But what we are doing right now is a) promoting fast releases disregarding quality completely and b) discouraging actally dedicated people who usually try to do their very best when scanlating from doing it.