Voting for scenario quality

[Ssnake]

I wonder how we can create a bigger incentive for voting on scenario quality. I've briefly checked the Downloads section and saw that many scenarios are downloaded about 300 times, with just one or two people ever giving a vote for it. I wish we could boost that to ten, or twenty people per scenario. That would be a much better sample size and therefore a more useful indicator especially for new players to pick the scenarios suited best for them.

[Ssnake]

May I also point out that many votes are also an incentive for scenario designers to work on more scenarios, which in turn is good for you players as well.

[Skybird03]

We could have a "competition" on the board, held every three months. Everybody gives his opinion on what the three best scenarios were that were released in this time. We could have a special oldie competition "vote for the best five scenarios that are already in the database right now".

By that, scenario names would be mentioned, as well as the names of their creators. That way, they find an easier way into the awareness of the community. I personally, for example, have downloaded some scenarios at random only. I am even not aware of what other stuff is there right now, and I also do not know much about the creators.

A third possebility is to vote for the scenario one did not like, and explain the reasons of it, in a constuctive manner. It is not about bashing the author, but to give him feedback helping him to avoid flaws or mistakes if he ever dares to write another scenario.

Prices could be some kind of special avatars that are unavailable to the ordinary forum members. They could serve the purpose of virtual little Oscar statues. :D

Really, I like the idea. It is constructive, and it sounds like fun to me.

[RENEGADE-623]

That is a great idea. Would be nice if more people take the time to rate the scenerios that they downloaded. I think it would improve the quality of the scenerios tenfold. It would give the scenerio designers more incentive to actually make a quality scenerio rather than just throw one together (not saying that that is what they do now). I remember in the old forums you submit your scenerio and there was a thread dedicated to talking about each scenerio. Lets people throw their opinions and pointers, I know when I submitted my very first scenerio, was a scenerio called "Amberg", Lonestar, and I believe Hackworth gave me some pointers on what was good and what needed improvement on it so I knew the next time what to do. I am currently working on a new Bradley Table VIII Scenerio, yes i have picked it up since now I know it wasnt SB that was messed up was my puter, and am taking my time on it to make it as detailed and as accurate as I possibly can. I know that making scenerios is a long process as i been working on this now for two days and still havent got to the meat and potatoes of it yet LOL but would like input on what peole think of it. So i think that Ssnake is on point with what he said.

[daskal]

Back in the "good old" times when life was easier and I was obsessed with scripting & making single player maps for Black Hawk Down we had a pretty good rating system on the website.

Each map or scenario got a rating from the website admins and a separate one from the users/players. Leaving comments to each map was useful too, because it helped the scenario designer to get the invaluable feedback this kind of work requires. It was useful for the users as well because they could get a relative good picture about the scenario itself - wheter it is worth downloading & playing or not.

Here is a link - just to showcase this kind of method:

http://www.dfbarracks.com/mapdb/index.php?action=dm&mid=9840

[HotTom]

Amen, Nils!

I have no idea why more people don't rate scenarios. The number of stars has at least some meaning. The number of downloads doesn't mean anything because I have downloaded many missions I have tried and then thrown away.

As a scenario writer (of admittedly modest talents) I would appreciate even more some written feedback. I ask for it every time I put up a new scenario but get very little, good or bad.

It IS encouraging when I receive compliments. And it IS helpful when players criticize or question my mission design because I do think about what they say. Often they see things I don't.

Case in Point: Last week, RecceDG questioned a number of things I did in Golan Chapter 5. The whole Golan series is an experiment in trying to recreate a historical campaign, not just a single mission.

I really think THE major flaw in SB as a single player game is the lack of campaigns and I'm trying to figure out how to write them.

I don't have the programming ability and I don't think eSim will make the investment to create truly dynamic campaigns, which would be ideal, but I think good linear campaigns are possible.

Golan may be a bit too scripted for many players, so I will offer more tactical options in the next campaign (after I finish Golan, which still has two chapters remaining).

The response I sent to DG was longer than "War and Peace" but what he forced me to do was to write out (or articulate) the compromises I made (usually for frame rates or to avoid long periods when nothing happens) and explain why I made them.

My response was a very helpful exercise for me in learning what works and what doesn't. If DG hadn't asked, I wouldn't have gone through the mental exercise.

Short version: Please give both stars and feedback. If you aren't comfortable posting your comments on the forum, PM me.

I really want to design scenarios that players enjoy.

My only suggestion to help the process is for the writer (or mod builder) to start a new thread for each item they post. Some of the most talented members of our community have a single ongoing thread covering ALL of their creations. It's very inconvenient to follow when a half dozen topics are being discussed at the same time.

:MP:

HT

[Skybird03]

I see a problem with a system that only gives a number of stars (that's why do not care for such ratings). You can see it in many other forums. In most cases, such a system tends towards a mean value in the middle. It's the old statistical problem any academic knows about: a mean value wirthout at least giving standard deviation and value spreading is close to meaningless. It says nothing about how many people gave extrmeely good and extremely bad notes. When you have seven guys saying this one is 9 star scenario, while eight guys say it is a one star scenario, you end up with score of 4-5 stars.

Now how could one make sense of this...? I yet need to see a place with a purely numerical rating system where that made sense, I haven't seen such a thing so far, neither in the flightsim nor in the racing sim genre, or on customer feedback portals. So, if you want to stick with the stars, at least it should be something like Daskal said - verbal comments that pleeeeaaase do not only sing anthems of praise, but constructively comment on the points that caught the attraction of the commentator, or that scared him away. so that a ghuy strolling buy gets an impression of what it is that he is about to download.

Really, stars alone is as useful as a parachute when boarding a sub. It said close to nothing at best, and is misleading at worst.

[PeterGrozni]

I'd vote more regularly If the scenarios had a short description that tells you what it's about and a small (but not too small) picture of the scenario map, for example. When I download several scenarios at a time I have no idea which was which sometimes. I never remember mission names, so some visual and "context" aid would help me quite a bit.

[RecceDG]

You want to see some quality scenarios get written?

Hold a contest to get scenarios made "official", where they become part of the game distribution. And pay the scenario designers that get accepted.

DG

[daskal]

Oh almost forgot to mention: how about a "map of the month feature"?

[Hub]

You want to see some quality scenarios get written?

Hold a contest to get scenarios made "official", where they become part of the game distribution. And pay the scenario designers that get accepted.

DG

I agree.

[sabot_ready]

If you are going to rate using stars.... You will need to add more catagories to show what is good about it.

Realism ****
Originality **
I crapped my pants factor *****

[3Star]

That's what the 'comments' factor is for.

I've been trying to work my way through the Single Player scenarios, methodically rating each one. I always put a comment in the 'comments' section. I'm on the 'T's right now... However, I agree, more participation from others would be a good thing.

NTM

[TheReaper]

I've commented on scenarios, but haven't rated any, because the ratings seem rather vague and I haven't seen enough scenarios to distinguish an excellent one from a merely good one. If the ratings were clearly defined, and didn't require comparing one scenario to others, that might encourage more reviews and more consistency between reviews.

A few examples where categorization is simple even for newcomers:

Lonely Planet: $$$ is "reasonable; entrées in the $15-$25 range; bottom line per person: $25-$35 (plus taxes and gratuity)."

Michelin Guide: "3 stars means the restaurant merits a special trip. The food, wine, decor and service will be exceptional "

Library site: "3 stars means it was worth reading, but I wouldn't really recommend it"

Kelly Blue Book: Defines that 3-stars (fair) means "that the vehicle has some mechanical or cosmetic defects and needs servicing but is still in reasonable running condition. This vehicle has a clean title history, the paint, body and/or interior need work performed by a professional. The tires may need to be replaced. There may be some repairable rust damage."

[sabot_ready]

Well Thats interesting....3STAR which of the above would you apply to your self? You are after all 3 stars?:biggrin:

[Sean]

There has never been much feedback around here, going all the way back to the beginning of time. Heres a interesting stat: 67,043 total downloads, 830 ratings. Stars is all we got - if people wont be bothered to do that, what makes you think they will spend more time to do a more detailed write up??? I do like the idea of the awards and the special avatars for people that hold them. Maybe we could hold nominations for best sce, best skin, best ter map and then take those nominations and vote on them in polls.

[Homer]

There is no technical solution to this. It's impossible to define what is good and what is bad because we all have different perspectives. A scenario that you think is lousy is probably not the same opinion of the scenario's creator. The whole star rating thing is too subjective. If anything, we should turn off the rating system all together.

I dont think we can come up with anything better than what we got now, the ability to voice your opinion through the comments. No rating system can be more accurate than that.

No matter what we end up with, it has to be used for it to work...

[ShoutingDog]

No matter what rating system is developed, how easy it is to use or how accurate and fair it is it won't be used unless there is some incentive/reward for those rating scenarios to actually rate them.

I would suggest that certain scenarios designed by the acknowledged best be placed into an area only accesible by those downloaders who have actually taken the time to rate the scenarios they've downloaded.

Want access to the really good stuff then take a few moments to rate the stuff you download.

Once in place scenarios that get dropped into the elite area can be determined by the ratings.

Can this system be abused - of course. There's alwaysa way to rig a system no matter how well designed. But what's the alternative?

[DrDevice]

The alternative is convincing the community of the value of feedback. Without a genuine desire to provide feedback as a tool to get better scenarios, I don't think any rating system will work.

Getting access to "good" scenarios by rating others is self defeating if all the user has to do is arbitrarily rate X scenarios - without any proof that they've played them, or actually provided constructive criticism.

Homer is dead on. I don't think the reason why people don't provide feedback is a technical one. It's just not a community habit yet.

[Kingtiger]

Homer is dead on. I don't think the reason why people don't provide feedback is a technical one. It's just not a community habit yet.


I second that...

[ShoutingDog]

The alternative is convincing the community of the value of feedback.Self evident. How does one go about doing that without appealing to users' self interest.
Without a genuine desire to provide feedback as a tool to get better scenarios, I don't think any rating system will work.Again - self evident and I agree with you 100%.Getting access to "good" scenarios by rating others is self defeating if all the user has to do is arbitrarily rate X scenarios - without any proof that they've played them, or actually provided constructive criticism.Excellent point.
Homer is dead on. I don't think the reason why people don't provide feedback is a technical one. It's just not a community habit yet.And it never has been.

I've been coming here under various screen names since this forum's inception. This topic has come up off and on since way back in 2000 or so. Same topic, same comments, same solutions, different folks.

It'd be great to get fair and accurate scenario ratings - that may encourage more people to design quality scenarios (key word MAY). BUT! Even without a rating system those folks that enjoy the process, IMO, will make scenarios regardless. Those that do it well will be recognized by their fellows thru word of mouth, the popularity of their scenarios in online play (that group seems to focus on a few at a time and then moves on when something even better comes along) and private offline messages. The best will design fun, exciting and quality scenaris just because - even without a rating system.

One has to ask what real benefit would accrue were a rating system successfully implemented? The cost may very well outweigh the benefits. The SB editor is quite possibly the best scenario design tool ever made available to the players of any wargame ever published. Consequently just about anyone can toss together a scenario fairly easily. The consequences of that though are that there are a lot of scenarios made that - to be kind - are less than - well - adequate.

That doesn't necessarily mean a designer lacks talent and imagination. It may just mean he hasn't acquired the knowledge and experience yet to put together a quality scenario. An honest rating system in this case may very well discourage a potentially talented and prolific scenario designer. Do we really want that? How could we prevent that - use comments - but it's difficult enough now to get people to just go in and click on a number of stars - why should we believe that they'd take time to actually comment.

IMO - the rating system as it is currently implemented is fine. If users wish to use it, it is there for them to use. Current and past indicators are that it will not be used though and probably never will.

[3Star]

Perhaps the scenario owner can copy/paste comments from their thread on their scenario into the comments box in the download area?

Or otherwise put a link in the description to the thread on the subject. (You know the threads I mean to, the ones that say "Scenario X up"

NTM