Review Guidelines

By Phil
Dec 2, 2005


Introduction: Reviewing History & Philosophy

NOTE: Please read before posting reviews. Ignorance of our reviewing policies will not be considered an excuse if you receive a warning or a ban.

General Guidelines

The first thing to always keep in mind is that reviews serve a double purpose. They are written for both the scenario designer and for the site visitor who is considering downloading the file. As such:

  • Reviews need to praise the designer for things that are done well and point out areas where the designer can improve.
  • Reviews also need to provide enough information about the scenario so that potential downloaders will know if the scenario matches their interests. Obviously, don’t give away the plot or reveal secrets that should be discovered while playing, but let the downloader know what the scenario is about, if it’s mainly fighting, or mainly rpg, or mainly puzzles or a mix of everything.
  • Whatever the case, a downloader should know what to expect from a scenario after reading a review.

Try as hard as possible to avoid vague statements in reviews.  Make sure that your review answers more questions than it raises.  Don’t ever say something like “The first part of the 2nd scenario was good” or “the part with the wolf could be improved” without providing further explanation.  Try to always include an example from the scenario to back up any points that you make.  If you are pointing out something to the designer that you feel could be improved, try to provide some ideas that the author could build on.  Do as much as you can to help the designer improve his work.

All scenarios have good aspects and bad aspects.  Try to always say at least one good thing about any scenario you review and never, NEVER insult a designer.  They might not be as good of a designer as you but even if you are handing out a score of 1.0, you still should never insult the designer.  Be honest about the scenario but make every effort to encourage the designer to do better next time. We cannot stress this point enough; the purpose of the review system is to encourage quality creations from designers, not discourage people from designing altogether.

Always spell check your reviews.  I’ve seen far too many reviews that take points off a rating because of poor spelling yet the review itself is full of errors.  Don’t embarrass yourself – spell check your work.

Lastly, the review should contain a short explanation of why you scored each category the way you did.   This does not need to be lengthy, sometimes a sentence is enough but other times, a paragraph for each category is needed.

Category Scoring

Before we detail what we expect for each category, there are some general scoring guidelines to make note of.  All of the categories are subjective, some more than others, but try to be as consistent as you can with your own scoring.  Also, take special note of a few things that should NOT affect the score of a scenario.  These things should be noted in the review, but they should not affect the rating scores in any way. 

  • First is the length of a scenario, or how many scenarios are included in a campaign.  There is no rule that says a scenario must last more than 15 minutes or that a campaign must include at least 3 scenarios.  The scores should only reflect how good the scenario was while it was being played.  A great 5 minutes should score much higher than a mediocre 2 hours.  No reduction in score should be made based on the length of a scenario.
  • Lastly, a scenario should not be penalized for not including special extras like music files or custom AI files.  These extra items are great if used effectively and certainly can boost a score but a scenario should not be rated poorly just because of a lack of extras.  A scenario should still be able to achieve a score of 5.0 even without using special extra files.   The RoL scenario design tool is so rich with extras already that a designer should not be required to use custom files if they can achieve their design goals using what is already built into the design tool.

So, with that said, on to the categories. Note that the Categories for Movies are slightly different.

Playability

Playability is probably the most subjective element of the scoring.  It is simply a gauge of how much fun you had playing this particular scenario.  One thing to look out for when reviewing is to only play scenarios that use a style you enjoy.  For example, if you hate playing RPG scenarios, don’t try to review one since you are bound to not enjoy the scenario.  Try to keep within styles that you enjoy.

There really is no specific criteria on how a score is given in Playability but there are quite a few things that can effect playability in a negative manner.  Trigger bugs, victory condition bugs and any other playability-destroying bugs obviously can ruin a scenario’s playability.  Lag is another playability issue that a scenario can be marked down for.  If a player is ever confused about the next goal to accomplish, that’s a playability problem.  If a player can complete an objective in a way that the author obviously did not intend to be possible (i.e. there’s a hole in a wall that allows the player to skip half the scenario), that’s a playability problem.  Anything that adversely affects your enjoyment of a scenario can be deducted from the Playability score.

Most perfectly playable scenarios should not be able to be completed without the player losing at least once or twice.  If a player is able to complete the entire scenario the first time, the scenario is probably too easy.  On the other hand, a player should not need to reload 15 times to get by a certain part of a scenario.  That is frustrating and the scenario is probably way too difficult.   The ideal scenario balance happens when a player gets stuck, but he knows that it’s possible to complete the objective if only he did something a little differently.  A player should not win by luck, the scenario should be constructed so that a player can learn from mistakes and use his skill to complete the objective.

When reviewing movies: This category is called “Plot”. Since movie scenarios are not actively played by the player, but only watched, you need to be the movie critic and give your opinion of the plot, of the content of the movie. Is the story believable and interesting? Are the dialog lines corny, and the sort of thing that nobody would say, or cool and realistic?

Playability answers the questions:

  • Is the scenario captivating and enjoyable?
  • Upon completing the scenario, do you want to play it again because it was so fun?
  • (Esp. for movies)Is the story believable and interesting?
  • (Esp. for movies)Are the dialog lines corny, and the sort of thing that nobody would say, or cool and realistic?

Balance

Balance is slightly different from Playability and Creativity although it involves elements of both. Balance represents the difficulty and flow of the work. Was it beatable but not too easy? If you had to build and upgrade your units between battles, was this done well and fairly, not attacking you with too many powerful units before you could fight them off? Are you still challenged by the scenario even if you’ve already won it? If the answer to all these is yes, it gets a high balance score.



Keep in mind that each player is a different skill level and what might be perfectly balanced for one player might be way too easy or way too hard for another. As a reviewer, you must take your own skill level into account when giving a balance score. A perfectly balanced scenario should provide a challenge for a veteran player. Most people who are downloading scenarios from the internet have at least played through the campaigns included with the game and have a good knowledge of the game.



When reviewing movies:Balance is irrelevant to movie files.

Creativity

This area is probably second in subjectivity behind playability, and basically is the originality and expressiveness of the scenario. Creativity is found in all aspects of a scenario, from trigger tricks, to map design, to the story, to what units a player is given, to the objectives, to sounds used, etc…   Every aspect of a scenario factors into creativity.  One thing to be careful for is not to knock points off of creativity if the designer uses a trick you’ve seen used in another scenario.   There’s nothing wrong with using the same trick that someone else used and no reason to deduct points because of that.

Probably the biggest creativity factors are the starting position and the victory conditions.  For example, any scenario that starts with a just a city and a few units with a conquest victory condition is simply not very creative.  The farther a player gets from a random-style scenario, the better the creativity score.

Creativity answers the questions:

  • Is the scenario a retread of other people’s ideas or is there something truly unique about this scenario?
  • Does the author make good use out of the design tools provided?
  • Does the scenario stand out in your memory because it features something not found in other scenarios?

Map Design

Map design is one of the few categories that’s very easy to define and give a rating to.  We have pretty clear-cut rules on how map design is scored and this is how it should work.  A random map is a 3.  All a designer needs to do to score a 3 is to use a generated random map.  Random maps look good, they function well and there’s nothing wrong with using a random map in a scenario, but it’s just average.   From that basis, it’s easy to figure out where scores of 1, 2, 4 and 5 come from. 

A rating of 1 is for a pathetic map… these usually consist of large blank areas with lots of square areas and straight lines.  These maps look completely unrealistic and are quite unattractive.   A rating of 2 is somewhere between a pathetic map and a random map.

A rating of 5 is for an outstanding map with lots of special details and concentrated effort to make the map much better than a random map could possibly provide.  Obviously, a rating of 4 is given for maps that are slightly better than a random.

One final note on score map design… only the portion of the map that can be seen during play should be scored.  If there are large empty areas that a player never sees, that should not affect the map design rating.

When reviewing movies: This category is called “Visuals” and refers not only to the scenery but also to the cinematography of the scenario, i.e. the camera shots and the lighting. Anything that you see on the screen, when reviewing a movie, counts in the Visuals category.

Map Design answers the questions:

  • Is the map enjoyable to look at?
  • Does the overall map view in the lower right corner look attractive?
  • Are cliffs and elevation used in a reasonable and practical way?

Story/Instructions

This is another pretty clear-cut category.  If there is no story or instructions, the score is easy… it’s a 1.  If there are instructions but no story, the max score is a 3.   If there is any story at all, the rating goes up to a 4 and if the story is really good, the rating can be a 5.  If the instructions are wrong, misleading or confusing, the rating goes down.  Also, keep in mind, the instructions and the story goes far beyond the pre-scenario instruction screen.   Often the story is continued throughout the scenario by using trigger events to move the story along.  Also, since objectives can change in the middle of a scenario, the quality of the instructions must be judged throughout the playing of the scenario.

Some other guidelines on scoring this category:  An introductory .jpg and/or .mp3 is a nice touch and a good image can often raise the score, however, an introductory .jpg and sound is not required to score a 5.  It certainly helps, but it’s not an absolute requirement.  Hints and History can also be judged here… these two areas are not required, but they can also help boost a scenario’s score.  While .jpgs, .mp3s, hints and history are not required, it would be difficult to give a rating of 5 if all three areas are missing.  The rating should not be affected based on whether the story is fictional or historical.   It doesn’t make a difference as long as there’s a story that draws the player into the scenario.

The last item that factors into the rating of the story and instructions is grammar and spelling.   A designer should be diligent in this area of his scenario, since it’s very easy to copy the text into a word processor and spell check the instructions.  There is no excuse for having spelling errors in a scenario. It simply shows a lack of effort on the part of the designer.  The only exception we make is for designers whose primary language is not English; we are usually quite a bit more lenient with them.

When reviewing movies: This category is called “Exposition” and refers to the part of the scenario that provides the background information needed to understand the characters and the action. Having all the background information you need, but not having it told in the most interesting way in the world, warrants a 3. The more interestingly the background information is told, the higher the score. For the most part, the exposition will be in the introductory text. If there is no introductory text (and hence, no written exposition), then it is expected to have the exposition in the beginning of the scenario itself, which is what you are supposed to judge by.

The instructions on the above are specifically for writing scenario, campaign and movie reviews. However, we are now allowing reviews of all file types available for download so you can rate and write reviews on Mods, Random Maps, AI Files, Utilities and Recorded Games! However, these reviews will not use a 5 category system, instead you will give a single rating to the file. When you write your review, simply include enough information to explain why you gave the rating that you did.

Now that you’ve made it all the way through this document, you are ready to write reviews! If you have any further questions about what is expected of you, please direct them to . Good luck and thanks for your interest in writing reviews!