The map we tested was the one originally submitted by fro3po. The map was most likely updated after the submission period ended which is why we didn't see any of the weapons on pads or any significant frame drops. aPK's post was hidden because it contained information about the contest which was not made public yet. It spoils the excitement behind an official announcement. Furthermore, our conversation was private and I asked that he respect that privacy. I also let aPK know that I would happily unlock his thread after official announcements were publicly made. I approached aPK out of respect and let him know that we were changing the prize pool structure to top 5 since he was very much against it the first time he heard about it on March 14. At the time of our original announcement, there was a lot of negative criticism against our prizing structure, people felt that maps shouldn't be judged in a numerical hierarchy. Instead of dismissing that feedback, I gave it some serious thought and came up with an alternative solution. One that we will use for all future contests, if we're still motivated to host them These debates only clarify how subjective and different our interpretations actually are when reviewing works of creative expression. I'm aware that changing prizing structures after official announcements are made is unprofessional, however these changes more accurately reflect our thoughts as a judge panel. We had tremendous difficulty establishing a top 3 due to the quality of the map submissions being so much higher than expected. The alteration made to our prizing structure benefits all parties involved as it creates more winners and essentially increased the overall prize pool from $1000 to $1300. It was one of those situations where the negative attributes were outweighed by the positive ones. @Schnitzel - completely derailed...so sorry man!
When we tested the map, we used the version Chunk saved from fro3po's files at all times until approximately the 8th or 9th testing round. All contest maps were bookmarked meaning that if updates were applied they would carry over to the versions we tested. If memory serves me well, we began testing on March 7th so there was a period of time where all submissions could have been altered without us knowing. Its entirely possible that fro's map was not the only one that was altered after March 1st. In all of our testing we've never played a version of the map with the weapons on pads.
Have you read the 'rules'? The judging criteria listed in the contest thread was extremely ambiguous, and intentionally so. This is because what makes a map good/great isn't something that can be quantified or determined by utilizing some predetermined formula. The guidelines posted were just our way of giving a general idea of things that would be taken into consideration during the judging process. Here's exactly what it said: Eligible map submissions will be judged based on: Game-type Support Does the map function for the game-type as necessary? Are all required objects and systems in place to successfully play a game? Fun-Factor Is the map fun to play on? Is the space fun to navigate through? Do the weapons, spawns, and layout promote a positive game experience? Is the map balanced and fair for the given game-type? Performance Does the map suffer from any negative visual or performance issues? Originality / Creativity Is the map interesting and unique in both design and appearance?
If we are taking a vote. I'm on the side that after the map submission deadline, maps should be judged as is. The amount of time it takes to fix an issue is irrelevant, asking permission is irrelevant. Matchmaking worthiness should be judged separately from the contest. There's no reason why Orxgens couldn't do poorly in the contest due to flaws on the map that was submitted, yet edited and still allowed in MM. Was every map given this treatment? fixable problems noticed, where it was fixed and retested? If not, which I doubt it would matter... i'm sure a case of Equal Dignity could be argued as well. Onto legality of changing rules/payout structure: The South Central Communication Corp court case ruling said that Once a Contest begins, you cannot change the rules. I'm still looking for case law on changing the prize structure on a contest, but I am doubting that's legal as well. Sources: http://transition.fcc.gov/eb/Orders/2014/DA-14-358A1.html http://www.jonesday.com/files/publi...a311c5-f53d-4a89-97fc-d85993081c2e/bennet.pdf In conclusion: this contest was poorly run... if the prize wasn't so small, i'd worry about hiring a lawyer.
^^^ Putting on my grown up pants for a second, especially for the monetary side, everything Bodey said above. This is a learning experience for all.
One thing: about one month ago it was said that it was being difficult to judge the remaing maps (maybe their number was 25? I don't remember) because they were all really good. Well, was that map so good to enter since the initial testing in the top 5? From what I read, it was a little modified before entering the top 5. But if all the map were really good, why they didn't receive all the same treatment? They would have been little changes, like the incrimined map. Another thing: I would like a secret testing and judging. Why are there some people who know the results and what was said in the testing? I don't care if you trust them, it's just not fair to me and to lots of other people.
Results get leaked/people can make prediction on maps doing well if they are played more by the judges/offhand comments are made. No malice there.
When theres 140 odd maps, people assist in the very early stages to eliminate maps. Once they narrow down to the top 15/20/25 bubble, thats when the contest is reigned back into the four judges. Although I wasnt involved, this is pretty standard.
Understood, didn't know that. It's understandable, but the form has to be changed. Maybe more official judges?
By the time I got to page 4 I had to stop reading as the whole thing just seems pointless. Good luck with the contest.
I believe that Orxgens should be disqualified for the monetary portion of the competition. The editing by the judges seems unethical and unfair to me. I'm sure that many maps could benefit from editing - even if it's subtractive.
Is this the "judges have open mic" thread? Maybe I can post now. Anyway, I know that most Forgers view me as an inexperienced designer. I'm the first and the loudest one to say it, so I'm not too keen on having an echo, but I acknowledge it nevertheless. Additionally, I am aware that I have been vocal about my design preferences. It was not and never will be my intention to patronize anyone by parading these in their faces; if I ever sound haughty, it's directed towards my own abilities and the standards I hold myself to. And it goes without saying that those grow, evolve and mature as my understanding does, but I digress. I knew my strengths and my position within the panel and I stuck to what I felt I was capable of contributing. As the assessments of the submissions began, I observed every map as I would any map: I asked myself what the designer intended to accomplish with the map, and how far it went to that end. I looked at the quality of the forgework and how much attention was paid to its presentation. I studied the science behind the geometry and how the items placed therein impacted the games that were played on them. I'd say that this was the most straight-forward part of the process. I was impartial about almost every map, and even those I was fond of were put through this lens, therefore I believe I gave every map a fair initial assessment. I'd like to say that I did all of this independent of my personal biases, for example: I didn't hold it against a map if I didn't like the visuals, but I did make note of visuals that may have impeded player orientation or depth perception. However, when it came to comparing the map to another map, that's where things got tricky. At that point, it's very difficult to remove your preferences. It's human nature to choose the one we like the most after all. Therefore I turned the aforementioned attributes into categories by which every map was ranked. In some categories, A was ahead of B C D E and so on. In other categories, the letters were jumbled. This system ensured that, should a map excel in most categories but fail in a few, it would still have a chance. Even when this tier system wasn't verbalized, it was being drawn upon subconsciously. To go off of what @a Chunk said, it was very unlikely for a map to be cut for failing in just one of these areas; it was usually a combination that brought them down below the rest. At the end, I believe I had a fair comparison between various attributes on the map, and a list naturally arose from that. It was then and only then that I consciously chose to weigh my personal enjoyment in the event of any tiebreakers. Of course, the ability to look at designs this way is a skill in its own right. I know one of my biggest regrets during this contest was some off-color comments I made about a map within earshot of its author. It wasn't something that was brought to my attention, but as I thought about it, I reassessed my stance, which in turn helped me evaluate the remaining maps in a more professional manner. All this means is that, while the judges have not always agreed, everyone sought to approach their evaluations in the fairest possible way. And as individuals, those approaches and therefore our selections inherently varied between us.
Back in the H2A 1v1 contest, my map (Verse) made it pretty far. Verse didn't make top 3 because it had performance issues, even though the design was equal to or better than the winning maps in at least one of the judges opinions. As you can imagine I was pretty bummed to hear that, especially because I could have fixed the frame rate with a little more work, but I completely understood and still understand why it was edged out. I'm not saying that Orxgens wouldn't have placed if it still had performance issues or a terrible weapons setup, that doesn't matter at all, but War took that possibility away when he went in and fixed those things. It was completely unethical and unjustified, and I think Orxgens should be disqualified as a result.
But in this case, it's not about the map being cut or not. It's about the map progressing to the significant stages of the contest. To put it very bluntly, the edited version of the map stole a spot in the top 10 (and a playlist spot) and it stole a spot in the top 5 (prize money). It is very likely that no one would have an issue with the edited version being used for the future tournament. Most people would have probably been fine with the edited version being a supplementary map to the top 10 as opposed to being amongst the top 10. Is it unfortunate that an otherwise solid map would be eliminated for having severe frame-rate issues due to it having a few extra lights? Sure, but this happens all the time in other forms of contests and award cermonies. "X" had a spectacular season and would have won the MVP if he wasn't injured for a portion of the season. "Y" would have won the Grammy if it wasn't for the subpar mastering and that one disappointing track. "Z" would have won the tournament if that one player didn't underperform. This is not a map tryout. Contests are not judged on potential. Submissions to contests are intended to be judged completely as-is, and any detriments regardless of whether or not they can be easily remedied or not should not be overlooked whatsoever.
I don't necessarily disagree, but I'm being intentionally vague because the results are not final yet. I will however go on record to say that my evaluation of the map Orxgens had placed it outside of the Top 10 before these issues came to light.
I suppose I should have clarified that the entire post wasn't necessarily just in response to you, but more of just a response to the entire contest that stemmed from a portion of your post.
I understand. We're in a bit of a weird position at the moment because Orxgens has been given an unfair amount of criticism for our mistakes, and any map that potentially switches place with it will come under scrutiny as well. If I was the author, I would find this very discouraging. I have not seen @thefro3po's response to this situation, but he would earn my utmost respect if he were to keep his head up and continue to Forge.