i would fine this very useful as its good to get feedback on both, and ive always thought aesythetics and gameplay should be equal in your maps, this feature would help in future projects and fine tuning current projects, if i make a visually stunning map but it doesnt play well it would be a good way of quickly assessing the faults of the map, if i have a 10 for looks but a 6 for gameplay i can visually see whats going on and rethink an approach to map. I feel as though with this site ( and nothing personal as its just the nature of the beast) but maps go on the wayside if you are not well established already, are not doing an exact replica, or recreating or inventing a new minigame/race. thats a trap that all forgers go through because sadly we are all forgers and hence really look at validation from are own maps and not exploring others creations or at the bare minimum never comment on them. i figure this may need its own topic
I was thinking along these same lines. A lot of the concern seems to be around the lack of a standardized rating system so something more simple would be an effective solution. As long as the user writes a detailed review, a numbered scale isn't necessary to get a sense of how good they thought the map was overall. Personally, I think the "Is it fun?" criteria is the most important when reviewing a map so the thumbs up/down scale would work for me.
this is very informative, i knew there was a system in play but numbers help establish how the process works. it does seem unbalanced in terms of filtering so its nice to see how that unbalance came to be
Per-version ratings? So if i publish a literal turd right now and everyone and their mom rates it 0/10 - but I come in and publish an update to the map maybe polish that turd, those ratings could be stored to reflect the version they were submitted for. Yea i can see this being abused but to what end? Having no current ratings? At least it would promote rating maps more and giving good numerical value feedback because its not the final say. Forgers can still fix the maps, reviews can still change, this wouldnt make it perfect by means but it would at least make it better. sorry if someone mentioned this already
I'm sorry that's how you feel about the site. We were all new at one point, and I would hope the actions of the experienced don't discourage new members from being active in the site. In terms of becoming "established", you can better your reputation in the community by testing and giving valuable feedback to other members. The more people you befriend, the greater your chances of success will be (in terms of your map's popularity). However, you should understand that maps are being posted to the site at a pretty frequent rate for the time being. Because of this, only the most polished and well-designed maps are going to draw attention. Naturally, the experienced forgers will be more likely to create the higher-quality maps, so that is why it is hard for new members to make a name for themselves. You either have to have a map that grabs peoples' attention, or you have to build your reputation over time.
After reading through the thread, I'm in favor of either a site-wide standard of the same nature as the one @SecretSchnitzel posted, allowing someone to change their review, increasing weighting on qualitative feedback reviews, and rewarding people who left big reviews. In my opinion, having a rating system is good - @WARHOLIC's point on filtering the 400/month rate is definitely valid, and on top of that, a rating is something that everyone from veteran to newbies just groks right away, no explanation. I think the concerns @aPK raises about the limitations are worthwhile - I noticed a lot of them stem from the fact that information is lost when you bring a review down to a number. Site standard: Setting an expectation of what a Forgehub score is should address the concerns about the various tastes. You'll still have variation, but it's a step in the right direction. Whether or not this is across two scores doesn't matter that much to me. For what it's worth, I agree with the sentiment that Goat's % system is best suited for competitions, but is a bit too complicated for everyday reviewing. Changing review: Allowing me to alter a score after it's been posted would be a good thing, that way I don't have to worry about giving someone's beta a bad score that sticks with the final version. It could actually be encouraging to a new user to go from their initial 4, to a 6, then a 7... as they take and implement feedback. And, it would reinforce iteration on designs, something that's always good for new Forgers to learn. If we did that, it'd be really cool to notify reviewers when an edit/update is made, and ask them to take a look. I saw someone post a similar suggestion earlier up. It might also be interesting to notify the Forger when someone increases their original review score. I wouldn't want to notify them when someone decreases it, because depression and such, but seeing an alert that somebody liked my changes would absolutely brighten my day. Qualitative Feedback: What if we gave higher weight to reviews that left a text comment? You could do a tier system... the guy who writes an essay likely isn't -pardon my french- circlejerking as much as the guy who drop in to give his friend's map a 10. Something like this, with actual numbers: 0-50 words = Nothing (+.0) 50-100 words = Tier 1 (+.25) 100-500 words = Tier 2 (+.75) 500+ words = Tier 3 (+1.25) I don't know how hard it would be to tackle the technical implementation... maybe just tie weighting to a wordcount in the map thread on a user's post? Don't know enough about the backend to suggest anything useful there =/ Rewarding Quality Reviews I saw something (Warholic's?) about the 5+ users liking a review to designate a quality review... Taking a cue from Stack Overflow, an alternative method might be a way for a map maker/OP to mark a review as helpful, since they are ultimately the one who needs to benefit from that review. (Or maybe just a like from OP). Positive reviews would probably be an issue with this system, being more likely to draw a "like," but a word count minimum could help ensure that it consists of more than "loved it lol." And, since the person receiving the feedback is the judge of what is useful, it would give incentive for reviewers to be civil when they didn't like something. I don't want a full-blown reputation system, and I don't even know how useful it would be to make this visible, but I do think that the person receiving feedback is in the best position to judge that feedback. Lots of words, but this is an interesting discussion =)
Christ, you guys overthink this ****. Aren't maps posted in the map section supposed to be (near) finished anyway?
At this rate (pun intended), the site is going to have to separate the maps into two separate ratings - User Rating and Critic Rating - like Metacritic does. User Rating can be based on "how fun the map is" and Critic rating can be based on "How good the map is". And that would just be a simple tweak to the current system. The former can just be whatever - get your friends to boost it if they like your map. The latter however would of course need a standardized scale, otherwise it'd be meaningless. Under this system, a map will only receive a "critic rating" if it garners significant attention or is somehow solicited for one - perhaps in a thread posted specifically for authors looking for critical feedback. That way the critics can put the maps up in lobbies. Of course, any of the Forge critics can comment on whatever they want, but it would be their own responsibility to review maps outside of their comfort zone. Case in point - 5 star maps on the forum right now that IMO should not be 5 star maps. But there's absolutely no reason for me to go in and leave an actual review on the map just to bring the rating down to "where it should be".
I kind of like that system. If we're taking the time to designate members as critics/talented whatever, it'd be pretty cool to be able to just scope out their feedback. And if we're publicizing the critics rating, it falls to them to keep feedback objective. Not saying they aren't doing that currently - just need to be sensitive to the new member perception of the process.
I agree with this being the main flaw behind a critic and non-critic system. Considering the amount of traffic ForgeHub is getting in terms of map uploads there would need to be a lot of active critics to balance everything out.
Well, with the flow of maps, there has also been more of a flow of activity as well. It could balance out. I could see forge maps spiking at the release of new forge stuff though. It's a hard call, but I think its worth a shot.
Fair point, I'm open to trying any new system if there are enough users who would like something different. As long as the option to write stays available I'm content.