Vote of the week: Words or numbers?

September 25, 2009, 7:13 AM · It's waaaay early, but I am preparing for next spring's Best Attraction in America tournament already. This time, the brackets will be set by New Year's, to allow me to introduce a daily feature profiling each of the participating attractions, before the tournament starts in mid-March.

This year's tournament will feature four brackets, for best roller coaster, themed ride, live show and multimedia show, respectively. Which 16 attractions get in each bracket will be determined by you - through the cumulative reader ratings submitted on the site.

That's why it is so important that Theme Park Insider readers click through our park listings and vote of the rides and shows they've experienced. Please take a look at our Guidelines for Rating, to get a sense of what you should be thinking about when deciding your ratings.

Although we rate attractions on a scale of 0 (Intolerable) to 10 (Perfection), when you are submitting a rating, you won't see numbers. Instead, you'll be asked to pick which one of 11 adjectives best describes the attraction:

Here's my question for you: Is that the way we should do it? Or should we simply ask you to pick a number - from 0 to 10, instead?

I switched from numbers to the descriptions years ago for two reasons - 1) to eliminate confusion as to whether 10 was the bad rating or the good one, and 2) to discourage "ballot stuffing" by people who just clicked 10 or 0 for everything.

But is this system discouraging people from voting by making it too confusing? You tell me.

For what it is worth, I have implemented a system to throw out votes from people who do try to manipulate the system with extreme voting. TPI's publish system converts all the ratings to their corresponding numerical value when computing attractions' average reader rating. But it throws out all votes from readers who vote all or almost all 0s and 10s. (Techie explanation: If the standard deviation of your votes is too high, or too low, bye-bye.) Doing this helps amplify the voices of the readers who take the time to submit thoughtful ratings that use the entire range of available choices.

If you have a suggestion, please leave a comment on how we can encourage better use of Theme Park Insider's rating system. And thanks for reading TPI!

Replies (10)

September 25, 2009 at 7:47 AM · Ok...I have a an idea to help with voting. It utilizes both numbers and words. You would still have to have a 0-10 vote, but then you would describe in a sentence or two why you voted that way. It could be set up to where it's an open box (just like the one i'm typing in now) and state your reasoning, or we can set it up to where we'll ask for you A) pros, B) cons, and C) Describe it in one word. This way we can make it easy to see which votes are true votes and not just (Hey! Go to this site and vote a 10 for such and such a ride!) And if it is, we can throw out the vote. But this system would allow people to vote a 0 or a 10, but only if their reasoning is sound.

Ok here's some examples of possible votes. Both of these will be for Expedition Everest:

Example 1:

Vote: 8

Reasoning: I love the themeing of this attraction from the time you enter the queue to the time you leave the gift shop. The ride itself is fun (the backwards section was a surprise) and the turn at the bottom of the big hill was a blast. However, I feel that the ride could be a little longer and I wish that the Yeti was more reliable.

Here's another way you can do it:

Example 2:

Vote: 8

Pros: Great themeing throughout the ride, fun elements of the ride, backwards section was surprising and fun, yeti is cool (when it works)

Cons: Ride could be a little longer, yeti could be more reliable

One-word description: Awesome

Now it could use some hammering it out but I'd thought I throw it out on the table

September 25, 2009 at 9:21 AM · I enjoy the way it works now. Pretty much anything over 7 is worth it. I do not think having either way is really going to change the difference. Lets face it, we all have our favorites. I love TPI, but was much more impressed with DCA than TPI gave it credit for in the past. I mean, I am not a fan of Dudley Do Right's Ripsaw falls and I know somebody here is not a fan of the Jungle Cruise...
September 25, 2009 at 11:15 AM · Standard deviations FTW!

Robert - I'm sure you'll be excited to hear that as part of my masters in mechanical engineering, I'm taking Advanced Engineering Data Analysis and Experimental Design (read: Advanced Stats). Now we can have stat chats on the site.

Use both numbers and words. I've been doing it in my recent Orlando trip report.

September 25, 2009 at 2:40 PM · I'm assuming some sort of dance-off is out of the question?

I'm lookin' at YOU Rao!

September 25, 2009 at 4:13 PM · I prefer to let the Yeti do my dancing for me!

Anyway, I like the rating system currently in place, however there is nothing you can do to fix the issue of bad voters... I mean is Primeval Whirl really a 7? When someone asks, how was Primeval Whirl do you answer "it was a good ride!" or do you say something more along the lines of "what a piece of uninspired midway sh--!" and hit the person asking? And are the Kiddie Swings at Kennywood really worthy of a 10? I am gonna go out on a limb and say "I don't think so."

As long as the system in place requires voters to register, log in, and rate an attraction (once only), and accounts for the guys who put 10's on everything (Jake Rivers, you know who you are - or do you?), then I think you have done about as good a job as you can do. Sorry, Robert, there is just no way to fix the bad voters issue 100% of the time.

September 25, 2009 at 11:05 PM · You might want to consider replacing "lame" with another adjective. To many people, especially those in the disabled community, using "lame" to express negativity is pretty offensive.
September 27, 2009 at 6:21 AM · I think it works well as it is although numbers and words would be good too. I think the words used make you rethink whether a ride actually desrves a 9 or a 10 more than the numbers alone would.
September 27, 2009 at 3:42 PM · I agree with the Politically Correct Curtis. Let's replace "lame" with the phrase "Pirates-of-the-Caribbean-Like."
September 27, 2009 at 5:16 PM · Ouch...that hurt...

But to be honest, thinking about it for a while, what's the difference between a number and a word, if they pretty much mean the same thing?

September 27, 2009 at 8:22 PM · If "Pirates-of-the-Caribbean-Like" is the new lame, then what do we do about a Jungle Cruise attraction (I use the term loosely) that rates lower than POTC across the bored...err, I mean...board?

How about a new rating called Jungle Snooze to represent those attractions that are pathetic, dilapidated, and obsolete.

I like it!

This article has been archived and is no longer accepting comments.

Park tickets

Weekly newsletter

New attraction reviews

News archive