Feedback Ratings

I recently received all of the feedback forms from the conference in November.  In going through them, I realized that I’m conflicted on a number of topics associated to these forms.  This first post is going to focus on the ratings.

How this conference (and many others) handle speaker ratings is on a 10 point scale where 10 is excellent and 1 is poor.  This conference calculates the rating by 10 x # of 10 ratings, 9 x # of 9 ratings, etc.  So if I got 9 10s, 3 9s and 8 8s, this equals 181 then divide by the number of evals (20) = 9.05.

At face value, this is valuable information.  At one glance, I can see overall whether my session was well received or not.  However, there are several aspects that trouble me:

• This conference in particular doesn’t invite back poor speakers. The problem I have is what’s a good rating? Is anything above an 8 good?  Or above a 9?  I really don’t know the answer to this. Personally, I’m satisfied when the rating is above 8.0.  I’m happy when the rating is above 9.0.  I’m thrilled when the rating is above 9.5.  However, I have no idea if I’m expecting too much or too little.
• Not everyone fills out the forms (online or paper doesn’t matter).  Now you could argue that they were ambivalent and thus didn’t fill out the form but that’s not always true.  And even if it is, shouldn’t that be taken into consideration with the ratings?  I know I pay attention to how many submitted evals there were vs the attendance number.  Yet, I’ve experienced someone approaching me to tell me that they didn’t fill out a form but wanted me to know they really appreciated my session.  I asked them why and I absolutely loved their answer:  “I’m challenging myself to give feedback directly to people”.  Another scenario, I often attend sessions for friends of mine.  I do this to support them and to see if there are any new nuggets of information to learn.  I don’t often fill out the form because it feels like I’m padding the results.  Yet now I’m an attendee count.  So I’m torn on whether the no feedback numbers should be included.
• So far I have not received an overall score under 8.  I do receive individual scores that are low.  Majority of the times that this occurs, it comes with zero additional information.  It means I see a 4 with absolutely no indication as to why.   I’m left with not knowing what I could do differently or why it was miserable for them.  My first reaction is “tell me why”.  My second reaction is “why did you stay?”.  I fully believe in the law of two feet at conferences and that not everyone will like a speaker’s style or content for a variety of reasons.  Regardless, I’m left to ponder why and sometimes it overshadows the trend of positive feedback.  A person can get twenty 10s but that one 4 is what you think about.  So without the comments, what does individual ratings really tell me?  How do they help me get better?
• This last area is tricky and I recently attended a session that helped me put this into words:  When I decide on a subject, I am doing this to inspire and/or challenge people to look at something differently.  Depending on where you are in your journey, this could be well received or completely hated.  I want to evoke emotion…whether it’s positive or negative.  The presenter actually said “I want you to walk out of here thinking I love or hate this presenter.  I don’t want you to be ambivalent”.  I have experienced situations where I’ve walked out of a room thinking that was awesome to then overhear someone else say it was a waste of their time.  So I began accepting that great sessions are not about pleasing everyone.  Yes, you should be a good speaker but if your topics are pushing boundaries, you might ruffle feathers that will end up in the feedback.  Yes, you should have quality content and be able to stand in front of an audience with basic skills. However, after that…personal preference and situations are a huge factor.  Take something as simple as a speaker who swears.  Some audience members gain respect and some lose respect.  You can’t satisfy everyone.  Personally, as much as I’m a sailor in real life, I try to avoid this during conferences.  I can get my point across regardless.  Yet, I understand why some speakers take the stance of incorporating this into their session.  The goal is not to please everyone…especially at a conference when you have no idea who will show up.    This takes me full circle back to the first question…what really is a good rating?

I’m very thankful for the feedback.  And when I receive the packet, I always pay attention to the trend of the rating but for my own sanity, I try to keep in perspective that this is about helping at least one person not everyone.  Most importantly going forward, I will make more of an effort to not just put that rating down but give a small note for each feedback form I fill out in the future.  As that’s the really valuable part for a speaker to learn.

What do you think of ratings for speakers?