As promised, I will continue to share my experience and thoughts as a program chair for Agile 2015. This post is my personal retrospective on the Agile 2015 submission process. Note: there is a program/track wide retro that will be conducted the Sunday before the conference to get complete feedback. This is just my two cents.
The simple view of the submission process is: Speakers have a window to submit a session for help (this closed in Feb), draft or ready for evaluation. If a speaker submitted to help, they received at least one coach providing them with feedback on how to improve their submission before they submit for evaluation. If they selected draft, no feedback/review occurred. If they selected ready for evaluation, this was reviewed to determine if selected for the program. You may or may not have received any feedback once in evaluation. Why?
The 3 feedback SLA from previous years was removed this year. Although the intent of the SLA was very good, there was a ton of wasted effort by both the program team and the submitters. So we instead implemented an optional help state for those that wanted feedback. As well as a draft state for submissions (often reviews would come in saying not enough detail and the submitter would say “i know, just a placeholder”. Our review team was then given the option to ask clarifying questions when needed but you didn’t have 3 people asking the same question to meet the SLA. Our primary goal is to create a stellar program for attendees. A secondary goal is to help the community with their submissions. I’m a huge fan of this conference because we do both (other conferences it is a complete black box with zero opportunities for feedback). This approach was trying to keep both while minimizing work for everyone. Refer to this post for some of the challenges associated with the 3 review SLA.
What went better:
- Early agreement on pair chairs for each track
- Chairs, Coaches, Evaluators…they all kicked butt!
- Value add of the help queue
- Removal of the feedback SLA for value add work (submitters and team)
- Wrong track situations were addressed by chairs quickly
- Draft status for submissions
- Program team updates to the track team
What can be improved:
- Regular program team updates to the track team
- Late approval for help queue and help chair
- Additional communication on help vs ready for evaluate (for team and submitters)
- Directions for track chairs on help experiment and reviewer selection was given late
- Do coaches (people helping on the help track) really need to be different than evaluators (people evaluating on the tracks)?
- Helping submitters understand what is considered during selection
Now I’ll be spending the next couple of months thinking of ideas to help improve the process. Please feel free to send me any ideas you have!
I submitted a talk and thought that the process was pretty clear. I didn’t know, though, if I would get more feedback once I submitted my talk finally. I found the help status to be very clear and effective, though. I got some great feedback from my coach and it really helped me update my talk. He suggested I change tracks and I’m glad he did because I think that made my talk much more helpful. My talk was accepted this year 🙂
Thank you for sharing this feedback with me! Congrats too 🙂