Thursday, February 03, 2005

Online Surveys and Social Network Analysis

I've been deep into survey administration the last couple months. For more about the project history, see my previous posts on Six Sigma Church, which we now officially call Renewing Our Calling.

Our project included a major survey which we made available on paper and online for the whole congregation (roughly 400-500 people). We just finished collecting responses and were happy to get 167 of them. You can try a dummy version of survey right here.

We used SurveyMonkey to support the online survey. This is an amazingly powerful service and it's cheap, too. In many situations you can use it for free. We needed the "professional" version which is still a steal at $20 per month.

My experience with SurveyMonkey wasn't all good. We originally formatted our survey into multiple columns per screen, but found that SurveyMonkey has a bad habit of randomly skipping questions presented in this way. Thankfully we paid close attention to survey responses as they came in, so we didn't lose much data. However, my experience with SurveyMonkey tech support was decidedly mixed. They responded quickly to my initial complaint but then did a poor job of following up and they never actually acknowledged that there is a data integrity problem with multi-column surveys in their system, despite overwhelming evidence to the contrary. (Some of our anonymous survey respondents actually wrote helpful debugging comments in their responses, as they struggled to make sense of questions garbled by SurveyMonkey; and my personal experience doing data entry confirmed their remarks.)

Many of my readers are probably especially interested in administering social network analysis surveys. To help you along, I recommend two resources. First, check out The Hidden Power of Social Networks by Rob Cross and Andrew Parker, which includes a sample survey form in an appendix. Then, read "Ethical and Strategic Issues in Organizational Social Network Analysis," by Steve Borgatti and Jose Luis Molina, who explain the delicacy of administering a social network survey, which by its very nature lacks the safeguard of anonymity we usually expect.

Based on these two resources, here is a generic social network survey I have written, which you may find helpful.

And finally, go take this survey, commissioned by the National Research Council to "study the possibility of identifying 'Network Science' as a cross-disciplinary area of research worthy of enhanced attention and funding."

20 comments:

Bruce Hoppe said...

We are working with XXX to help reduce conflict, improve communication, and clarify roles and responsibilities. The following questionnaire focuses on communication within XXX and is an important part of this process.

In order for this survey to be effective, we need participation from as many managers of XXX as possible. It should only take you 15 minutes to complete this survey.

One result of this communication survey will be a map of who communicates with whom at XXX. By its very nature, this map will reveal individual communication patterns. We will present the results of this survey to the collected management team of XXX.

Therefore, unlike other interviews and questionnaires we have administered, your answers to this survey will be shared with your fellow managers at XXX. Of course, all information you provide is voluntary.

Please provide the following information about yourself:

Name:
Job Title:
Primary Function:
Tenure in XXX (months) _________

Location:
Basement [ ]
First floor [ ]
Other _______________


In the following table is space for you to identify up to twenty people. Please list the most important people in terms of providing you with information to do your work or helping you think about problems posed by your work. These may or may not be people you communicate with on a regular basis and can come from within XXX or outside XXX.

Q1 Q2 Q3 Q4
Name Proximity Org Aff Time Known Rel Level

______/___________/_________/___________/__________

______/___________/_________/___________/__________

etc.

For each of the people listed above, please answer the following four questions

Q1. What is each person’s physical proximity to you?

1 = same floor
2 = different floor
3 = different building
4 = different city


Q2. Please indicate the organization in which each person works.
1 = same group in XXX
2 = different group in XXX
3 = outside XXX

Q3. How long have you known each person?

1 = less than 1 year
2 = 1-3 years
3 = 3-5 years
4 = 5-10 years
5 = 10+ years

Q4. Please indicate each person’s hierarchical level relative to your own.

1 = higher than yours
2 = equal to yours
3 = lower than yours
4 = not applicable

[Page 2]

Below are three questions about your communication with the management team of XXX. To the right of each question is the response scale to use in answering the question. Please fill in the table below with the appropriate responses from 1 to 5.

Q1: Information – Please indicate the frequency with which you typically turn to each person below for information on work-related topics.

0 = I do not know this person
1 = Never
2 = Seldom
3 = Sometimes
4 = Often
5 = Very Often


Q2: Awareness – I understand this person’s skills and knowledge. This does not necessarily mean that I have these skills or that I am knowledgeable in these domains, but that I understand what skills this person has and what domains they are knowledgeable in.

0 = I do not know this person
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree


Q3: Communicate More – I would be more effective in my work if I were able to communicate with this person more.

0 = I do not know this person
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree


Q1 Q2 Q3
Name Information Awareness Communicate More
[Mgmt team of XXX listed below]

Jon Doe________________________________________

Stephanie Smith___________________________________

Lily Jones_______________________________________

Mark Abercrombie_______________________________

Doug Craig_____________________________________

Nigel Ogdorf___________________________________

Celia Missenwhoop______________________________


Thank you for completing this survey! Your participation is crucial to helping XXX communicate more effectively.

Malchus said...

Bruce:

Happy to hear that our discussion and presentation on surveys at MBODLG was "just in time." I'll be interested to learn more about your SurveyMonkey columnar response issues in a follow-up. I've used the tool for about 3 years, and have had pretty good results - no question skipping of note based on question type. What I HAVE experienced is survey participants' reporting that they landed on page 2 or 3 of the survey upon sign-in. The simply have to use the arrows at the bottom of the page to go back, but it was initially distracting.

Good to hear that the final result was helpful. And thanks for the update.

Anonymous said...

Check out express.perseus.com -- always free. Not buggy when it comes to having respondents complete survey. I heard it can be buggy to design if using AOL.

Bill Higgins said...

I agree SurveyMonkey has much going for it but have you ever experienced emailing them for assistance? Their responses are very short, lack full explanation and only reflect one aspect of your questions. I have had simple questions asked and only one answered at a time and then only very briefly.
Writing back to them and asking them for responses to my questions [5 times with same and related questions, including my survey file to review], still failed to prompt them to practice common sense customer service.
All the good things about SurveyMonkey shrink in comparison when your having a problem, repeat the questions and still not get the answers.
Life is too short...I'm going elsewhere for my next survey!

TS said...

Nice Blog!!!   I thought I'd tell you about a site that will let give you places where
you can make extra cash! I made over $800 last month. Not bad for not doing much. Just put in your
zip code and up will pop up a list of places that are available. I live in a small area and found quite
a few. MAKE MONEY NOW

TS said...

Nice Blog!!!   I thought I'd tell you about a site that will let give you places where
you can make extra cash! I made over $800 last month. Not bad for not doing much. Just put in your
zip code and up will pop up a list of places that are available. I live in a small area and found quite
a few. MAKE MONEY NOW

harvir said...

Hi i am totally blown away with the blogs people have created its so much fun to read alot of good info and you have also one of the best blogs !! Have some time check my link to !!Earn money fast and easy

TrafficMegaDeals said...

OVER 1 MILLION WHOLESALE PRODUCTS

at Low Factory WHOLESALE Prices in the U.S.

and Around the World!



From The Best Manufacturers, Exporters,

Wholesalers, Drop-Shippers and More!
network
Profit from selling Thousands of Products at

yard/tag sales, flea markets, auctions,

newspapers car boots etc! ect!

http://massivesavings4u.com

network

lokokid said...

GOOD content thats what i like and just surfing around on blogs and finding good content is good feeling i am so glued to those blogs dont stop making good content keep it up i am not a pro on making pages but i see if you can visit my site maybe its something that you will like!!Work at home computer jobs

Clickbank Mall said...

Simple Tasks Create Lots of Cash
money for surveys
Are you interested?
money for surveys
Then Try This!
money for surveys

WorldTrader said...

Hey, you have a great blog here! I'm definitely going to bookmark you!

I have a perfect home business site, Check it out. It pretty much covers ##KEYWORD## related stuff.

Come and check it out if you get time :-)

----------------------------------------------------------------------------------------------

Admin said...

Great blog, keep up the good work. Glad to see sites like this.

Here is another good site I said I would pass along.
Domain Names For Sale
Thanks

Clickbank Mall said...

Simple Tasks Create Lots of Cash
money for surveys
Are you interested?
money for surveys
Then Try This!
money for surveys

Anonymous said...

Great Blog, check out this business. This is the Goose that lays you Golden Eggs! business home internet site top

Enjoy!

Smartie said...

I love your information on six sigma I bookmarked your blog and will be back soon. If you want, check out my blog on six sigma exposed - please come by

blaze said...

You have a very good site on business home start This is something I also have a large interest in and have set up a blog about business home start please visit and let me know what you think.

Weedlet said...

Wow - you're blog is full of good info. It's getting hard to find blogs with useful content and people talking about six sigma these days. I have just started my Latest six sigma news blog and would really appreciate you coming by - thanks again

paige said...

I really liked the information on your blog about six sigma I have my own six sigma exposedblog if you would like to come and see what I have on mine

Anonymous said...

If you think SurveyMonkey is cheap, check out this site
http://www.stellarsurvey.com
They are twice as cheap as surveymonkey at 10 bucks per month. I recently did my survey campain with them and I didn't have any data issues with them. Oh yeah, they also have a free subscription. Good luck.

Get paid to take surveys said...

I have used Survey Monkey successfully in the past. Recently I am totally frustrated at their non-existent customer service. Their design possibilities are very limited.