The world's most popular sociology experiment

Many of my colleagues are off to Greece this week for the annual "Sunbelt" conference of the International Network of Social Network Analysis (INSNA). Greetings to you all--wish I were there.

In past years I have vented about the incomprehensible spaghetti networks displayed far too often at Sunbelt. This year I have a new gripe: the habit of presenting "organizational case studies" where the "organizations" are classrooms full of the researcher's own students.

Sunbelt would do itself a favor by clearly labelling these "classroom case studies" with a big asterisk. It's hard to coordinate sociology experiments on a scale that deserve public presentation, far easier-- and less meaningful-- to conduct experiments on a captive audience presided over by the principal investigator.

I recently drafted my 60 students into a classroom case study where we created our own set of Google rankings, based on students' links to each others' websites (programming projects created over the course of the term).

I concluded the exercise by requesting each student to write an essay, "What do these rankings mean?" The strongest essays responded with pointed criticisms of the classroom case study technique: a classroom population is too small and the classroom environment too artificial for meaningful sociological measurement to emerge from such a setting.

Daniel Essrow writes: "The Google ranking algorithm provide[s] a clear and consistent way to organize recommendations of websites.... In the vast world of the Internet, some voices do indeed matter more than others; but this a world with billions of sites made by millions of people over handfuls of years. Our class, with its limited experience, only artificially made some opinions appear more important than others. Moreover, those voices that were artificially amplified were chosen by a system very susceptible to cumulative advantage, and even vandalism. The small network made up of members of the class lacked the time and nuance needed to truly determine which voices deserved greater importance, and instead relied too heavily on an arbitrary system to make those decisions."

Evan Scott writes: "Google ranking utilizes eigenvector centrality.... This strategy works for Google because when someone performs a Google search, they are searching (whether they know it or not) for the most authoritative websites on their topic. Thus, calculating the indegree of websites, while taking into account the authority of the sites recommending them is a logical way of ranking authority. However in the context of our class, there is no central topic that we are attempting to create websites about. This begs the question: what are the most voted for (authoritative) websites authorities on? Because of vague criteria, the results are highly susceptible to the negative traits of “cumulative advantage.”

This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License and is copyrighted (c) 2007 by Connective Associates except where otherwise noted.