What Price Success? Improving Posting Quality on HPfGU

Tim Regan timregan at dumbledad.yahoo.invalid
Fri Feb 18 17:16:37 UTC 2005


Hi All,

Gosh, I wish I'd posted into these threads earlier. I kept being 
sidetracked by work stuff so that when I finally have time and 
decide to print off the discussion to read it runs to 106 pages!

Here are my thoughts on list quality, posting volume, how to baton 
down the hatches in preparation for July's book release etc. 
Although I am a proud list elf I am writing here without my tea-
towel on, just as me. (Naked posting, will I start a trend?)

I think a lot of this discussion hangs on list metaphors. David 
started a great thread on those in December 2003 in post 202 ( 
http://groups.yahoo.com/group/HPFGU-Feedback/message/202 ). The 
discussion veered between seeing the list as a cocktail party, a 
seminar, the letters page of a newspaper, etc. Carolyn and Barry's 
comments strike me as wanting to see the list as a peer reviewed 
academic journal. All posts would be reviewed by elves before 
hitting the list. I spend my day trying (and mostly failing) to get 
published in academic conferences and journals and I really do not 
want to see the list go that way. I do not see the list as the place 
where well rounded theories are published, but the place where well 
rounded theories are born.

This comes to all the proposals on curtailing list volume, which in 
general I am against. For me it's a gambling argument. If you want 
to maximise your chances of rolling a six then the best way is to 
roll the dice over and over again. Granted, good posting is not just 
luck, but there is still no substitute for practise. A colleague of 
mine ( http://research.microsoft.com/~shellyf/ ) once chastised me 
for spending too long in the thinking phase of a project; she 
said "Tim, you learn more from use than from speculation" and I 
think that holds true here. If we want to read twenty thousand good 
posts then we'll need to wade through eighty thousand bad ones. I 
know this from experience. I have written some good post, but I've 
written more poor ones. And often when I hit the send button I did 
not know which category the post fits into.

Clearly we can introduce rules and mores that guide list members 
towards better quality posting (no one liners, no "me too", canon 
points only, ...) and the presence of a good catalogue of the best 
theories will help enormously but in the end it is only through the 
to and fro of weak threads that we sometimes see the to and fro of 
great threads.

I am also against periods where we do not accept new members. I have 
a bad feeling that JKR will one day find herself at a LAN party with 
Philip Pullman, the Pope, the Dali Lama, and <insert your favourite 
thinker here>, and they all decide to sign up to an HP discussion 
group together. Finding ours closed they move on.

However there are those that would disagree strongly with me. Clay 
Shirky in his piece "A Group Is Its Own Worst Enemy" 
http://shirky.com/writings/group_enemy.html deduces that good groups 
attract members, become bad groups from having too much posting, 
loose members, and go round and round this cycle (or die). His first 
three conclusions pretty well match HPfGU: identities members can 
invest in; a way for members to attain good standing; and barriers 
to participation (our moderation of new members). But his last point 
bears out the proposals made here for closing HPfGU to new members 
periodically. I'll quote him as it is very well written (IMHO) even 
though I disagree:

>>> And, finally, you have to find a way to spare the group from 
scale. Scale alone kills conversations, because conversations 
require dense two-way conversations. In conversational contexts, 
Metcalfe's law is a drag. The fact that the amount of two-way 
connections you have to support goes up with the square of the users 
means that the density of conversation falls off very fast as the 
system scales even a little bit. You have to have some way to let 
users hang onto the less is more pattern, in order to keep 
associated with one another.[...] Sometimes you can do soft forking. 
Live Journal does the best soft forking of any software I've ever 
seen, where the concepts of "you" and "your group" are pretty much 
intertwingled. The average size of a Live Journal group is about a 
dozen people. And the median size is around five. [...] IRC channels 
and mailing lists are self-moderating with scale, because as the 
signal to noise ratio gets worse, people start to drop off, until it 
gets better, so people join, and so it gets worse. You get these 
sort of oscillating patterns. But it's self-correcting. And then my 
favorite pattern is from MetaFilter, which is: When we start seeing 
effects of scale, we shut off the new user page. "Someone mentions 
us in the press and how great we are? Bye!" That's a way of raising 
the bar, that's creating a threshold of participation. And anyone 
who bookmarks that page and says "You know, I really want to be in 
there; maybe I'll go back later," that's the kind of user MeFi wants 
to have.<<<

But metaphors are just metaphors, and each list actually has its own 
unique and mutable characteristics. Some of these characteristics 
come from the affordances of the technology used (blog vs newsgroup 
say), some from the membership, some from the moderation process, 
and some from the subject matter itself. And it is through the study 
of online communities that we may be able to answer Pippin's 
question about post quality. There are several research groups 
tackling online communities and there seem to be two broad 
approaches to understanding post quality. The first (most obvious) 
is to ask the members of the community what they think. Jenny Preece 
( http://www.is.umbc.edu/onlinecommunities/ ) often uses observation 
and questionnaires in her work to get at these kinds of issues. The 
second approach is to take a more analytical and abstract approach. 
Marc Smith's Netscan ( http://Netscan.research.microsoft.com/ ) does 
this so that one can see quality metrics emerge from the shape of 
the visualization of posters or of threads (e.g. people who only 
ever answer, people who only ever initiate threads etc). Apparently 
he has a bunch of visualization tools for mailing lists so one day 
I'll be running them on HPfGU and see what pops out.

There are fairly sophisticated models of post quality that do work. 
For example http://slashdot.org/ provides a mechanism for members to 
rate posts as worthy. The ratings themselves are also open to rating 
allowing key judges to emerge whose ratings are given more weight. 
You can filter so that you only see the posts that have been given a 
high ranking by worthy judges.

I guess I want to air one last point, and it is perhaps the most 
difficult to make here. I do not think that the Harry Potter books 
are truly great. What makes them great for me is a combination of 
factors: they are well written, they are gripping, they are 
unfinished, and they are popular. The last two ensure that we have 
(in HPfGU) a large group of thoughtful adults eager to propose and 
debate theories about how the books will pan out. There are 
interesting things for academics to write about HP, but I feel that 
the cultural phenomena is the deepest aspect for them. So if we are 
too high brow, to exclusive, too driven by post quality, we will 
loose that which is for me the essence of HP enjoyable discussions.

But even if I think many of the proposals for improving post quality 
are misguided, we are clearly going to be in big trouble following 
the publication in July of the next book. Sorry I haven't really 
helped move that discussion forward. While I'm apologising sorry 
that I haven't responded more to previous posts, or woven them in to 
what I've rambled about.

Cheers,

Dumbledad.







More information about the HPFGU-Feedback archive