Streamlining question closure: approximate roadmap

Question

This is a rough outline of what I'd like to see in terms of closure reform. It will take time. It'll probably evolve with feedback as the process proceeds.

Stage 1: collate available information.

1. Survey old Chinese.SE meta posts about closure, and create a summary of what has been said thus far; what questions are on/off topic, and what noteworthy arguments have been made for/against closure. There's only 271 questions on meta, so I think I can just go through all of them manually, and summarize the pertinent information.

   Update: July 2020: survey of the on/off-topic posts on Chinese.SE meta and July 2020: survey of comments on questions containing the substring "close" (much discussion has taken place in comments).

2. Survey the other language sites. We can learn from other sites when deciding what's on/off topic, and how to handle borderline cases: what works, and what doesn't work. We don't need to reinvent the wheel (particularly wheels that didn't work properly).

   Update: August 2020: survey of the off-topic close reasons at language-related Stack Exchange sites

3. Discuss when to migrate to other sites. An underutilized feature at Chinese.SE is question migration. Sometimes we should be migrating to sites like Language Learning and Japanese and Korean, but this requires understanding what's on/off-topic at their site. It's part of my motivation for whipping up Alternative Stack Exchange sites for “not really about the Chinese language” questions. Furthermore, if a question can/cannot be adequately answered at another SE site, it influences whether or not we should declare it off-topic here.

   Update: Which topics should be redirected to other SE sites?

4. Discuss lowering closure to 3 votes (from 5 votes). Moved to stage 2.

Stage 2: re-affirm community consensus.

1. Re-ask "is this on/off-topic?". The aim is to reach an up-to-date community consensus as much as possible; users are informed about when to close, and can easily link to the relevant meta posts.

   In particular, we need individual meta posts for clear-cut on/off-topic reasons. In this way, users can comment The community declared "watermelons" off-topic. linking to the Chinese.SE meta post where consensus is established, and subsequent users can easily verify this is correct and contribute their close votes. Likewise if someone votes to close, they can instead comment The community declared "watermelons" on-topic. so subsequent users know not to vote to close (or vote to reopen).

   And, to be blunt, the relevant 8-year-old meta posts are obsolete: maybe we have different opinions to back then. A lot of other things have also changed in the meantime (e.g. being more welcoming, new SE sites).

2. Flesh out boundary issues. For some topics, we may not get a consensus for on/off-topic-ness. In this case, we need to identify the boundaries: what are the two sides of the story? Basically, we break these decisions up into sub-problems on which we can establish consensus.

   We definitely need to explore in depth:

   1. What resource exists for [blah]?
   2. What does this say?
   3. How should I study Chinese? (Update: Touched on in my answer here.)
   4. What's a good [book] for [blah]?
   5. Is [three random Chinese characters] a good Chinese name?
   6. (What does "show effort" actually mean?)

   Update: Asked thus far:

   1. Are image transcription requests on-topic? ("What does this image say?") and 17 examples of image-transcription requests: how on/off-topic are they?

3. Discuss lowering closure to 3 votes (from 5 votes). To my knowledge, we can change this threshold, and some sites are changing it to 3. I don't believe it has to be 3; e.g. it could be 4 or 6 or whatever. I'm not keen on doing this if we don't have to, but maybe we could do it temporarily. [Also what happens to reopen votes?]

Stage 3: re-inform the community.

1. Create informational notices. Some users are not active on meta, yet are perfectly able and willing to curate the site. We should create for them a kind of "cheat sheet": a summary of the major points they need to intelligently close/leave-open/reopen questions most of the time.

   I get the feeling that it would be worthwhile clarifying when to use close reasons like "Needs details or clarity", "Needs more focus", and "Opinion-based". Also I think the community could be better informed about downvoting and autodeletion.

   Also we should come up with ways to guide users who are unfamiliar with the site into writing better questions. Perhaps via "question templates".

2. Update the on-topic page. In my opinion, the current on-topic page is outdated, and there is way too much to read: people on the Internet don't have such a long attention span. You can see a draft I whipped up at Islam.SE to check out my writing style. In particular, note the succinct dot points, and links to consensus posts (and discussions of boundary points) and direct quotes from meta.

3. Update the "Draft your question" notice. This is the notice shown next to the Ask a Question box.

   Step 1: Draft your question

   The community is here to help you with questions about Chinese language. Provide details and share research with your question.

   Helpful links

   Find more information about how to ask a good question here

   Visit the help center

   Ask questions about the site on meta

   I feel like this is currently a missed opportunity: Chinese.SE-tailored information should be right there.

Stage 4: update the closure reasons.

Update: I brought this up here: Let's look at replacing the custom close reasons. I ended up asking about this earlier than planned: if we flesh out every boundary case, we might never reach this stage.

The current close reasons are:

1. Questions asking for translations are off-topic unless prior research effort is clearly indicated; we're here to help you learn, not provide a bulk translation service.

   For starters, we should certainly delete the snarky comment at the end: they're just asking one good-faith-assumed question. Instead, we should be providing guidelines to the community's expectations, and resources to help them meet those expectations.

   My experience here is that there is maybe 90% exception 10% rule here.

   I feel we should avoid saying "XYZ is off-topic" in favor of saying "The community has declared XYZ off-topic", linking to the relevant meta post.

2. Questions asking for learning resources are off-topic as they can become quickly obsolete and are difficult to maintain. We do maintain a list of generally-useful resources for learning Mandarin that you might find helpful.

   Again 90% exception and 10% rule. Also, why is it "learning resources" and not just "resources". And why do we only provide a link for Mandarin?

3. This question belongs on another site in the Stack Exchange network

   We need to populate this with other SE sites: which sites should be determined by the community via a meta post.

Also, would it hurt to say something like "while we appreciate your contribution, ...", etc.?

Stage 5: systematically apply the streamlined close reasons

Once we've clarified what is on/off-topic, it's time to systematically go through Chinese.SE questions and apply the close/reopen reasons. This will likely take a long time, and will likely need to be synchronized with editing (such as retagging, and improving titles).

There are 8251 questions. Perhaps we can split this into 82 batches of ~100 questions in a meta post. Users can "reserve" a batch, and mark it as "done" after verifying those 100 questions have been appropriate closed/reopened.

And hopefully after all this... we have streamlined closure reasons, which are consistently applied, and people generally know when to vote to close.

Answer 1

Thank you for this write-up.

I have only one remark about Stage 1, Point 4:

Discuss lowering closure to 3 votes (from 5 votes)

I suggest we move this to Stage 2, as, in my view, it depends on how hard it is to form a consensus about what's on/off-topic.

If there is loose consensus, lowering the close vote threshold might be construed as contrary to self-moderation principles of Stack Exchange, i.e. anti-democratic.

For the sake of the argument, if the community is split, say, 60-40 on certain close reasons, reducing the threshold might effectively invalidate the opinion of the 40%, while those opinions represent almost half the total, and more easily lead to misunderstandings ("That question shouldn't have been closed!"). And more complaining on meta.

Instead if there is strong consensus, lowering the threshold is sensible, as it allows more active members to close questions effectively.

It boils down to the following point: how much are active users, who do cast close votes, representative of the opinions of the community?

If the answer is "not so much", the 5 votes threshold is fair.

If the answer is "very", the threshold can be lowered.