As you may or may not know, the British Standards Institute have now published the 2nd draft of the BS8878 British Code of Practice on Web Accessibility.

Briefly, the standard is designed to be an end-to-end guide to helping web product (website, mobile site, IPTV site) owners ensure their products condier the needs of disabled and elderly people at all stage of the web production process, from initial requirements gathering, through selection of technologies and platforms, testing, launch and maintenance.

It’s a standard for web producers, project and product managers, who need to understand the process of how to create accessible web products, rather than how to create the technology, design and editorial of web products (which WCAG already does very well).

It will supercede the previous PAS-78 guidelines, updating them to include guidance on such topics as:

• The Equality Act 2010
• The relationship between inclusive design and user-personalised approaches to accessibility, including whether to provide additional accessibility provisions
• Creating accessible web products for computer, mobile and IPTV platforms
• How to procure accessible web products
• How to assure a product’s accessibility throughout the production process, including the value and costs of different forms of accessibility testing
• Dealing with feedback and complaints on accessibility from users

The standard is a guide to the use of the different accessibility guidelines out there (bringing together references to the work of WAI-WCAG and WAI-AGE, as well as others), rather than replacing or replicating those existing guidelines.

This Draft for Public Consultation is a chance for people to tell us whether our drafting committee have got the content and style of the standard right and both given them an idea of the breadth of the parts of their production process that accessibility issues impact, and also demystified accessibility so they feel confident they know how to proceed.

DPC BS 8878 can be viewed at www.bsigroup.com/drafts until 30 June 2010.

Jonathan Hassell

I certainly aim to pop along and comment, and I’d urge others to as well.

Never assume. It makes an ass of u and me.

Anon

It may be hackneyed, but it does ring true for many of the observations I’ve made around surveys. Here’s my list of the 10 most common assumptions to avoid when conducting an online survey.

1. Never assume prior knowledge

An obvious one, really, but I’ve seen surveys use acronyms and jargon that I have no clue about, and then ask for comments on them. Without explaining the terms of reference, you can’t possibly expect to get useful feedback.

2. Never assume that views are polarised

Again, hopefully an obvious one but I’m often asked to create a survey where the answers are:

• Strongly agree
• Agree
• Disagree
• Strongly disagree

This is clearly missing an option – “Don’t know” or perhaps “Neither agree nor disagree”. But those who conduct surveys often don’t want neutral results, and try to omit this middle option to force people off the fence. This will, of course, skew the results and possibly even deter people from completing the survey.

3. Never assume people will read the instructions

This one has become painfully apparent recently. You can make the instructions as clear as humanly possible, but you can’t force people to actually read them. Experience has shown that people will often skip straight to the questions, disregarding any introduction or specific instruction. That means you can’t rely on people reading about the special rules and conditions that they need to know before completing the survey, and actually you should try to design the survey in such a way that doesn’t require instructions in the first place.

4. Never assume people’s response

I’ve seen a lot of surveys which appear to infer the respondent’s opinion before even asking them. People hate leading and loaded questions, and this is especially crucial to avoid if the survey is part of an important consultative process. Keep it neutral, and consider whether the survey supports both negative and positive responses with equal weight.

Running a survey with an expectation that it will back up a previously held assumption is particularly dangerous. If the results come back differently it could seriously derail your plans. Be genuine in your use of surveys and build in the capacity to actually react to the findings. A token, meaningless survey, done just to tick boxes, will often generate more anger than running no survey at all.

5. Never assume people will finish

The completion rate of a survey varies wildly according to many factors, not least of which are the length, complexity and relevance of the questions being asked. If you have questions which you especially need answers to, consider putting them towards the start, to avoid them being missed by those who lose interest half way through. Equally, if there are questions you don’t really need to ask, consider dropping them completely.

6. Never assume people will identify themselves freely

I’m often asked to make questions about personal details mandatory – so that we force people to give us their contact details, for example. Of course, this is always requested with the best intentions – it enables individual comments to be followed up and concerns addressed. But it is also completely unacceptable. In almost all instances, you should be giving the user the choice of responding anonymously. In fact, data protection laws say that you shouldn’t collect anything more than you need anyway, so you’ll need a pretty good reason to ask for these details. And if you still insist on doing so, explain to people why you need the details and what you’ll do with them, linking it to your privacy policy.

7. Never assume people will take it seriously

Is your survey idiot-proof? In other words, if someone took it upon themselves to target your survey with silly or disingenuous responses, would that ruin the results?

Any survey should come with certain caveats about its accuracy and coverage, and as I’ve blogged before, the more controversial surveys are big targets for those with devious intentions. Multiple repeat submissions and false replies can seriously skew your results, and are common if people have a strong agenda relating to the topic of the survey.

8. Never assume everyone can access it

Not everyone has access to the internet, so you may need to make alternative arrangements to gather offline responses. You might want to circulate paper copies of a survey, for example. Be aware, though, that you’ll probably need to dedicate some time to entering all of those paper copies into the electronic version, to allow the responses to be fully collated.

There’s also the issue of web accessibility – how easily the online survey can be used by individuals with disabilities or impairments. There’s some quite useful research on the accessibility of survey tools, although it’s from 2008. This is a crucial subject, as failure to make a survey accessible could mean you are discriminating against individuals.

9. Never assume that the survey is the end

Some people will happily complete a survey then forget about it. Others, however, will expect to see action. They may wish to see the results, or a summary of the main findings. They may want to know what you’re doing in response. Surveys are not a standalone process – they should be part of a wider consultative or research strategy.

10. Never assume you actually need a survey

Surveys are very easy to create nowadays, with a wide variety of free and user-friendly tools available. Hopefully the above points highlight the amount of thought that should be going into each and every survey you run. And you should also always ask yourself whether you actually need to run a survey. I’ve seen real evidence of ‘survey-fatigue’, where hitting people with multiple requests for feedback over a short period of time will seriously dent the response rate. Keep them simple, keep them relevant and keep them to a minimum.

HUWY

Earlier today I went along to the e-Science Institute in Edinburgh to attend the launch of a new project aimed at getting young people talking about policies and laws which affect the Internet, to channel their ideas to the policy makers.

According to the delegate pack, the HUWY (Hub Websites for Youth Participation) project believes that young people are valuable expert stakeholders in current Internet governance issues like:

• cyberbullying
• child abuse and child safety
• freedom of speech and censorship
• privacy and phishing, security, identity, hacking, e-commerce
• file-sharing and copyright

The project has partners from the UK, Germany, Estonia and the Republic of Ireland, as is sponsored by the European Commission. At this morning’s launch event we heard from some of the partners about the work they are already involved in, including online networks of over 100,000 young media makers in Germany (Jugendpresse Deutschland), platforms to allow young people to have frank discussions about everyday issues (Youth Net), and sites supporting citizenship and digital engagement (Young Scot).

We also saw videos and messages from key officials who were unable to attend the event. Hille Hinsberg (from the Estonian State Chancellory) spoke about the need for bigger take up of engagement opportunities in policy making, saying that all too often it takes a scandal before we are moved to discuss such issues. She praised the value that HUWY will bring to our efforts to listen to young people, finding out what they want to discuss, what their concerns are and, critically, what their solutions are.

In another video, German MEP Matthias Groote spoke about the importance of involving young people, and that the Internet is an essential tool in allowing them to participate actively and creatively. He described the huge potential which we “absolutely have to use”.

Sadly I couldn’t stay for the afternoon, which featured workshops on how young people use the Internet and how we translate engagement into change. Nevertheless, I’m excited about what the project will teach us about engaging with young people and using online platforms to review policies and effect political change.

More information

• An introduction to the HUWY project
• HUWY blog post about the launch event
• To get involved in the UK project, contact Feargal O’Kane at Queen’s University Management School, Belfast.
• For more info about the international project, contact Ella Taylor-Smith at the International Teledemocracy Centre, Edinburgh Napier University

Biased results are a risk of online polls

I’ve just updated my list of social media lessons learned the hard way with details of an online poll which appears to have backfired.

In summary, part of a multi-million pound advertising campaign by Christian charity Alpha International has potentially backfired when an online poll on their website, asking whether people believed in God, showed an abnormally high 98% saying ‘No’ (source: The Register).

Alpha International has suggested that the skewed results are down to an online sting, with Spokesman Mark Elsdon-Dew adding: “I don’t think this is indicative of people’s faith in this country.” This seems highly likely – especially as the poll allows multiple (in fact unlimited) responses on the same computer. Many online polls use cookies, IP logging or such to prevent people from responding to a poll more than once – on each subsequent visit to the poll page, the user would normally be taken straight to the results page instead. Without this, the accuracy of your poll is at risk from repeat responses.

It’s not clear whether the skewed results were the result of an automated sting, using software to generate high numbers of repeated responses, or a manual sabotage, with willing individuals logging on and submitting multiple responses. It seems probable that it was one or the other – it’s unrealistic to expect that kind of response from the normal users of such a site, who are more likely to be in the ‘Yes’ or ‘Probably’ camps.

Organised sabotage?

A quick Google search reveals all sorts of online discussion around the poll, one of which explicitly invites fellow members to head over to the poll and “make it look even worse”. Within 2 days, 80 people had replied to that post, one of whom had spotted the technical flaw in the poll:

Ooooo – it lets you vote more than once. Vote early and vote often folks!

Other more productive replies talk about the lack of decent response options, with one person pointing out:

So they have “yes”, “no”, and “probably”… where’s “probably not”?

(to which I’d also add a middle option of “Don’t know”)

Luckily for Alpha International, the completely inaccurate results are obvious to all, potentially lessening the damage done. But, it has to be asked, what was the point of the survey in the first place? What would the ideal response have been for Alpha International?

This is a crucial question when considering online polls. Are we using them for real fact-finding, or are we just trying to prove a point? I recently saw a site which asked for parent’s opinions about local school closures. For me, this seemed a little unwise. A strongly negative response may have adversely affected any attempt by the authority to close schools, whilst a strongly positive response would have seemed rigged. That site did at least employ cookies to limit multiple responses, but with such an emotive subject it wouldn’t be hard for campaigners to direct vast numbers of people to the poll to make their opinions heard.

Quantity vs quality

And there is another risk of online polls, which is the lack of qualitative reinforcement. Quantitative figures can only show us some of a picture – if we are really trying to gage public opinion we should be entering into discussions, teasing out issues which we may not have thought of, and possibly even turning around some of the negative responses by offering valid counter-arguments and supporting information.

In conclusion

The Alpha poll is a perfect example of how over-simplified, under-restricted public polls can seriously backfire. They can be handy for giving people a quick and easy way of starting to engage in a discussion, but only really serve as the start of such a process. Adding forum functionality, to allow people to qualify their response, is an ideal way of taking this further, allowing pollsters to engage and discuss. Without this, polls are at best a fairly worthless set of figures, and at worst a PR nightmare waiting to happen.

Although I can understand that it is useful to receive submissions as early as possible, to allow staff to manage their workloads, it seems a bit unreasonable to present a false deadline which is then discretely extended. Many people will have put themselves to great inconvenience to complete this form in time, especially given the time of year, and to find that the deadline has been extended by almost three months will no doubt cause annoyance. There is also the risk that citizens will stop taking deadlines seriously, and may miss the window of opportunity in the future, where stated deadlines are genuine.

Far better, perhaps, to give the genuine deadline, but state that early submission is recommended. In some cases (for example, applying for grants from a limited fund) you could even suggest that early submissions will receive preference – that should get people moving!

Consultations

Similar issues have dogged some recent consultations that we’ve held, with deadlines being pushed back and back to try to squeeze out more responses. The problem here is that those who aimed for the original deadline may not have given themselves sufficient time to compose a full and accurate response. Those who replied early on are likely to be those who feel strongest about the subject of the consultation, and therefore the kind of people you want to listen to carefully and not annoy.

Of course, there are some instances where extending a deadline is sensible. In a consultation where new information has come to light, for example, or where there has been a technical problem preventing people from completing the process. In such cases, the reason for the extension should always be made clear. Otherwise, the organisation risks looking unorganised and unprofessional.

Project deadlines

The same applies to project deadlines. I recently finished a job which had to be completed by a certain date. No problem there – I’m used to tight deadlines. But when the delivery day arrived, the client came back with a few tweaks and some new requests, and it turned out the original deadline wasn’t as crucial as first made out. These fake deadlines don’t do the developer any favours – many things may get rushed or dropped entirely as a result. Also, it’s usually a lot harder to change a final product, rather than factor in any modifications as part of the build process. It’s therefore far more productive to set milestones, where you deliver certain things by certain timescales. This allows for a far smoother progression from planning to the final product.