Department for Education's survey is a 'disgrace'

Quite apart from whether a survey of parent views launched via Twitter can ever generate statistically valid information (what was the survey population, how representative were the replies, how do you know that DFE officials aren’t generating multiple versions of the answers their ministers want to hear?) this survey beggars belief for its lack of professionalism.

The 'financial analysis' of ratios that the DFE produced earlier in the year was bad enough, but this is a disgrace. It would fail any test of basic survey design. Let’s look at the questions in detail:

Question 1

Do you have a three- or four-year-old at nursery? We’d like to hear your views if your child or children are at a private nursery, school nursery or voluntary/charity nursery.

Is the survey just aimed at parents of 3 and 4 year olds (as implied by the first sentence) or to anyone with a child at nursery (as is implied by the second)?

Question 2

Do you know what the qualification levels are of staff in your child’s nursery?

Yes

No

I'm aware of some peoples' qualification levels

At least this asks an intelligible question. But it’s a pity about the misplaced apostrophe in people’s.  Are you there, Mr Gove?

Question 3

Do you know how many members of staff there should be for the number of children in your child’s nursery (the staff:child ratio)?

Yes

No

This question specifically refers to the children 'in the nursery', not just for the relevant age group (whatever that is). To answer it properly, parents will need to know the precise numbers in each age group, which can vary from day to day.

Question 4

If yes, what is the ratio? If no, what do you think it should be?

What about the people who think they know the answer but in fact don’t?  This question will produce a mixture of correct ratios, incorrect ratios and ratios that people would like to see. And, since we don’t know what the previous question means and therefore what the correct ratio is anyway, there will be no way of telling the difference.

Question 5

When choosing a nursery, did you make a decision based upon

Location

Cost

Numbers of staff

Qualification levels of staff

Other (please specify)

Basic error of survey design, respondents aren’t told whether to select one answer or all that apply. And who chose this particular list of features? What about all the other factors in choosing a nursery-safety, Ofsted report, reputation, recommendation, attitude of staff, to name just a few?

Question 6

Do you think it’s important to have staff qualified to graduate level e.g. Qualified Teacher Status, Early Years Professional Status or another suitable Level 6 qualification?

Yes

No

The phrase 'another suitable ….qualification' implies that the other qualifications listed are also suitable.  So the question suggests its own answer. Not to mention the ambiguity about whether the question is referring to 'all staff', 'some staff' or 'someone in the setting'.

Question 7

Would you ask the nursery manager to provide qualified staff?

Yes

No

What on earth does this mean?  Under what circumstances 'would you ask' the manager to do this?  How many qualified staff? What level of qualification? This is pure drivel.

Question 8

Does the number of staff compared to the number of children concern you?

Yes

No

‘Concern’ in what sense? Does this mean 'do you think it matters?' or 'are you worried about numbers in your nursery?'

Question 9

If staffing levels changed, would it influence you to change your child’s nursery?

Yes

No

Changed how? Up or down? By how much?

Question 10

Would you ask the nursery to provide more staff?

Yes

No

Again, under what circumstances would you ask this? You would hardly ask them to do this if staff numbers had gone up. So, assuming they had gone down, is asking them to provide more staff something you do before or after being 'influenced to change your child’s nursery', in the previous question? And why the subtle change in wording, previously you were asking the nursery manager, now it’s the nursery? Is this significant, or just the result of sloppy drafting?

Question 11

Do you look at the nursery’s Ofsted report? Which aspects in particular would concern you?

Facilities/building

Ratio of staff to children

Safety

What the children learn

Food quality

Toys/games/activities

Other (please specify)

'Do you look?' (factual question). 'Which… would concern you?' (hypothetical question). Same ambiguity over the meaning of ‘concern’.  Same question about what is, and is not, included in the list and why.  Same schoolboy error in not specifying whether this is single or multiple choice.