In 2005, I was a member of a research team studying gender and occupational stratification in Naples, Italy. Previous studies had relied on a ‘measure’ of occupational stratification whereby individuals were asked to grade, or judge, occupations on a one-dimensional continuum scale. No previous research had investigated whether such judgements about occupations would modify according to the gender of the occupations whose status was to be judged. This case in methodology provides an account of 3-year-long research, giving the reader an insight into some specific methodological problems that arose in the course of the research. The case focuses on the main challenges in using the Thurstone scale to construct a valid and reliable occupational stratification scheme and in testing the stability of such scheme. These challenges refer to sampling, methods of administration of questionnaire and testing techniques. Thinking about such challenges can lead to a more general consideration of the relationship between survey and experimentation in social research. Particular attention is paid to the use of non-probability samples, face-to-face interviews and non-parametric tests as a way of controlling validity of survey data collection instruments.