Dale Lehman writes:
I'm sure you're all familiar with the World Values Survey, but I hadn't until I saw it described in The Economist this week (August 12, 2023). It has probably been used for many academics' careers and is a monumental effort to collect survey data on values from around the world over a long period of time (the most recent survey included approximately 130,000 respondents from at least 90 countries). ). Please note that I have no experience working with this data and have not read anything about its methodological development. I was struck by what appeared to be a shoddy and ill-conceived research effort. To start with the trivialities published in the Economist article, I have attached a screenshot of some of the content published in the print magazine (the online version builds this view interactively, so the print version is more complete, but provides less context). I have a problem with visualization – some may call it arrogance, but I call it a big problem – and I don’t know if the blame lies with The Economist or WVS. But that's what first alerted me to this problem. data. Changes in value over time are indicated by a line segment ending in a circle marker in the most recent survey waveform. Why didn't you use an arrow instead of a circle at one end? I think this is inexcusable. Arrows invoke careful up-front visual processing, whereas lines and circles require constant re-evaluation of the image to understand how things are changing. In other words, the visuals presented don't work. Arrows are much better. I don't believe this is just sloppiness. I think this reveals something more fundamental. That's what really concerns me about WVS.
As you progress through the graph, you quickly become troubled by the dimensions of the graph. The methodology is explained in detail on his WVS website (https://www.worldvaluessurvey.org/WVSContents.jsp), but I have not reviewed it in detail. However, there are many problems with these measurements. Among them are:
– The dimensions of survival and self-expression seem counterintuitive to me. Since related issues (such as the importance of religion and the importance of environmental protection) are related to wealth, and much of the WVS research is concerned with changes in values that accompany changes in wealth, why directly influence wealth? Why not measure it? My preference is for clearer (relatively speaking) indicators like GDP over these derived indicators, which seem vague to me.
– I have a similar problem with another aspect: traditional versus secular rationality. Neither seems intuitive to me, and the underlying question doesn't help things. There are questions about „national pride'' and self-description questions about whether you feel „very happy.'' I find it very difficult to understand how these map cleanly to the dimensions being used.
– These studies are done in many countries and over time, so I think the meaning of the words can change. For example, to ask whether people can be trusted, the concept of „trust“ must mean the same thing in different places and times. I can't see any evidence of this, but I can imagine that people interpret phrases like this differently. In general, the wording of these survey questions doesn't seem to be carefully thought out or tested (though maybe I'm just not familiar with the question development).
– I don’t feel comfortable using a single dot to represent an entire country. In fact, there is much debate about how heterogeneous countries are, but the graph uses an average measure to represent countries as a whole. As with many things, the average may not be as interesting as the variation. This concern is further emphasized by the aggregation of these countries into groups such as 'Protestant Europe' and 'Orthodox Europe'. These groups also don't seem particularly intuitive.
– I am not convinced that a two-dimensional view of value is the best way to analyze value. Are there two dimensions that are most important? Why two? Perhaps changes over time reveal how valid the dimensions are, rather than an essential change in the values people hold. It may be only.
There are others, but I want to stress again that I have no background in this data. The Economist found almost all descriptions cumbersome in terms of what was being measured and how it related to the basic methodology of this two-dimensional value system. I can say that it was difficult to even read the article. I'm also not sure how much of my concerns lie with the Economist article or his WVS itself. But there is an overwhelming sense that WVS is simply a vehicle for telling stories about how values differ between countries and groups of people, and how they change over time. It remained. Those stories are certainly interesting, but I don't think the methodology or data favors any particular story over another. This seems like the perfect mechanism for academic career development, but little else.
My answer is „I don't really know!“ I myself have never participated in a World Values Survey. Perhaps some of our readers can share their opinions.