Why I use the term “big data”

I do not like the term “big data” but I use it anyway, though not systematically. I share my reasons to the community of research engineers.

From the perspective of an engineer, big data is the kind of technical term that the marketing guys repurposed for their own needs. A bullshit word. As an engineer in the industry, you tolerate it because you understand its purpose: selling, which ultimately pays your high salary. But as a research engineer, you try to maintain a state of affairs where words have a meaning, and the term “big data” does not.

Big data had a meaning, and lost it. The term was coined in the nineties and referred to something specific: data sets too big for the common computer. Engineers had to develop entirely new technologies to overcome this practical limitation. This meaning is still relevant today. When data do not fit in a computer, we have to use a specific infrastructure (eg. cloud technologies), where normal strategies do not apply, which requires specific competences. These skills are still different from everyday computer science, which justifies a new field with its own name, “big data”.

In the private sector, every company started using the label “big data” because it made you look innovative, regardless of the actual “too big for a computer” problem. At some point engineers accepted it had become another bullshit word, and moved on.

In the social science, we have a different problem. Scholars started using “big data” when exotic competences became necessary to deal with digital traces. But exotic to them, not to engineers or computer scientists. In short, “big data” just meant “data science”. So someone with a list of items too long to read, and thus requiring computer processing, would call it “big data”. Which also conflicts with the original meaning. So research engineers stopped using the term, and moved on.

At some point the term had become a marker of non-academic language. The underlying reason is legit: its meaning is disputed between at least three different fields. Only someone ignoring that would use it, naively causing painful misunderstandings.

But from the techno-anthropology perspective, “big data” is an empirical thing. It is no less and no more than what people say it is. And in that sense, it has a definition. Ruppert and Scheele remark that “while there are many definitions, statisticians usually adopt what is commonly referred to as the “3Vs” of big data: huge in Volume, high in Velocity and diverse in Variety of types and formats of structured and unstructured data.”, citing a paper by Kitchin. Like them, I need the term to be able to observe that big data performs, enacts politics.

It makes sense to me to frame network analysis as part of the world of big data, because it enacts similar politics. Otherwise I like Kitchin’s characterization. In that sense, I am fine with defining “big data” as data that are at least one of the following:

  • Massive, ie. too big for usual devices
  • Highly granular, ie. representing a population of beings, non aggregated
  • Dynamic, ie. finely described over time
  • Relational, ie making networks

This definition is general enough that it can cover the three meanings, even if it disagrees on the definition boundaries. But empiricism can rarely afford strong demarcations, so I am happy to make this sacrifice in order to gain the ability to describe and reflect on how people use data. Even if it means restating my definition every time.


One thought on “Why I use the term “big data””

  1. Very insightful take, although you leave sampling out of your analysis. From a statistical perspective, “big data” tends to be associated with a set of techniques that eschew the requirement for sample-to-population inference (in the same way that machine learning is largely subsumable to “unsupervised” data analysis on “features” without “labels”). Most of the time, that definition will imply your engineer definition of data that require some form of distributed storage and computation.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.