I would like to thank the participants of the conference ‘Measuring and monitoring implementation of the SDGs: Fit for purpose?’ held at the Graduate Institute in Geneva on June 9, 2016. The interesting and insightful presentations and discussions have decisively influenced this article. I am also deeply indebted to my collaborators, in particular David Cunningham, David Backer, Erin McGrath, Eric Dunford and Cody Buntain, with whom I have worked on this topic over the past years.
1Big data is seen by many in the policy community as a powerful new tool particularly suited to helping us understand the pressing political and societal questions of our time. This includes, in particular, the application of big data approaches to the measurement and monitoring of the implementation of the Sustainable Development Goals (SDGs). Yet, at the same time there exists no general consensus on what big data actually entails. And while the weaknesses of big data approaches have recently been intensely scrutinised (e.g. Ruths and Pfeffer, 2014; Jungherr and Theocharis, 2017), less attention has been paid to how exactly they could be used to address highly policy-relevant questions.
2This article focuses on one such topic with high policy relevance—monitoring political instability—that is closely related to SDG 16: Peace, Justice and Strong Institutions. Since the second half of the twentieth century, violent political instability within countries has replaced interstate wars as the most dominant form of conflict (Fearon and Laitin, 2003). The escalation of peaceful protests in Syria into a major civil war with international involvement illustrates the drastic consequences of political instability for a country’s economy, society and the safety of its citizens. From the perspective of both policymakers and practitioners, it is therefore of central importance to arrive at a better understanding of the trajectories of political instability within countries or regions; if possible, detecting early-warning signs as to where and when it arises in order to allow for more timely responses and interventions. The question, then, is what exactly can big data approaches contribute in this context?
3It is instructive to first approach this question through the broader, historical lens of big data research, its innovations and its missteps. The Google Flu Trends (GFT) project (Ginsberg et al., 2009), one of the early success stories of big data research and prominently cited when discussing the promise of big data approaches (see Mayer-Schönberger and Cukier, 2013), may here serve as an example. The project’s innovative idea was to use the volume of flu-specific Google search queries to predict flu epidemics. Initial evaluations indeed demonstrated that the tool, solely based on internet search term volume, achieved predictions better or on a par with reports from the US Centers for Disease Control and Prevention (CDC) but in a much timelier manner. Yet, subsequent analyses revealed that, starting in early 2011, the Google-derived measurement systematically over-predicted prevalence of flu (Lazer et al., 2014). Not even a decade after its original publication, the tool has now been discontinued. The idea of using human flu-related search activity to predict the occurrence of flu is intuitive and convincing and there is important, highly granular information coded in the search activity of millions of people. What then can lead big data measurements such as GFT astray?
4David Lazer and colleagues refer to it as ‘Big data hubris’ (Lazer et al., 2014, 1203), the assumption that big data can completely replace rather than supplement traditional means of data collection and analysis. As in the example of GFT, big data are often true artefacts of human interactions and not carefully collected data sets following systematic, stable coding protocols. In fact, big data sources are no less prone to bias; typically they are more so, not least because they are dependent on ever changing technological systems and the algorithms that produce them (Ruths and Pfeffer, 2014). Having much more data does, in many cases, simply not make up for the fact that these data might be very biased, noisy, or invalid measurements of the variables of interest. In the example of GFT, Google search terms may be an excellent proxy for whether people are concerned with flu or not; but in the case of a media-induced panic, concern ceases to be a good measure of actual flu prevalence (Butler, 2013; Copeland et al., 2013).
5Big data thus have to be used with at least the same level of care and methodological rigor as any other empirical data, carefully considering their strengths and limitations. This requires a deep, qualitative understanding of both the empirical setting and the specific data used. Ideally, it also entails systematic tests of construct validity, and in-depth empirical validation that goes beyond simple correlations. If done correctly, the volume, velocity and richness of these novel data then enable analyses that are otherwise not possible (Lazer et al., 2009; Vespignani, 2009; King, 2011). Yet this typically does not, and should not, involve replacing existing traditional measurements and approaches. It rather amounts to enriching existing analyses using data that complement empirical measurement where traditional measures are absent or inadequate. To stay with the example of GFT, why not use the search term-derived measure to improve CDC predictions rather than replace them—that is to say, leverage its superior spatial and temporal granularity (Lazer et al., 2014)?
6Research on conflict and political instability in political science was early in recognising and exploring the value of the automatised analysis of very large data sets. The recent big data approaches to monitoring political instability discussed in this article follow this tradition but also carefully incorporate lessons learned from prior work, fully recognising the strengths and weaknesses of the data and methods they rely on. They are thus taking important first steps in the direction of robust methodologies that can be responsibly used for policy-relevant analyses.
7The remainder of the article is structured as follows. The article’s next section clarifies what constitutes big data, before turning to an in-depth discussion of advantages and challenges in the specific context of monitoring political instability in the following section. The last section then introduces a general framework for monitoring political instability that represents best practices derived from current big data research on this topic. The article concludes with a discussion of the broader relevance of big data for the measurement and monitoring of the implementation of the SDGs.
8The term ‘big data’ is often used without a clear definition. In fact, what different people and different disciplines consider to be big data differs widely. When, in 2014, the School of Information at the University of Berkeley asked over forty ‘thought leaders’ for their definition of big data, they received as many definitions as answers (Dutcher, 2014). But, the contributions did highlight a number of important common characteristics of big data: their size, complexity and granularity; the specific approaches associated with these data, including technical prerequisites for handling them; and the variety and diversity of sources. These commonalities can broadly be grouped into two complementary definitions—one technical, the other conceptual—that capture key aspects of what characterises big data.
9A generally accepted technical definition, following the Oxford English Dictionary, is that big data constitutes ‘data of a very large size, typically to the extent that its manipulation and management present significant logistical challenges’ (OED, 2016). In other words, in this definition big data are any data that require specialised hardware infrastructure and software solutions if they are to be collected, stored and analysed. Given that these hardware and software solutions are constantly improving, this definition is relative of course. But it highlights an important distinction from classical research data: big data is not only typically too large and complex to store on a single computer, but also much too large to ever be inspected or analysed manually. And there is a further, implicit technical assumption: big data is machine-readable data—something that can, at least in principle, be analysed using automatised, computerised processing.
10A complementary conceptual definition is that big data are massive, very complex data—on artefacts of human interactions—that are of varying quality and are drawn from a large variety of sources. Typically, these data are collected without a specific research question in mind—that is to say, they are ‘found data’ (Connelly et al., 2016, 3), and are then analysed post hoc to identify relevant predictors or measurements. It is important to contrast these conceptual aspects with the technical prerequisites highlighted above: challenges in the management of big data typically by now have well-established (technical) solutions, whereas issues related to data quality, sample properties, or the generation and validation of robust measurements often do not. In fact, many of the measurement issues associated with big data, broadly speaking, are very akin to issues related to traditional research data, as discussed in more detail in the next section. Given these definitions, what then constitutes big data?
11Social media data, Twitter and Facebook data in particular, are big data and so are the metadata—call logs and cell tower data—collected by cell phone providers, or the logs of search activity on Google (e.g. Ginsberg et al., 2009). Similarly, most wireless routers, but also many networked devices, generate detailed activity logs that are stored and analysed by the companies and institutions that use or produce these devices (e.g. Bessis and Dobre, 2014). Such data are true artefacts of human interactions—that is to say, they are generated as a by-product of the proper functioning of technological infrastructure and are originally not intended for any further use. Data on money transfers—between banks or individuals—or on credit card activities (de Montjoye et al., 2015), the trading logs of stock exchanges (Seddon and Curry, 2017) and the tracking of cargo shipments and automatised inventory logs (Waller and Fawcett, 2013) are similarly artefacts of the technological systems that provide these infrastructures.
12In essence, any time a task is automatised, computerised or executed with the help of technology, such digital trace data are generated. They are big data not just because they are massive and require specialised infrastructures in order to collect, store and analyse them, but also because they are highly complex, noisy data from very diverse sources and are not collected for specific research purposes. The majority of recent big data studies in the social sciences build upon these kinds of data (Metzler et al., 2016), but they may, in fact, be challenging to use in the context of monitoring political instability because of issues of accessibility or even basic availability in settings where technological infrastructures are much less developed. Their potential and limitations are discussed in more detail in Section 3 and, mindful of these, they feature prominently in the framework for monitoring political instability discussed in Section 4.
13There are two other types of big data that are similarly important but often overlooked: remote sensing data and administrative data.
14Remote sensing data are typically generated through satellites or airborne sensors that sweep regions of the globe generating high-resolution images. Different sensors capture different aspects of the local situation on the ground: for example, radar missions to map local topographies (Jarvis et al., 2008), high resolution daytime imagery to capture local structures and their changes (UNITAR, 2014), or night light emissions measurements as a measure of local economic activity (Weidmann and Schutte, 2017). These data tend to be very large, requiring specific hardware and software infrastructures for storage, manipulation and analysis. They are also often updated at very short intervals; for example, as regularly as at 3‒6 hour intervals when using weather satellites for flood monitoring (Toté et al., 2015).1 Remote sensing missions also usually rely on an array of different measurement devices, constituting complex and often very noisy data from various different sources.
15These data are especially useful in situations where on-the-ground reporting is infeasible due to a lack of infrastructure or to challenging political and security contexts. They are thus particularly important in the context of monitoring political instability; for example, remotely monitoring risk factors such as vegetation changes (Thenkabail et al., 2012) or directly capturing major population movements and the impact of conflict (Witmer, 2015). In the framework discussed in Section 4, remote sensing data thus features as one key layer of highly resolved information with a coverage that extends to politically volatile regions and contexts.
16Big administrative data are, broadly speaking, data that arise as artefacts of the proper functioning of administrative systems (Connelly et al., 2016). This includes, for example, data on matters as diverse as construction permits, records of high school exams, or public health inspections. These data are usually not collected with a specific research or policy question in mind, but rather constitute detailed logs intended to monitor and track the proper functioning of an administrative system. In this sense, they are very similar to the artefacts of technological systems discussed above, but their volume and velocity are typically much smaller. Even if all data from every local administration were consolidated in one data set, the total volume of data would tend to be much smaller than that generated by the countless interactions on, for example, Twitter every day. And data are updated as administrative processes are set in motion or are completed—that is, on a timescale of days, weeks, months or even years as compared to seconds, minutes or hours.
17In many countries, though, administrative systems are not fully computerised, and the different systems are not necessarily integrated with one another. In other words, even in the global North not all administrative data may be in a machine-readable format and it may be distributed across different systems using different collection procedures, data formats etc. These issues are exacerbated in countries with weaker institutional capacity. In addition, even if data are available, incentives to publicise them may be very low—especially in politically volatile settings. The potential applications of these data in the context of monitoring political instability are consequently rather limited. Instead, in developing countries they often have to be replaced by more indirect measures; for example, using cell phone data to approximate local wealth distributions (Blumenstock et al., 2015).
- 2 John Akred as cited in Dutcher (2014).
18Big data thus refers to very diverse and large (digital) data sources that enable the proximate measurement of relevant empirical processes. But it is more than just data; rather, ‘“big data” refers to a combination of an approach to informing decision making with analytical insight derived from data, and a set of enabling technologies that enable that insight to be economically derived from at times very large, diverse sources of data.’2 But how does this approach fare in the context of monitoring political instability? The previous section already raised a number of potential issues regarding the availability of data itself and the applicability of big data approaches in politically volatile contexts. The question is thus not only which novel big data measurements can we derive in principle, but—more importantly—where in practice this is feasible and how robust, accurate and useful these measurements are.
19The fundamental advantages of big data over more classical research data are often summarised as volume, variety and velocity (Laney, 2001). In many cases, big data additionally have a much higher spatial granularity than more classical research data. These characteristics, in particular, permit superior quantitative renderings of certain aspects of societal processes, but not of society as a whole. They thus typically do not replace more traditional research data. Instead, they complement them with more detailed, richer and more granular coverage of select processes.
- 3 Facebook data are typically much more limited compared to data available from Twitter.
20One of the most defining features of big data is its scale. The typical examples of this are social media sites such as Facebook or Twitter, where every interaction, mouse click, and message is logged and can thus be analysed.3 Correspondingly, data volumes collected from these sites tend to be very large. The same is true for many other technical systems in which all, or close to all, aspects of a given system can be monitored. This, in principle, gives researchers the opportunity to study specific system behaviour in great detail, even focusing on certain aspects post hoc that may not have seemed relevant at the time of collection but are nonetheless captured in these data.
21 The second key characteristic associated with big data is its variety—the previous section has already highlighted the breadth and wealth of information collected. Even within a given system, very diverse data might be reported. Social media services, for example, not only track the content of conversations but also their metadata. Together these data thus have the potential to provide a much more nuanced picture of user behaviour than any one of them could alone. This variety promises to provide a much more complete picture of societal processes than, for example, a survey covering just one specific aspect and administered for only a limited period of time.
22This alludes to the third key characteristic of big data: velocity—that is, the speed at which the data are collected or updated. As already mentioned above, this is most pertinent to true artefacts of human interactions and applies, to some degree, to remote sensing data—administrative data are typically collected at a much lower frequency. Consider social science data collected traditionally by research initiatives or institutions, data that are updated at most at yearly intervals. Census data, for example, typically undergo at least yearly revisions but are only updated comprehensively every ten years in most countries. In many regions of interest—in particular in relation to monitoring political instability—the lack of institutional capacity may mean much less frequent data collection or may limit the degree to which detailed data can be collected in the first place.
23Big data, in comparison, is updated at a much higher frequency. For true artefacts of human interactions new data are continuously collected, whether these are tweets, Facebook messages, or metadata on telephone calls. Combined with its comprehensive coverage, such data thus provide a near uninterrupted record of human interactions—in a given system or using a given technical infrastructure. This allows one to trace out social processes at the time scale on which they actually occur, a critical prerequisite for identifying salient causal relationships. Research on social influence, for example, has profited greatly from the ability to track social interactions in near-real time (e.g. Bond et al., 2012). Similarly, the example of near-real time rainfall monitoring using remote sensing, as mentioned above, illustrates the added value that the high velocity at which big data is collected provides.
24One further characteristic of big data—aside from the three Vs—that provides unique leverage for the analysis of social processes is its spatial granularity. For example, mobile phone mobility traces provide a unique, fine-grained recording of the location of many individual mobile phone users. These data are typically much more highly resolved, often even to a higher resolution than the smallest statistical units in the global North. If correlated with census information, these trace data can thus augment existing measurements of population densities and significantly increase their spatial resolution (Deville et al., 2014), thus providing uniquely fine-grained renderings of spatial dynamics.
25How then can big data and its unique features be leveraged for the monitoring of political instability? There are at least three distinct contributions identified in the literature on political instability and (intrastate) conflict. The first relates to the use of remote sensing techniques to infer local economic conditions—a key risk factor for the onset of instability (Buhaug et al., 2011). These approaches make use of satellite data on night light emissions (NGDC, 2014), data that exist at high spatial resolutions and feature comprehensive, up-to-date coverage of regions where other data on local economic conditions are often of low quality or simply non-existent.
26Research on national and subnational scales has previously demonstrated a strong correlation between night light emissions and economic activity in developing countries (Elvidge et al., 1997; Henderson et al., 2011; Sutton et al., 2007). The recent work by Weidmann and Schutte (2017) uses comprehensive data from the Demographic and Health Surveys (DHS) to examine and explicitly validate this relationship in 39 developing countries. The study confirms that night light emissions are accurate predictors of economic wealth—also at a detailed, subnational level. This is true both for identifying new locations in a known country and for generating predictions for previously unstudied countries and locations. In a similar vein, remote sensing has also been used to monitor environmental changes such as changes in vegetation (Thenkabail et al., 2012) that may be indicative of impeding environmental shocks and their implications for political instability (Hsiang et al., 2013).
27While such indicators based on remote sensing technologies are usually imperfect, they can provide excellent proxies for local conditions in the absence of other highly granular statistics collected on the ground. These data are thus usually not meant to replace other more reliable measurements but can serve as highly granular subnational proxies for key covariates. By doing so, they provide invaluable insights into countries or regions where the local political or security situation does not otherwise permit the regular and/or granular collection of comparable statistics.
28The second contribution of big data for monitoring political instability relates to the measurement of instability events using machine-coded data sets such as ICEWS, GDELT or SPEED.4 Relying on the automated analysis of very large quantities of media articles, these projects aim to trace conflict and conflict-related events around the world. The data they produce are publicly accessible in close to real time—a significant advantage over other data sets on conflict and political instability, which rely on much slower methods of data collection and on processing involving human coders. The timeliness of the indicators derived through machine coding and their comprehensive global coverage in particular thus represent a unique potential for monitoring international crises.
29The third way in which big data methodologies can contribute to the monitoring of political instability relates to the political discourse itself, in particular to the perceptions of individuals with regard to salient issues. Political communication on social media is playing an increasingly important role in domestic and international politics. This is particularly true in the context of social movements, demonstrations and protests, especially during election periods. During the Arab Spring, social media contributed decisively to political debate, the coordination of protest activities and the global visibility of movements (Howard and Parks, 2012; Lotan et al., 2011). Other than traditional media, social media such as Facebook or Twitter allow private citizens to communicate on a large scale and in real time. They can thus benefit oppositional voices that cannot rely on the support of established media outlets. This is true, in particular, in autocracies, where such non-traditional media are often one of the few uncensored public spaces open for political communication.
30Spaiser et al. (2017), for example, systematically analyse political communication on Twitter during the period surrounding the Duma and the presidential elections in Russia in 2011‒12. At a technical level, the analysis reveals that Twitter can indeed be used to monitor political discourse and map political events and how they are perceived. The analysis also sheds light on the strategic use of Twitter by both oppositional and government actors, tracing the effectiveness of their respective campaigns over time. This suggests that Twitter (and social media more broadly) can, in principle, be used as a ‘sensor’ of voiced opinions, enabling researchers to trace, in near-real time, if and when specific political issues become salient—information that is of central relevance to understanding when and why political instability arises. This is especially relevant if perceptions of salient issues can be linked to specific geographic locales or regions.
31In practice, the same characteristics of big data that provide unique empirical leverage—volume, variety, velocity and spatial granularity—also pose fundamental challenges to their application for monitoring political instability.
32The first set of limitations arises from the often unstructured or semi-structured nature of big data, which necessitates extensive data cleaning and processing before they can be used for substantive analysis. This process is not fully objective, requiring specific choices about how to process raw input data—extracting the relevant information needed for a given analysis is ultimately a filtering problem. As data are typically not collected for the purpose of analysis, the information we are interested in may only be a small fraction of the total information collected. In other words, big data typically have a very poor signal-to-noise ratio. This is most prevalent for artefacts of human interactions, but also applies to remote sensing data where the raw sensor inputs undergo many elaborate transformations to yield those data used in practice (see Jarvis et al., 2008).
33The data processing approaches required are, further, often inherently algorithm dependent—that is to say, the different algorithms used to process data will not necessarily yield exactly the same results. This is a function of the exact choice of the methodologies and tools used to treat the data, but also depends on the subjective choices of the researcher and analyst in designing data treatment and processing. This may have very substantive implications for the inferences made, which may vary widely or be outright contradictory, even if the same original data are used. Generally, this subjectivity of the analysis process tends to be understated and thus underappreciated by those that build on specific findings and insights derived from big data.
34In the context of using night light emissions to measure local economic activity, Weidmann and Schutte (2017), for example, compare remote sensing measurements to survey data cautioning against assuming a too simplistic linear relationship between night light emission intensity and local economic conditions. Night light emissions are also not necessarily suitable for monitoring regions with very poor populations, simply because luminosity levels are generally low and show very little variation. Relying instead on high-resolution daytime satellite image data and machine learning algorithms, Jean et al. (2016) show that even under such circumstances remote sensing technologies can nonetheless lead to reliable estimates of local economic conditions.
35Similarly, the central concern with regard to the machine-coded data collection concerning instability events is whether automated processing can match the accuracy of the slower, methodical data collection employing human coders. Recent work by Wang et al. (2016) systematically compares these two modes of data collection and tests the validity of machine-coded data in tracing political instability. The comparison reveals significant shortcomings of machine-coded data, highlighting how they—to different but substantial degrees—tend to significantly over-report incidents and are prone to duplicate coding. But the study also highlights the significant potential of these data and emphasises the wide applicability of such measures if the quality of existing machine-coding approaches could be improved further. In these authors’ view, some of the problems faced by these data today simply arise from the fact that the methodologies used remain underdeveloped. More coherent and better-funded initiatives might help overcome the challenges inherent in big data approaches and make these data much more useful and reliable in future.
36In the analysis of text-based data—such as tweets or the content of news media articles—making ‘sense’ of unstructured data similarly poses one of the biggest challenges. While metadata are well structured and, typically, more easily interpretable, unstructured texts require natural language processing if they are to be analysed. This process remains far from perfect and is often less powerful than what would be required for use in quantitative analyses. In fact, even techniques that use human supervised machine-learning algorithms for the analysis of unstructured messaging content can, for example, usually only reliably determine such messages’ tone at the level of the entire message (e.g. Golder and Macy, 2011).
- 5 See, for example, Pew Research Center Report on Twitter use in the US (Duggan, 2015).
37The second set of limitations arises from the characteristics of the original sources these measurements are based on. Machine-coded data sets for political instability events rely on media-based reporting, which implies significant biases in their coverage—a problem that has been extensively studied in the context of conflict event data (Donnay and Filimonov, 2014; Weidmann, 2015; 2016). Similarly, data derived from artefacts of human interactions provide complete information on a given system but the user base of that system may actually not be representative of the population as a whole. If we then use these data to make inferences about the broader system, our analysis may again be biased. Twitter is a prime example, with a user base that is significantly younger than the general population.5 In developing countries, its user base also tends to be much smaller and skewed towards wealthy and privileged members of society.
38Moreover, both the user base and the technological system itself are typically changing quite rapidly. This implies that the same kind of data collected at one point in time may not easily compare to data collected at other points in time. This poses fundamental challenges to the external validity of measurements made. At the same time, the ‘attention span’ of big data is typically very short—topics trending on Twitter, for example, tend to have lifetimes of days rather than months or years. And, even though this is often implied, access to communication technologies is by no means available to all, and may, in fact, be subject to systematic discrimination driven by political interests (Weidmann et al., 2016).
39All of this poses significant challenges to the use of social media as reliable indicators of social processes. In addition, their bottom-up, distributed organisation and the direct dissemination of information make them susceptible to ‘fake news’ campaigns, as witnessed in the context of the recent US presidential election. They are also similarly vulnerable to concerted campaigns by governments spreading propaganda and marginalising oppositional voices (see Spaiser et al., 2017). Section 4 discusses in detail how they can nonetheless provide valuable information in the context of political instability monitoring.
40The third limitation concerns the external validity of big data measurements. It has been argued that the large volume of big data alone allows for exclusively data- and correlation-driven analysis of social systems (see Mayer-Schönberger and Cukier, 2013). The argument is that given enough data on a specific system, one does not need to understand the causal mechanisms at work to be able to predict its dynamics. Correlation does not imply causation, but given enough information on relevant system dynamics correlations should still be highly predictive.
41The underlying fundamental misconception is that good in-sample predictions—that is, the fact that correlations can predict dynamics well within a given system and for a given sample—imply good predictive power also out-of-sample. It may be true that in well-defined technical systems big data are indeed invaluable for predicting system dynamics and that correlations do provide fundamental insights—as long as the system itself is not changing. In all other circumstances, however, in-sample predictive power and today’s correlations are not necessarily a reliable indicator of future system dynamics. On the contrary, reliable out-of-sample predictions typically require a deeper understanding of the causal processes at work.
42Instead of using big data to predict system behaviours, the objective must therefore be to derive specific theoretically well-founded proxies that provide complementary information to existing measurements and are valid across contexts. This implies not only showing the construct validity of the proxy in question but also systematic empirical tests of its external validity. It is typically, then, the ‘extra’ bit of information gleamed from such measures that is most valuable—whether it is because it captures system dynamics that are otherwise unobservable, or has a higher temporal or spatial resolution, or both. In other words, such measures do not necessarily have to be perfectly accurate, but they have to provide substantial added value for a given analysis. Before introducing, in Section 4, what this specifically entails in a framework for political instability monitoring, the next section first reviews key ethical and legal considerations.
43Ethical and legal considerations may impose further restrictions on which data can or should be used. This section focuses on issues pertinent to the big data approaches reviewed above; others have addressed these questions in more depth—for example, Stevens and Laurie (2014) in the context of big administrative data.
- 6 For details, see the European Commission webpage: Reform of EU data protection rules, http://ec.eur (...)
44The most fundamental issue regarding big data as artefacts of social interactions is what appears to be a lack of informed consent by those whose data are being used. Especially on social media sites this lack of consent often simply arises because the scope of what big data are used for is not known—even cannot be known—at the time of collection. This is true for analytics done by those who collect these data but also for their secondary use, including their use in the context of monitoring political instability. Typically, agreements covering the collection and use of artefacts of human interactions are, then, necessarily generic. Recently, legal precedents have been set, for example by the EU’s General Data Protection Regulation, adopted in 2016.6 These rules provide more stringent legal frameworks in which actors—both data consumers and providers—can make informed decisions about how and by whom data are used. Yet, existing legal frameworks for the use of big data in general still appear to be very weak.
45The lack of truly informed consent is particularly pertinent in the context of monitoring political instability. Even though only public tweets can be accessed—that is to say, individuals are aware of their tweets’ public nature at the time of posting—users are most likely often unaware of our secondary use of these data. Specifically, they may be unaware of the fact that their statements could be used—together with many other similar statements—to approximate a population’s sentiment with regard to salient issues.
46The researcher then has a particular responsibility to use individual-level information in a responsible manner that ensures the absolute privacy of each individual entering the analysis. Especially in politically volatile settings, revealing any kind of political leanings or voiced opinions of individuals as part of the analysis has to be strictly avoided in order to not jeopardise their personal safety. In other words, the analysis should—beyond the public statements that were already made—not draw any additional attention to individual users, thus avoiding exposing them to additional risks. On a technical level, this implies that any measures derived from artefacts of human interactions should only be reported at a sufficiently aggregate level to avoid any inference about the user sample it was drawn from.
47There may also be unintended consequences of using social media to monitor political instability in a given context. Knowledge of the fact that opinion voiced publicly on social media is used to monitor a country’s or region’s vulnerability to conflict may trigger or increase the repression of public discourse by those in power. This effect is likely to be most severe in settings already characterised by repression and can adversely affects individuals’ access to information, and even their personal safety—not only for those individuals active on social media, but for the population as a whole. More generally, it is a serious concern that the better big data monitoring of political instability functions in practice, the greater the risk of its misuse by those in power. This may, ultimately, further endanger the very same population such a framework is intended to protect.
48These issues underscore the notion that considerations of ethical and legal quandaries should precede any big data inquiry. In many cases, such considerations would not necessarily make specific analyses impossible but they may warrant a closer consideration of how exactly a measure or monitoring technique should be employed.
- 7 See, for example, Chadefaux (2014, 7) for a comprehensive overview.
- 8 These approaches also tend to fare better at predicting the absence of instability rather than its (...)
49There is a well-established literature in political science focusing on the prediction of the onset of conflict—within or between states.7 These studies typically rely on structural indicators such as whether or not a country has recently experienced conflict, its level of democracy or its economic situation. Such broad, structural predictions of political instability are generally quite accurate (Backer et al., 2017) but have at least three significant shortcomings.8 First, they tend to focus on country-year predictions—that is to say, whether a country experienced political instability in a given year. This implies limited temporal and spatial specificity in monitoring where and when political instability is likely to occur. Second, these perspectives typically adopt an over-aggregated view of what political instability entails, typically excluding (non-violent) instability events such as protests and failing to properly disaggregate violent events ranging from isolated incidents to terrorism and full-scale civil war. Third, by construction these approaches tend to err in situations where several countries (or regions within countries) are, based on structural conditions, similarly likely to experience political instability but where we observe instability only in those countries where structural issues are becoming salient.
50This lack of resolution and specificity reflects systematic shortcoming in the measurement approaches traditionally used in these kinds of analyses. And there are typically no reliable or highly resolved measurements of perceptions of (political) issues that would indicate whether or not they are becoming salient—if such measurements exist, they tend to be derived from surveys with sparse temporal coverage that are not representative at a subnational level. Innovative big data approaches can, here, provide complementary measurements and help address these issues, thus enabling more sophisticated quantitative analyses of political instability in a consistent framework (Figure 1).
Figure 1. Big data vs. traditional measurements related to political instability
Source: Author.
51The use of both remote sensing and machine-coded data helps to address the first two concerns regarding the lack of spatio-temporal granularity and over-aggregation. As discussed earlier, key covariates such as local economic conditions can be resolved at a detailed subnational resolution relying on night light emissions-based measurements (Weidmann and Schutte, 2017). The same is true for environmental risk factors, and even major population movements can be tracked using remote sensing (Witmer, 2015). These data are not only geographically specific, they can also often be traced at fine-grained temporal resolutions. Other key covariates such as population density (CIESIN et al., 2005) or the share of ethnic groups excluded from power (Vogt et al., 2015) are also available at a subnational level, albeit typically with less granular temporal resolution. The example of night light emissions discussed previously illustrates that remote sensing measures are amenable to systematic empirical validation using independently collected data. Moreover, as Weidmann and Schutte (2017) emphasise, once validated, these data have substantial out-of-sample validity and may thus be used as reliable proxies in settings where spatially and temporally granular coverage would otherwise be absent.
52Machine-coded data, on the other hand, provide information on political instability at the level of single events that are geo-coded and time-stamped and can be disaggregated by type of incident. Their near-real time availability—a key advantage over manually coded event data—enables active instability monitoring. A study by Wang et al. (2016) suggests though that these data are only imperfect proxies of political instability as compared to other comprehensive, manually curated data sets—the Uppsala Conflict Data Project-Georeferenced Event Data (UCDP-GED) (Sundberg and Melander, 2013) or the Global Terrorism Database (GTD) (START, 2013) fall into the latter category and provide global coverage. Using a newly developed methodology for the automated integration and disambiguation of event data (Donnay et al., 2016),9 the reliability of machine-coded data could, in principle, be systematically evaluated against these hand-curated data. If it is known if and how machine-coded data errs, these data may then still prove to be powerful proxy measurements of political instability because of their superior, near-real time coverage. And if Wang et al.’s assessments are accurate, the quality of machine-coded data will only improve.
53A study by Chadefaux (2014) on the relationship between geopolitical tensions and conflict suggests an innovative way to monitor whether political issues become salient—that is to say, when actors feel inclined to act upon them. The study shows that information about perceived tensions between countries (or conflict actors) derived from news media coverage is sufficient to systematically improve upon the prediction of conflict onset—including the detection of early-warning signals of war up to one year prior to the onset of interstate conflict.
54In the context of monitoring political instability within specific countries or regions, such tensions typically arise as a consequence of perceived grievances—that is to say, the (self-) perception that an individual or group has been disadvantaged with regard to a specific (real or imagined) issue and feels aggrieved towards the actor deemed responsible. Grievances feature prominently in the literature on civil conflict as key mobilising factors (Cederman et al., 2011), regardless of whether perceived grievances represent objective inequalities or not (Miodownik and Nir, 2016). This implies though that—other than in Chadefaux (2014)—monitoring tensions in the context of intrastate conflict and instability requires an approach that is able to resolve subjective, individual-level sentiment with regard to politically relevant topics.
55The increasingly important role of social media in domestic and international politics paired with the ability to track individual expression regarding political issues on these platforms (Spaiser et al., 2017) suggests that monitoring conversations on social media, Twitter in particular, may be a viable approach. Specifically, the objective would be to dynamically detect dominant politically relevant topics of conversation on Twitter and then systematically estimate how individuals perceive these topics. The more extreme (and negative) the conversations about a given topic, the more salient these issues are and the higher the associated tensions. Combined with (structural) data on issue areas such as the economy, political representation or horizontal inequalities, these proxies thus, in principle, provide exactly the missing information that distinguishes otherwise structurally similar cases, and may be instrumental in monitoring political instability.
56This social media-based approach is already under development (McGrath et al., 2016) and initial results indicate that this strategy is indeed feasible. The framework has yielded empirically sound estimates of the salience of politically relevant topics and, thus, political tensions in the context of elections across Africa (McGrath et al., 2017). These measurements are especially sensitive to extreme sentiment. When tensions are low, issue-specific sentiment on Twitter is also relatively low with low variance. This applies to the bulk of politically relevant conversations. But if tensions are high, more extreme statements are made and sentiment towards these issues is much more variable. This sensitivity to extreme sentiment—that is to say, issue salience—is instrumental in countering some of the key weaknesses inherent in social media-based measurements.
57The fundamental argument is that a proxy of political tensions does not have to be exact but it does have to reliably and dynamically detect situations of extreme tension. This implies, for example, that the performance of any given topic modeling technique used should not be of critical relevance as long as it can reliably detect dominant topics in politically relevant communication on Twitter. Similarly, it is not necessarily strictly required that the Twitter population be representative—it may be sufficient that it is concerned with the same key political issues as the broader population. And the detection of the sentiment of individual statements on Twitter itself can be noisy and imperfect, as long as extreme sentiment is reliably captured. Rather, the key limitation is, in fact, often the ability to obtain sufficiently large and complete information on social media conversations in a given country and for a given time frame—something that in practice may require purchasing data from data resellers.
58The information on social tensions derived from Twitter is also amenable to systematic validation. For example, one can sample from the population of users contributing to the politically relevant conversation in a given context and administer surveys by directly contacting the users through Twitter—such approaches have recently been successfully used in the context of studying online mobilisation (Coppock et al., 2016). These surveys could, then, validate both the detection of dominant issue areas and the salience of specific topics. It is important to note that both these surveys, and Twitter-based measurements as a whole, can be performed while fully protecting the anonymity of individual users—including only highly aggregated reporting of any results or inferences, reporting that does not permit the inference of individual-level properties.
59As emphasised above, the combination of more granular measurements of relevant risk factors and instability events, together with information on political tensions, has the potential to significantly improve the monitoring of political instability within countries or regions—just as demonstrated by Chadefaux (2014) in the context of country-level instability. Initial results obtained in the context of African elections are encouraging (McGrath et al., 2017). The approach used there fully leverages the increased resolution of empirical measurements, employing a multi-level statistical modeling approach that incorporates both slowly changing structural measurements and highly temporally resolved measurements of tensions and instability events to predict political instability. In this framework, political tensions detected on Twitter are shown to be, indeed, highly predictive of subsequent instability events.
60A recent editorial by Lars-Erik Cederman and Nils Weidmann, two eminent conflict scholars, cautions against the ‘hope that somehow big data will yield valid forecasts through theory-free “brute force”’ (Cederman and Weidmann, 2017, 476). This is certainly the case. Big data approaches should be treated as complementary to existing tools and methods, rather than as their replacement. As the framework for monitoring political instability discussed above suggests, big data has the most to contribute if it is carefully combined with existing insights and validated against ‘classical’ data sources. Especially when it comes to the design of novel indicators, big data could then play a decisive role in enriching the repertoire of available measurements. This, however, has to be done mindful of the inherent limitations of big data approaches.
61The approach to political instability monitoring discussed above thus attempts to strike a careful balance. It does not aim to produce comprehensive long-term forecasts of whether countries or regions will become politically stable—a task that is notoriously difficult and highly contingent on the complex historical context of each given case (Cederman and Weidmann, 2017). Instead, it focuses on how big data measurement can aid in monitoring the occurrence of political instability events in structurally volatile settings, such as elections. In this context, it has the potential to enrich the current state-of-the-art by providing more fine-grained measurements and contextualising the structural covariates of conflicts with estimates of associated political tensions.
62Anticipating how political instability within a country will develop, whether tensions will escalate into armed conflict, and if instability events are likely to occur provides valuable insights to policymakers and practitioners alike. Among the primary recipients of reports on political instability are international aid organisations and NGOs. But early warnings on how trajectories of political instability are likely to develop are also instrumental for many UN organisations, including peacekeeping missions. In many cases, such reports may only complement assessments made by personnel on the ground, but nonetheless provide useful guidance and perspective. And, from a more general perspective, arriving at a better fundamental understanding of how the dynamics of political instability develop is not only relevant for academic research, it may also help guide future policy.
63Big data has the potential to provide an ever-increasing array of tools and methods that are directly relevant to monitoring the implementation of the SDGs. In many areas, such as remote sensing, these collaborations are already well under way. The framework discussed here related to SDG 16 illustrates how, in practice, such big data measurements can be seamlessly and beneficially integrated with existing monitoring approaches while remaining fully conscious of their potential limitations and biases. And, more importantly, it demonstrates that measurements derived from big data may indeed often be flawed, but that this does not necessarily imply that they are not useful.