Page 1 of 9Free preview · 1–5

About this document

This 2015 conference paper reviews Big Data research and existing definitions to clarify the concept. Using a corpus of 1,437 abstracts from Scopus, the authors identify four principal themes: Information, Technology, Methods and Impact. They discuss data generation, distributed processing, analytical skills, business value and societal concerns, including privacy and unequal data access. The paper proposes defining Big Data as information assets whose high volume, velocity and variety require specific technology and analytical methods for transformation into value. The review is explicitly non-exhaustive rather than systematic, and formal quantitative consensus analysis is left for future work.

Free preview text

The first 5 pages, shown as text so search engines and readers can find this document.

Page 1

What is Big Data? A Consensual Definition and a Review of Key Research Topics Andrea De Mauro1, a), Marco Greco2, b) and Michele Grimaldi2, c) 1Department of Enterprise Engineering, University of Rome Tor Vergata, Via del Politecnico 1, 00133 Roma, Italy 2Department of Civil and Mechanical Engineering, University of Cassino and Southern Lazio, Via Di Biasio 43, 03043 Cassino (FR), Italy a)Corresponding author: andrea.de.mauro@uniroma2.it b)m.greco@unicas.it c)m.grimaldi@unicas.it Abstract. Although Big Data is a trending buzzword in both academia and the industry, its meaning is still shrouded by much conceptual vagueness. The term is used to describe a wide range of concepts: from the technological ability to store, aggregate, and process data, to the cultural shift that is pervasively invading business and society, both drowning in information overload. The lack of a formal definition has led research to evolve into multiple and inconsistent paths. Furthermore, the existing ambiguity among researchers and practitioners undermines an efficient development of the subject. In this paper we have reviewed the existing literature on Big Data and analyzed its previous definitions in order to pursue two results: first, to provide a summary of the key research areas related to the phenomenon, identifying emerging trends and suggesting opportunities for future development; second, to provide a consensual definition for Big Data, by synthesizing common themes of existing works and patterns in previous definitions. Keywords: Big Data; Analytics; Information Management; Data Processing; Business Intelligence. INTRODUCTION Big Data1 has now become a ubiquitous term in many parts of industry and academia. As often happens in these cases, the frequent utilization of the same words in different contexts poses a threat towards the structured evolution of its meaning. For this reason it is necessary to invest time and effort in the proposition and the acceptance of a standard definition of Big Data that would pave the way to its systemic evolution and minimize the confusion related to its usage. In order to describe Big Data we have decided to start from an “as is” analysis of the contexts in which the term most frequently appears. Given its remarkable success and its hectic evolution, Big Data possesses multiple and diverse nuances of meaning, all of which have the right to exist. By analyzing the most significant occurrences of this term in both academic and business literature we have identified four key themes to which Big Data refers: Information, Technologies, Methods and Impact. We can reasonably assert that the vast majority of references to Big Data encompass one of the four themes listed above. Understanding how these themes have been dealt with in existing literature and how they are mutually interconnected is the objective of the first section of this paper and is propaedeutic to the attempt of proposing a thorough definition, which is what the second section aims to provide. We believe that having such a definition will enable a more conscious usage of the term Big Data and a more coherent development of research on this subject. 1 We have chosen to capitalize the term ‘Big Data’ throughout this article to clarify that it is the specific subject we are discussing. International Conference on Integrated Information (IC-ININFO 2014) AIP Conf. Proc. 1644, 97-104 (2015); doi: 10.1063/1.4907823 © 2015 AIP Publishing LLC 978-0-7354-1283-5/$30.00 97

Page 2

REVIEW OF MAIN RESEARCH TOPICS This section represents a comprehensive but non-exhaustive review of research topics in the area of Big Data. We have examined a large number of abstracts of peer-reviewed conference and journal papers and identified recurring topics by looking at the appearance frequency of top keywords and making an educated guess on their interrelation. We needed to apply this heuristic approach in order to produce a depiction of the ample range of concepts related to Big Data while using a relatively small number of topic categories. A systematic literature review is beyond the scope of this paper and left as an opportunity for future work. The input list of documents was obtained from Elsevier’s Scopus, a citation database containing more than 50 million records from around 5,000 publishers. On the 3rd of May 2014 we exported a list of 1,581 conference papers and articles that contained the full term “Big Data” in either the title or within the author-provided keywords2. We have removed those entries where the abstract text was not available and this left us with a corpus of 1,437 documents. By counting the appearance frequency of words included in the abstracts we have identified the most recurring items. Figure 1 shows a static tag cloud visualization (also known as “word cloud”) of the most popular words in the abstracts we analyzed, obtained through the online tool ManyEyes [1]. By analyzing the most frequent keywords included in Big Data-related abstracts and considering their mutual relationships we have identified four top research themes in current literature, namely: 1. Information, 2. Technology, 3. Methods, 4. Impact. We believe that the great majority of papers written on Big Data touch upon one or more of these four topics. For each of them we will now describe content, trends and enlist a number of relevant works. FIGURE 1. Static tag cloud visualization (word cloud) of key terms appearing in abstracts of Big Data-related papers. The Fuel of Big Data: Information One of the fundamental reasons for Big Data phenomenon to exist is the current extent to which information can be generated and made available. Digitization, i.e. the process of converting continuous, analog information into discrete, digital and machine-readable format, reached broad popularity with the first “mass digitization” projects. Mass digitization is the attempt to convert entire printed book libraries into digital collections by leveraging optical character recognition (OCR) software in order to minimize human intervention [2]. One of the most popular attempts of mass digitization was the Google Print Library Project3, started in 2004, that aimed at digitizing more than 15 million volumes held in multiple university libraries, including Harvard, Stanford and Oxford. More recently it has been proposed a subtle differentiation between digitization and its next step, datafication, i.e. putting a phenomenon in a quantified format so that it can be tabulated and analyzed [3]. The fundamental difference is that 2 We have used the following search query in Scopus: “AUTHKEY("Big data") OR TITLE("big data") AND (LIMIT-TO(DOCTYPE, "cp") OR LIMIT-TO(DOCTYPE, "ar") OR LIMIT-TO(DOCTYPE, "ip"))” 3 For more information you can visit the Google Books History page, available at http://www.google.com/googlebooks/about/history.html. 98

Page 3

digitization enables analog information to be transferred and stored in a more convenient digital format while datafication aims at organizing digitized version of analog signals in order to generate insights that would have not been inferred while signals were in their original form. In the case of the previously cited Google mass digitization effort, the value of datafication came when researchers showed they were able to provide insights on lexicography, the evolution of grammar, collective memory, the adoption of technology, the pursuit of fame, censorship, and historical epidemiology by using Google Books’ data [4]. Digitization and datafication have become pervasive phenomena thanks to the broad availability of devices that are both connected and provided with digital sensors. Digital sensors enable digitization while connection lets data be aggregated and, thus, permits datafication. Cisco estimated that between 2008 and 2009 the number of connected devices overtook the number of living people [5] and, according to Gartner [6], by 2020 there will be 26 billion devices on earth, more than 3 devices on average per person. The pervasive presence of a variety of objects (including mobile phones, sensors, Radio-Frequency Identification - RFID - tags, actuators), which are able to interact with each other and cooperate with their neighbors to reach common goals, goes under the name of the Internet of Things, IoT [7,8]. This increasing availability of sensor-enabled, connected devices is equipping companies with extensive information assets from which it is possible to create new business models, improve business processes and reduce costs and risks [9]. In other words, IoT is one of the most promising fuels of Big Data expansion. Another characteristic of the data generated today is its increasing variety in type. Structured data (traditional text/numeric information) is now joined by unstructured data (audio, video, images, text and human language) and semistructured data, such as XML and RSS feeds [10]. The diversity of data types is one of the challenges that organizations need to tackle in order to make value out of the extensive informational assets available today [11]. Equipment for Working with Big Data: Technology The term Big Data is frequently associated with the specific technology that enables its utilization. The extent of the dataset size and the complexity of operations needed for its processing entail stringent memory storage and computational performance requirements. According to Google Trends, the most related query to “Big Data” is “Hadoop” that indeed is the most prominent technology associated with this topic. Hadoop is an open source framework that enables the distributed processing of big quantities of data by using a group of dispersed machines and specific computer programming models. The main components of Hadoop are: 1. its file system HDFS, that allows access to data scattered over multiple machines without having to cope with the complexity inherent to their dispersed nature; 2. MapReduce, a programming model designed to implement distributed and parallel algorithms in an efficient way. Both HDFS [12] and MapReduce [13] are the evolution of concepts that were originally proposed by Google [14] and that were then developed as open-source projects within Apache’s framework. This proves the centrality of Google in the initiation of the current thinking about Big Data. The Hadoop framework contains multiple modules and libraries compatible with HDFS and MapReduce that enable the extension of its applicability to the various needs of coordination, analysis, performance management and workflow design that normally occur in Big Data applications. The distributed nature of information requires a specific technological effort for transmitting big quantities of data and for monitoring the overall system performance using special benchmarking techniques [15]. Another fundamental technological element is the ability to store a bigger quantity of data on smaller physical devices. Although Moore’s law suggests that storing capacity increases over time in an exponential manner [16], still it is required a continuous and expensive research and development effort to keep up with the pace at which data size increases [17] especially with the growing share of byte-hungry data types such as images, sounds and videos. Transforming Big Data in Value: Methods The analysis of extensive quantities of data and the need to grasp value out of individual behaviors require processing methods that go beyond the traditional statistical techniques. The knowledge of such methods, of their potential and, above all, of their limitations requires specific skills that are hard to find in today’s job marketplace. Both Manyika et al. [11] and Chen [18] propose a list of Big Data Analytical Methods, that include (in alphabetical order): A/B testing, Association rule learning, Classification, Cluster analysis, Data fusion and data integration, Ensemble learning, Genetic algorithms, Machine learning, Natural Language Processing, Neural 99

Page 4

networks, Network analysis, Pattern recognition, Predictive modelling, Regression, Sentiment Analysis, Signal Processing, Spatial analysis, Statistics, Supervised and Unsupervised learning, Simulation, Time series analysis and Visualization. Chen et al. [18] evoke the need for companies to invest in Business Intelligence and Analytics education that would be “interdisciplinary and cover critical analytical and IT skills, business and domain knowledge, and communication skills required in a complex data-centric business environment”. The investment in analytical knowledge should be accompanied by a cultural change that would span across all employees and urge them to “efficiently manage data properly and incorporate them into decision making processes” [19]. Mayer-Schönberger and Cukier [3] envision the rise of new specific professional entities, called algorithmists, that would master the areas of computer science, mathematics and statistics and act as “impartial auditors to review the accuracy or validity of Big Data predictions”. Also Davenport and Patil [20] describe data scientist as a hybrid of “data hacker, analyst, communicator, and trusted adviser”, having also the fundamental abilities to write code and conduct, when needed, academic-style research. These skills are not sufficiently available to meet the increasing demand: according to Manyika et al. [11], by the year 2018 there will be a potential shortfall of 1.5 million data-savvy managers and analysts, in the US only. The analysis of competency gaps and the creation of effective teaching methods to fill them for both future and current managers and practitioners is a promising research area that has still much opportunity to grow. Also the ability of making informed decisions is changing with the expansion of Big Data as the latter implies the shift from logical, causality-based reasoning to the acknowledgment of correlation links between events. The utilization of insights generated through Big Data Analytics in companies, universities and institutions provides for an adaptation to a new culture of decision making [21] and an evolution of the scientific method [22], both of which are still to be built and provide opportunities for future research. Being aware of the limitations of Big Data Methods and potential methodological issues is a fundamental resource for organizations who want to drive data-based decision making: for example, predictions should always be accompanied by valid confidence intervals in order to avoid the false sense of precision that the apparent sophistication of some Big Data applications can suggest. Analysts should also be capable of avoiding models’ overfitting that would facilitate apophenia, i.e. the tendency of humans to “see patterns where none actually exist simply because enormous quantities of data can offer connections that radiate in all directions”, [23]. In a summary, Big Data requires the mastery of specific techniques, awareness of their strengths and limitations, and a spread cultural tendency to informed decision making that in most cases has still to be built. How Big Data Changes our Lives: Impact The extent to which Big Data is impacting our society and our companies is often depicted through anecdotes and success stories of methods and technology implementations. When these stories are accompanied by proposals of new principles and methodological improvements they represent a valuable contribution to the creation of knowledge on the subject. The pervasive nature of the current information production and availability leads to many applications spanning in numerous scientific fields and industry sectors that can be very distant from each other. Sometimes, the same techniques and data have been applied to solve problems in distant domains. For example, correlation analysis was leveraged to use logs of Google searches to forecast influenza epidemics [24] as well as unemployment [25] and inflation [26]. The existing Big Data applications are many and expected to grow: hence, their systematic description constitutes a promising development area for those willing to contribute in the scientific progress in this field. Big Data can also impact society adversely. In fact, there are multiple concerns arising from the quick advancement of Big Data [23] first being privacy. Although large data sets would normally proceed from actions done by a multitude of individuals, it is not always true that consequences of using that data will not impact a single individual in an invasive and/or unexpected way. The identifiability of the individual person can be avoided through a thorough anonymization of the data set, although it is hard to be fully guaranteed as the reverse process of de- anonymization can be potentially attempted [27]. The predictability of future actions, made possible by the analysis of behavioral patterns, poses also the ethical issue of protecting free will in the future, on top of freedom in the present. Other issues to be considered are related to the accessibility of information: the exclusive control over data sources can become an abuse of dominant position and restrict competition by posing unfair entrance barriers to the marketplace. For example, as Manovich notices [28], “only social media companies have access to really large 100

Page 5

social data – especially transactional data” and they have full control over who can access what information. The split between information-rich and data-lacking companies can create a new digital divide [23] that can slow down innovation in the sector. Specific policies will have to be promoted and data is likely to become a new dimension to consider within antitrust regulations. Not only society but also companies are heavily impacted by the rise of Big Data: the call to arms for acquiring vital skills and technology to be competitive in a data-driven market implies a serious reconsideration of the firm organization and the full realm of business processes [29]. The transformation of data into competitive advantage [21] is what makes “Big Data” such an impactful revolution in today’s business world. A DEFINITION FOR BIG DATA A convincing definition of a concept is an enabler of its scientific development. As Ronda-Pupo and Guerras- Martin [30] suggest, the level of consensus shown by a scientific community on a definition of a concept can be used as a measure of progress of a discipline. Big Data has instead evolved so quickly and disorderly that such a universally accepted formal statement denoting its meaning does not exist. There have been many attempts of definition for Big Data, more or less popular in terms of utilization and citation. However, none of these proposals has prevented authors of Big Data-related works to extend, renovate or even ignore previous definitions and propose new ones. Although Big Data is still a relatively young concept, it certainly deserves an accepted vocabulary of reference that enables the proper development of the discipline among cognoscenti and practitioners. In the first part of this paper we have identified the four main themes of Big Data and we have observed that they are the prevalent topics in the existing literature. In the next paragraphs we will review a non-exhaustive list of previously proposed Big Data definitions and we will conceptually tie them to the aforementioned four themes of research. After considering the existing definitions and analyzing their commonalities we will propose a consensual definition of Big Data. Consensus in this case comes from the acknowledgement of centrality of some recurring attributes associated to Big Data, and from the assumption that they define the essence of what Big Data means to scholars and practitioners today. We expect that such a definition would be less prone to attack from previous definitions’ authors and users as it is based on the most central aspects associated until now to Big Data. A thorough consensus analysis based on Cohen’s K coefficient [31] and co-word analysis, as in [30], goes beyond the scope of this work and is left for future study. Survey of Existing Definitions Big Data has been often described “implicitly” through success stories or anecdotes, characteristics, technological features, emerging trends or its impact to society, organizations and business processes. In the existing attempts of explicit definitions for Big Data there is not even an agreement on what entity this term can be associated with. We have found that Big Data is used when referring to a variety of different entities including - but not limited to - social phenomenon, information assets, data sets, analytical techniques, storage technologies, processes and infrastructures. We have surveyed multiple definitions that have been proposed to date and listed them in Tab. 1: in this paragraph we will go through the most notable ones. A first group of Big Data definitions focuses on enlisting its characteristics. What is probably the most popular definition falls within this group. When presenting the Data Management challenges that companies had to face in response to the rise of e-commerce in the early 2000’s, Laney introduces a framework expressing the 3-dimensional increase in data Volume, Velocity and Variety and invokes the need for new formal practices that will imply “tradeoffs and architectural solutions that involve/impact application portfolios and business strategy decisions” [32]. Although this work did not mention Big Data explicitly, the model, later nicknamed as “the 3 V’s”, was associated to the concept of Big Data and used as its definition [33–35]. Many other authors extended the “3 V’s” model and, as a result, multiple features of Big Data, like Value [36], Veracity [37], Complexity and Unstructuredness [38, 39], were added to the list. A second group of definitions emphasizes the technological needs behind the processing of large amounts of data. According to Microsoft, Big Data is about applying “serious computing power” to massive sets of information [40] and also the National Institute of Standards and Technology (NIST) highlights the need for a “scalable architecture for efficient storage, manipulation, and analysis” when defining Big Data [41]. A few definitions associate Big Data to the crossing of some sort of threshold: for instance Dumbill [42] asserts that data is Big when it “exceeds the processing capacity of conventional database systems” and requires the choice 101

Tags

Doxxly editorial guide

What is Big Data? A Consensual Definition and a Review of Key Research Topics: summary, contents and background

This guide helps you understand the scope of What is Big Data? A Consensual Definition and a Review of Key Research Topics before or alongside reading the original document. It is based on the catalogue record, credited creator, source, classification and topics shown on this page. It does not replace the original work and does not invent claims about passages that Doxxly cannot verify.

Summary and overview

What is Big Data? A Consensual Definition and a Review of Key Research Topics is a 9-page documentcredited to Soren V. Cressley. This 2015 conference paper reviews Big Data research and existing definitions to clarify the concept. Using a corpus of 1,437 abstracts from Scopus, the authors identify four principal themes: Information, Technology, Methods and Impact. They discuss data generation, distributed processing, analytical skills, business value and societal concerns, including privacy and unequal data access. The paper proposes defining Big Data as information assets whose high volume, velocity and variety require specific technology and analytical methods for transformation into value. The review is explicitly non-exhaustive rather than systematic, and formal quantitative consensus analysis is left for future work. It appears in the Technology & Security collection, which groups it with other material that readers may use for comparison and further study.

The item can be approached in several ways. A reader looking for a quick orientation can begin with the free five-page preview, inspect the page structure and use the metadata to judge relevance. Someone undertaking deeper research can work through the complete file, search its indexed text, record useful page references and compare its treatment of Big Data, Analytics, Information Management, Data Processing, Business Intelligence, Research Review with the related resources linked below.

Rather than reducing the work to a single claim, this page treats it as a source that should be read in context. The title, creator, publication source, licence and format all help explain what kind of evidence it may offer. Readers should distinguish the original author's statements from Doxxly's catalogue description, and should cite the original edition when using the material in academic, professional or published work.

Key topics and reader takeaways

The catalogue connects this document with Big Data, Analytics, Information Management, Data Processing, Business Intelligence, Research Review. These terms offer practical entry points rather than a substitute for close reading. Start by identifying the definitions, examples, arguments, tables or visual explanations attached to each topic. Then note where the creator draws connections, makes distinctions or moves from background material to a conclusion, recommendation, calculation or narrative development.

A useful reading method is to separate description from interpretation. First record what the item actually presents: names, headings, figures, formulas, quotations or stated propositions. Next consider why those details matter within the whole work. Finally compare them with another source from the same subject area. This process helps turn a long digital file into evidence that can be evaluated rather than merely skimmed.

Table of contents and reading guide

The following is an editorial content guide based on the verified catalogue topics. It is not represented as the original author's formal table of contents. Use it to organise notes while moving through the 9 pages in the player.

  1. Part 1

    Big Data

    Use this section of the reading guide to identify how Big Data appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.

  2. Part 2

    Analytics

    Use this section of the reading guide to identify how Analytics appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.

  3. Part 3

    Information Management

    Use this section of the reading guide to identify how Information Management appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.

  4. Part 4

    Data Processing

    Use this section of the reading guide to identify how Data Processing appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.

  5. Part 5

    Business Intelligence

    Use this section of the reading guide to identify how Business Intelligence appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.

  6. Part 6

    Research Review

    Use this section of the reading guide to identify how Research Review appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.

When a heading in the original differs from this guide, follow the original heading in your citation. Page numbers can also vary between editions. Doxxly's direct page field is useful for navigation, but formal references should include the title, creator, source and edition details available in the file itself.

About Soren V. Cressley

Doxxly credits Soren V. Cressley as the author or creator of What is Big Data? A Consensual Definition and a Review of Key Research Topics. This page does not add an unverified biography, job title or institutional affiliation. The attribution is preserved so readers can distinguish the creator from the hosting library and follow the original record when an external author link is available.

Author context matters because a work reflects a particular purpose, audience and point of view. Before relying on the material, consider what the creator set out to explain, when the source was produced and whether later research or revised guidance should also be consulted. For literature and philosophy, compare translations and editions; for technical material, check whether a newer version has superseded the one shown here.

About the source organisation

The recorded source for this item is dhi.ac.uk. Doxxly provides access and discovery tools but does not claim to be the original publisher unless explicitly stated. Source links, creator attribution and the recorded Publicly available file collected for editorial review licence are displayed to help readers assess provenance and permitted use.

Provenance is especially important for reports, standards, presentations and working spreadsheets. Check the original source for amendments, supporting files and current policy. A trustworthy research workflow keeps the source URL with any notes or downloaded copy, verifies important figures against the original, and avoids treating an archived document as automatically current.

Visit dhi.ac.uk

How to read and use this document

Begin with the title and opening pages, then use the guide above to decide whether to read sequentially or move to a particular topic. The Doxxly player provides previous and next navigation, a direct page field, zoom steps, fit-to-width and fit-to-page modes, rotation and fullscreen. These controls are useful when a page contains small type, a wide table, an image, a scanned layout or a dense sequence of notes.

The first five pages are open to every visitor. A plan unlocks the complete item and full-document search. Search is best used for distinctive names, technical terms or short phrases; broad words can produce too many possible contexts. When you find a useful passage, read the surrounding pages so that a sentence, chart or formula is not separated from its explanation.

If downloading is enabled, members can obtain the original PDF file. Keep the original format when formulas, slide sequencing, pagination or layout carries meaning. Respect the Publicly available file collected for editorial review terms displayed on this page, and use the rights-report link if you believe the attribution or availability needs review.

Research questions to consider

As you read What is Big Data? A Consensual Definition and a Review of Key Research Topics, ask which audience the creator addresses, what problem or idea organises the work, what evidence is presented, and which assumptions remain unstated. Note whether important terms are defined consistently and whether examples represent a broader pattern or a narrow case. For quantitative material, record units, time periods, formulas and data sources. For interpretive material, separate the creator's language from your own summary.

A second set of questions concerns relevance today. Which parts remain durable, which depend on their historical or institutional setting, and what would need confirmation from a newer source? Following the tags and the Technology & Security collection can reveal useful comparisons. Agreement between sources can strengthen confidence, while disagreement can point to changing definitions, methods, evidence or priorities.

Finally, decide what you need from the item: background knowledge, a quotation, a worked example, a model, a visual explanation or a starting bibliography. That purpose should guide the depth of reading and the notes you keep. Doxxly helps locate and navigate the source; careful interpretation remains the reader's responsibility.

Explore more Technology & Security resources

Frequently asked questions about What is Big Data? A Consensual Definition and a Review of Key Research Topics

Practical answers about reading, downloading and understanding this document on Doxxly.

More in Technology & Security