This 2015 conference paper reviews Big Data research and existing definitions to clarify the concept. Using a corpus of 1,437 abstracts from Scopus, the authors identify four principal themes: Information, Technology, Methods and Impact. They discuss data generation, distributed processing, analytical skills, business value and societal concerns, including privacy and unequal data access. The paper proposes defining Big Data as information assets whose high volume, velocity and variety require specific technology and analytical methods for transformation into value. The review is explicitly non-exhaustive rather than systematic, and formal quantitative consensus analysis is left for future work.
Free preview text
The first 5 pages, shown as text so search engines and readers can find this document.
REVIEW OF MAIN RESEARCH TOPICS
This section represents a comprehensive but non-exhaustive review of research topics in the area of Big Data.
We have examined a large number of abstracts of peer-reviewed conference and journal papers and identified
recurring topics by looking at the appearance frequency of top keywords and making an educated guess on their
interrelation. We needed to apply this heuristic approach in order to produce a depiction of the ample range of
concepts related to Big Data while using a relatively small number of topic categories. A systematic literature
review is beyond the scope of this paper and left as an opportunity for future work. The input list of documents was
obtained from Elsevier’s Scopus, a citation database containing more than 50 million records from around 5,000
publishers. On the 3rd of May 2014 we exported a list of 1,581 conference papers and articles that contained the full
term “Big Data” in either the title or within the author-provided keywords2. We have removed those entries where
the abstract text was not available and this left us with a corpus of 1,437 documents. By counting the appearance
frequency of words included in the abstracts we have identified the most recurring items. Figure 1 shows a static tag
cloud visualization (also known as “word cloud”) of the most popular words in the abstracts we analyzed, obtained
through the online tool ManyEyes [1].
By analyzing the most frequent keywords included in Big Data-related abstracts and considering their mutual
relationships we have identified four top research themes in current literature, namely: 1. Information,
2. Technology, 3. Methods, 4. Impact. We believe that the great majority of papers written on Big Data touch upon
one or more of these four topics. For each of them we will now describe content, trends and enlist a number of
relevant works.
FIGURE 1. Static tag cloud visualization (word cloud) of key terms appearing in abstracts of Big Data-related papers.
The Fuel of Big Data: Information
One of the fundamental reasons for Big Data phenomenon to exist is the current extent to which information can
be generated and made available. Digitization, i.e. the process of converting continuous, analog information into
discrete, digital and machine-readable format, reached broad popularity with the first “mass digitization” projects.
Mass digitization is the attempt to convert entire printed book libraries into digital collections by leveraging optical
character recognition (OCR) software in order to minimize human intervention [2]. One of the most popular
attempts of mass digitization was the Google Print Library Project3, started in 2004, that aimed at digitizing more
than 15 million volumes held in multiple university libraries, including Harvard, Stanford and Oxford. More
recently it has been proposed a subtle differentiation between digitization and its next step, datafication, i.e. putting a
phenomenon in a quantified format so that it can be tabulated and analyzed [3]. The fundamental difference is that
2 We have used the following search query in Scopus: “AUTHKEY("Big data") OR TITLE("big data") AND (LIMIT-TO(DOCTYPE, "cp") OR
LIMIT-TO(DOCTYPE, "ar") OR LIMIT-TO(DOCTYPE, "ip"))”
3 For more information you can visit the Google Books History page, available at http://www.google.com/googlebooks/about/history.html.
98
Page 3
digitization enables analog information to be transferred and stored in a more convenient digital format while
datafication aims at organizing digitized version of analog signals in order to generate insights that would have not
been inferred while signals were in their original form. In the case of the previously cited Google mass digitization
effort, the value of datafication came when researchers showed they were able to provide insights on lexicography,
the evolution of grammar, collective memory, the adoption of technology, the pursuit of fame, censorship, and
historical epidemiology by using Google Books’ data [4].
Digitization and datafication have become pervasive phenomena thanks to the broad availability of devices that
are both connected and provided with digital sensors. Digital sensors enable digitization while connection lets data
be aggregated and, thus, permits datafication. Cisco estimated that between 2008 and 2009 the number of connected
devices overtook the number of living people [5] and, according to Gartner [6], by 2020 there will be 26 billion
devices on earth, more than 3 devices on average per person. The pervasive presence of a variety of objects
(including mobile phones, sensors, Radio-Frequency Identification - RFID - tags, actuators), which are able to
interact with each other and cooperate with their neighbors to reach common goals, goes under the name of the
Internet of Things, IoT [7,8]. This increasing availability of sensor-enabled, connected devices is equipping
companies with extensive information assets from which it is possible to create new business models, improve
business processes and reduce costs and risks [9]. In other words, IoT is one of the most promising fuels of Big Data
expansion.
Another characteristic of the data generated today is its increasing variety in type. Structured data (traditional
text/numeric information) is now joined by unstructured data (audio, video, images, text and human language) and
semistructured data, such as XML and RSS feeds [10]. The diversity of data types is one of the challenges that
organizations need to tackle in order to make value out of the extensive informational assets available today [11].
Equipment for Working with Big Data: Technology
The term Big Data is frequently associated with the specific technology that enables its utilization. The extent of
the dataset size and the complexity of operations needed for its processing entail stringent memory storage and
computational performance requirements. According to Google Trends, the most related query to “Big Data” is
“Hadoop” that indeed is the most prominent technology associated with this topic. Hadoop is an open source
framework that enables the distributed processing of big quantities of data by using a group of dispersed machines
and specific computer programming models. The main components of Hadoop are: 1. its file system HDFS, that
allows access to data scattered over multiple machines without having to cope with the complexity inherent to their
dispersed nature; 2. MapReduce, a programming model designed to implement distributed and parallel algorithms in
an efficient way. Both HDFS [12] and MapReduce [13] are the evolution of concepts that were originally proposed
by Google [14] and that were then developed as open-source projects within Apache’s framework. This proves the
centrality of Google in the initiation of the current thinking about Big Data. The Hadoop framework contains
multiple modules and libraries compatible with HDFS and MapReduce that enable the extension of its applicability
to the various needs of coordination, analysis, performance management and workflow design that normally occur in
Big Data applications.
The distributed nature of information requires a specific technological effort for transmitting big quantities of
data and for monitoring the overall system performance using special benchmarking techniques [15].
Another fundamental technological element is the ability to store a bigger quantity of data on smaller physical
devices. Although Moore’s law suggests that storing capacity increases over time in an exponential manner [16],
still it is required a continuous and expensive research and development effort to keep up with the pace at which
data size increases [17] especially with the growing share of byte-hungry data types such as images, sounds and
videos.
Transforming Big Data in Value: Methods
The analysis of extensive quantities of data and the need to grasp value out of individual behaviors require
processing methods that go beyond the traditional statistical techniques. The knowledge of such methods, of their
potential and, above all, of their limitations requires specific skills that are hard to find in today’s job marketplace.
Both Manyika et al. [11] and Chen [18] propose a list of Big Data Analytical Methods, that include (in
alphabetical order): A/B testing, Association rule learning, Classification, Cluster analysis, Data fusion and data
integration, Ensemble learning, Genetic algorithms, Machine learning, Natural Language Processing, Neural
99
Page 4
networks, Network analysis, Pattern recognition, Predictive modelling, Regression, Sentiment Analysis, Signal
Processing, Spatial analysis, Statistics, Supervised and Unsupervised learning, Simulation, Time series analysis and
Visualization.
Chen et al. [18] evoke the need for companies to invest in Business Intelligence and Analytics education that would
be “interdisciplinary and cover critical analytical and IT skills, business and domain knowledge, and
communication skills required in a complex data-centric business environment”. The investment in analytical
knowledge should be accompanied by a cultural change that would span across all employees and urge them to
“efficiently manage data properly and incorporate them into decision making processes” [19]. Mayer-Schönberger
and Cukier [3] envision the rise of new specific professional entities, called algorithmists, that would master the
areas of computer science, mathematics and statistics and act as “impartial auditors to review the accuracy or
validity of Big Data predictions”. Also Davenport and Patil [20] describe data scientist as a hybrid of “data hacker,
analyst, communicator, and trusted adviser”, having also the fundamental abilities to write code and conduct, when
needed, academic-style research. These skills are not sufficiently available to meet the increasing demand: according
to Manyika et al. [11], by the year 2018 there will be a potential shortfall of 1.5 million data-savvy managers and
analysts, in the US only. The analysis of competency gaps and the creation of effective teaching methods to fill them
for both future and current managers and practitioners is a promising research area that has still much opportunity to
grow.
Also the ability of making informed decisions is changing with the expansion of Big Data as the latter implies
the shift from logical, causality-based reasoning to the acknowledgment of correlation links between events. The
utilization of insights generated through Big Data Analytics in companies, universities and institutions provides for
an adaptation to a new culture of decision making [21] and an evolution of the scientific method [22], both of which
are still to be built and provide opportunities for future research.
Being aware of the limitations of Big Data Methods and potential methodological issues is a fundamental
resource for organizations who want to drive data-based decision making: for example, predictions should always be
accompanied by valid confidence intervals in order to avoid the false sense of precision that the apparent
sophistication of some Big Data applications can suggest. Analysts should also be capable of avoiding models’
overfitting that would facilitate apophenia, i.e. the tendency of humans to “see patterns where none actually exist
simply because enormous quantities of data can offer connections that radiate in all directions”, [23].
In a summary, Big Data requires the mastery of specific techniques, awareness of their strengths and limitations,
and a spread cultural tendency to informed decision making that in most cases has still to be built.
How Big Data Changes our Lives: Impact
The extent to which Big Data is impacting our society and our companies is often depicted through anecdotes
and success stories of methods and technology implementations. When these stories are accompanied by proposals
of new principles and methodological improvements they represent a valuable contribution to the creation of
knowledge on the subject. The pervasive nature of the current information production and availability leads to many
applications spanning in numerous scientific fields and industry sectors that can be very distant from each other.
Sometimes, the same techniques and data have been applied to solve problems in distant domains. For example,
correlation analysis was leveraged to use logs of Google searches to forecast influenza epidemics [24] as well as
unemployment [25] and inflation [26]. The existing Big Data applications are many and expected to grow: hence,
their systematic description constitutes a promising development area for those willing to contribute in the scientific
progress in this field.
Big Data can also impact society adversely. In fact, there are multiple concerns arising from the quick
advancement of Big Data [23] first being privacy. Although large data sets would normally proceed from actions
done by a multitude of individuals, it is not always true that consequences of using that data will not impact a single
individual in an invasive and/or unexpected way. The identifiability of the individual person can be avoided through
a thorough anonymization of the data set, although it is hard to be fully guaranteed as the reverse process of de-
anonymization can be potentially attempted [27]. The predictability of future actions, made possible by the analysis
of behavioral patterns, poses also the ethical issue of protecting free will in the future, on top of freedom in the
present.
Other issues to be considered are related to the accessibility of information: the exclusive control over data
sources can become an abuse of dominant position and restrict competition by posing unfair entrance barriers to the
marketplace. For example, as Manovich notices [28], “only social media companies have access to really large
100
Page 5
social data – especially transactional data” and they have full control over who can access what information. The
split between information-rich and data-lacking companies can create a new digital divide [23] that can slow down
innovation in the sector. Specific policies will have to be promoted and data is likely to become a new dimension to
consider within antitrust regulations.
Not only society but also companies are heavily impacted by the rise of Big Data: the call to arms for acquiring
vital skills and technology to be competitive in a data-driven market implies a serious reconsideration of the firm
organization and the full realm of business processes [29]. The transformation of data into competitive advantage
[21] is what makes “Big Data” such an impactful revolution in today’s business world.
A DEFINITION FOR BIG DATA
A convincing definition of a concept is an enabler of its scientific development. As Ronda-Pupo and Guerras-
Martin [30] suggest, the level of consensus shown by a scientific community on a definition of a concept can be
used as a measure of progress of a discipline. Big Data has instead evolved so quickly and disorderly that such a
universally accepted formal statement denoting its meaning does not exist. There have been many attempts of
definition for Big Data, more or less popular in terms of utilization and citation. However, none of these proposals
has prevented authors of Big Data-related works to extend, renovate or even ignore previous definitions and propose
new ones. Although Big Data is still a relatively young concept, it certainly deserves an accepted vocabulary of
reference that enables the proper development of the discipline among cognoscenti and practitioners.
In the first part of this paper we have identified the four main themes of Big Data and we have observed that they
are the prevalent topics in the existing literature. In the next paragraphs we will review a non-exhaustive list of
previously proposed Big Data definitions and we will conceptually tie them to the aforementioned four themes of
research. After considering the existing definitions and analyzing their commonalities we will propose a consensual
definition of Big Data. Consensus in this case comes from the acknowledgement of centrality of some recurring
attributes associated to Big Data, and from the assumption that they define the essence of what Big Data means to
scholars and practitioners today. We expect that such a definition would be less prone to attack from previous
definitions’ authors and users as it is based on the most central aspects associated until now to Big Data.
A thorough consensus analysis based on Cohen’s K coefficient [31] and co-word analysis, as in [30], goes
beyond the scope of this work and is left for future study.
Survey of Existing Definitions
Big Data has been often described “implicitly” through success stories or anecdotes, characteristics,
technological features, emerging trends or its impact to society, organizations and business processes. In the existing
attempts of explicit definitions for Big Data there is not even an agreement on what entity this term can be
associated with. We have found that Big Data is used when referring to a variety of different entities including - but
not limited to - social phenomenon, information assets, data sets, analytical techniques, storage technologies,
processes and infrastructures. We have surveyed multiple definitions that have been proposed to date and listed them
in Tab. 1: in this paragraph we will go through the most notable ones.
A first group of Big Data definitions focuses on enlisting its characteristics. What is probably the most popular
definition falls within this group. When presenting the Data Management challenges that companies had to face in
response to the rise of e-commerce in the early 2000’s, Laney introduces a framework expressing the 3-dimensional
increase in data Volume, Velocity and Variety and invokes the need for new formal practices that will imply
“tradeoffs and architectural solutions that involve/impact application portfolios and business strategy decisions”
[32]. Although this work did not mention Big Data explicitly, the model, later nicknamed as “the 3 V’s”, was
associated to the concept of Big Data and used as its definition [33–35]. Many other authors extended the “3 V’s”
model and, as a result, multiple features of Big Data, like Value [36], Veracity [37], Complexity and
Unstructuredness [38, 39], were added to the list.
A second group of definitions emphasizes the technological needs behind the processing of large amounts of
data. According to Microsoft, Big Data is about applying “serious computing power” to massive sets of information
[40] and also the National Institute of Standards and Technology (NIST) highlights the need for a “scalable
architecture for efficient storage, manipulation, and analysis” when defining Big Data [41].
A few definitions associate Big Data to the crossing of some sort of threshold: for instance Dumbill [42] asserts
that data is Big when it “exceeds the processing capacity of conventional database systems” and requires the choice
101
What is Big Data? A Consensual Definition and a Review of Key Research Topics: summary, contents and background
This guide helps you understand the scope of What is Big Data? A Consensual Definition and a Review of Key Research Topics before or alongside reading the original document. It is based on the catalogue record, credited creator, source, classification and topics shown on this page. It does not replace the original work and does not invent claims about passages that Doxxly cannot verify.
Summary and overview
What is Big Data? A Consensual Definition and a Review of Key Research Topics is a 9-page documentcredited to Soren V. Cressley. This 2015 conference paper reviews Big Data research and existing definitions to clarify the concept. Using a corpus of 1,437 abstracts from Scopus, the authors identify four principal themes: Information, Technology, Methods and Impact. They discuss data generation, distributed processing, analytical skills, business value and societal concerns, including privacy and unequal data access. The paper proposes defining Big Data as information assets whose high volume, velocity and variety require specific technology and analytical methods for transformation into value. The review is explicitly non-exhaustive rather than systematic, and formal quantitative consensus analysis is left for future work. It appears in the Technology & Security collection, which groups it with other material that readers may use for comparison and further study.
The item can be approached in several ways. A reader looking for a quick orientation can begin with the free five-page preview, inspect the page structure and use the metadata to judge relevance. Someone undertaking deeper research can work through the complete file, search its indexed text, record useful page references and compare its treatment of Big Data, Analytics, Information Management, Data Processing, Business Intelligence, Research Review with the related resources linked below.
Rather than reducing the work to a single claim, this page treats it as a source that should be read in context. The title, creator, publication source, licence and format all help explain what kind of evidence it may offer. Readers should distinguish the original author's statements from Doxxly's catalogue description, and should cite the original edition when using the material in academic, professional or published work.
Key topics and reader takeaways
The catalogue connects this document with Big Data, Analytics, Information Management, Data Processing, Business Intelligence, Research Review. These terms offer practical entry points rather than a substitute for close reading. Start by identifying the definitions, examples, arguments, tables or visual explanations attached to each topic. Then note where the creator draws connections, makes distinctions or moves from background material to a conclusion, recommendation, calculation or narrative development.
A useful reading method is to separate description from interpretation. First record what the item actually presents: names, headings, figures, formulas, quotations or stated propositions. Next consider why those details matter within the whole work. Finally compare them with another source from the same subject area. This process helps turn a long digital file into evidence that can be evaluated rather than merely skimmed.
Table of contents and reading guide
The following is an editorial content guide based on the verified catalogue topics. It is not represented as the original author's formal table of contents. Use it to organise notes while moving through the 9 pages in the player.
Part 1
Big Data
Use this section of the reading guide to identify how Big Data appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.
Part 2
Analytics
Use this section of the reading guide to identify how Analytics appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.
Part 3
Information Management
Use this section of the reading guide to identify how Information Management appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.
Part 4
Data Processing
Use this section of the reading guide to identify how Data Processing appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.
Part 5
Business Intelligence
Use this section of the reading guide to identify how Business Intelligence appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.
Part 6
Research Review
Use this section of the reading guide to identify how Research Review appears in What is Big Data? A Consensual Definition and a Review of Key Research Topics, note important terms, and connect the subject to the wider purpose of the work.
When a heading in the original differs from this guide, follow the original heading in your citation. Page numbers can also vary between editions. Doxxly's direct page field is useful for navigation, but formal references should include the title, creator, source and edition details available in the file itself.
About Soren V. Cressley
Doxxly credits Soren V. Cressley as the author or creator of What is Big Data? A Consensual Definition and a Review of Key Research Topics. This page does not add an unverified biography, job title or institutional affiliation. The attribution is preserved so readers can distinguish the creator from the hosting library and follow the original record when an external author link is available.
Author context matters because a work reflects a particular purpose, audience and point of view. Before relying on the material, consider what the creator set out to explain, when the source was produced and whether later research or revised guidance should also be consulted. For literature and philosophy, compare translations and editions; for technical material, check whether a newer version has superseded the one shown here.
About the source organisation
The recorded source for this item is dhi.ac.uk. Doxxly provides access and discovery tools but does not claim to be the original publisher unless explicitly stated. Source links, creator attribution and the recorded Publicly available file collected for editorial review licence are displayed to help readers assess provenance and permitted use.
Provenance is especially important for reports, standards, presentations and working spreadsheets. Check the original source for amendments, supporting files and current policy. A trustworthy research workflow keeps the source URL with any notes or downloaded copy, verifies important figures against the original, and avoids treating an archived document as automatically current.
Begin with the title and opening pages, then use the guide above to decide whether to read sequentially or move to a particular topic. The Doxxly player provides previous and next navigation, a direct page field, zoom steps, fit-to-width and fit-to-page modes, rotation and fullscreen. These controls are useful when a page contains small type, a wide table, an image, a scanned layout or a dense sequence of notes.
The first five pages are open to every visitor. A plan unlocks the complete item and full-document search. Search is best used for distinctive names, technical terms or short phrases; broad words can produce too many possible contexts. When you find a useful passage, read the surrounding pages so that a sentence, chart or formula is not separated from its explanation.
If downloading is enabled, members can obtain the original PDF file. Keep the original format when formulas, slide sequencing, pagination or layout carries meaning. Respect the Publicly available file collected for editorial review terms displayed on this page, and use the rights-report link if you believe the attribution or availability needs review.
Research questions to consider
As you read What is Big Data? A Consensual Definition and a Review of Key Research Topics, ask which audience the creator addresses, what problem or idea organises the work, what evidence is presented, and which assumptions remain unstated. Note whether important terms are defined consistently and whether examples represent a broader pattern or a narrow case. For quantitative material, record units, time periods, formulas and data sources. For interpretive material, separate the creator's language from your own summary.
A second set of questions concerns relevance today. Which parts remain durable, which depend on their historical or institutional setting, and what would need confirmation from a newer source? Following the tags and the Technology & Security collection can reveal useful comparisons. Agreement between sources can strengthen confidence, while disagreement can point to changing definitions, methods, evidence or priorities.
Finally, decide what you need from the item: background knowledge, a quotation, a worked example, a model, a visual explanation or a starting bibliography. That purpose should guide the depth of reading and the notes you keep. Doxxly helps locate and navigate the source; careful interpretation remains the reader's responsibility.