Purpose: Big data offer a huge challenge. Their very existence leads to the contradiction that the more data we have the less accessible they become,as the particular piece of information one is searching for may be b...Purpose: Big data offer a huge challenge. Their very existence leads to the contradiction that the more data we have the less accessible they become,as the particular piece of information one is searching for may be buried among terabytes of other data. In this contribution we discuss the origin of big data and point to three challenges when big data arise: Data storage,data processing and generating insights.Design/methodology/approach: Computer-related challenges can be expressed by the CAP theorem which states that it is only possible to simultaneously provide any two of the three following properties in distributed applications: Consistency(C),availability(A) and partition tolerance(P). As an aside we mention Amdahl's law and its application for scientific collaboration. We further discuss data mining in large databases and knowledge representation for handling the results of data mining exercises. We further offer a short informetric study of the field of big data,and point to the ethical dimension of the big data phenomenon.Findings: There still are serious problems to overcome before the field of big data can deliver on its promises.Implications and limitations: This contribution offers a personal view,focusing on the information science aspects,but much more can be said about software aspects.Originality/value: We express the hope that the information scientists,including librarians,will be able to play their full role within the knowledge discovery,data mining and big data communities,leading to exciting developments,the reduction of scientific bottlenecks and really innovative applications.展开更多
文摘Purpose: Big data offer a huge challenge. Their very existence leads to the contradiction that the more data we have the less accessible they become,as the particular piece of information one is searching for may be buried among terabytes of other data. In this contribution we discuss the origin of big data and point to three challenges when big data arise: Data storage,data processing and generating insights.Design/methodology/approach: Computer-related challenges can be expressed by the CAP theorem which states that it is only possible to simultaneously provide any two of the three following properties in distributed applications: Consistency(C),availability(A) and partition tolerance(P). As an aside we mention Amdahl's law and its application for scientific collaboration. We further discuss data mining in large databases and knowledge representation for handling the results of data mining exercises. We further offer a short informetric study of the field of big data,and point to the ethical dimension of the big data phenomenon.Findings: There still are serious problems to overcome before the field of big data can deliver on its promises.Implications and limitations: This contribution offers a personal view,focusing on the information science aspects,but much more can be said about software aspects.Originality/value: We express the hope that the information scientists,including librarians,will be able to play their full role within the knowledge discovery,data mining and big data communities,leading to exciting developments,the reduction of scientific bottlenecks and really innovative applications.
