BIG DATA

Description: Big Data is the next big thing in computing. This video explains Big Data characteristics, technologies and opportunities.

Source: http://www.explainingcomputers.com

Due to the issues raised by its volume, velocity and variety, Big Data requires new technology solutions. Currently leading the field is an open-source project from Apache called Hadoop. This is developing a software library for reliable, scalable, distributed computing systems capable of handling the Big Data deluge, and provides the first viable platform for Big Data analytics. Hadoop is already used by most Big Data pioneers. For example, LinkedIn currently uses Hadoop to generate over 100 billion personalized recommendations every week.

What Hadoop does is to distribute the storage and processing of large data sets across groups or “clusters” of server computers using a simple programming model. The number of servers in a cluster can also be scaled easily as requirements dictate, from maybe 50 machines to perhaps 2000 or more. Whereas traditional large-scale computing solutions rely on expensive server hardware with a high fault tolerance, Hadoop detects and compensates for hardware failures or other system problems at the application level. This allows a high level of service continuity to be delivered from clusters of individual server computers, each of which may be prone to failure. Processing vast quantities of data across large, lower-cost distributed computing infrastructures therefore becomes a viable proposition.     READ REST OF STORY 

Questions for discussion:

  1.  What is Big Data and why is it important?
  2. What potential applications do you see for Big Data and in what industries will this add the greatest value?

24 thoughts on “BIG DATA

  1. Tamara First Charger

    Asking the ethical dilemmas question on companies using big data in customer service is like asking if you have ethical dilemmas with the cashier that is scanning your items at the checkout counter. The cashier analysis your purchases and turns to you inform you of a sale next week on some similar items that you purchased. You appreciate her for passing the information on to you and you leave. Then you come back following week for those items, go to the cashier to pay for your sale items and leave. The only difference between how this information, was on how this information was delivered to you is in a email form.
    Company is watch what we are purchases well what is the problem we are in public out in the open, in a store for every to see what we are purchasing. I’m sure when individuals go out shopping and customers are going up and down the aisles and you pass another shopper with their cart of items, and you take a look into their cart and see what they are purchasing. Then you take your items to the ca

    Understanding how and why big data is being applied in customer services, would depend on the ethical dilemma but I don’t see any ethical dilemma when a company is assisting me in saving time and money. It’s my choice of as a customer whether I exercise my buying power or not and where.

    Reply
  2. Jennifer B

    Big data is about the enormous amount of data being generated today that cannot be analyzed by traditional computing techniques. There are three aspects about this data that make it difficult for traditional computing techniques to analyze it. The first is the large volume of data being collected these days which presents the greatest challenge but also the greatest opportunity for Big Data. Not is also being collected at huge volumes but also at an increasingly high speed, and lastly there is an enormous variety of data that is being collected. Big data allows us to generate and interpret this data into useful information. Big Data is expected to be the next big thing in computing.

    Many industries can benefit from using Big Data with government already investing in the use of this tool the opportunities seem endless. The article says that the good out weighs the bad, however, that statement concerns me. I see the possible good that can come of this if in the right hands, but what about the possible negative effects if in the wrong hands. I think its great farmers can see when bad weather is coming, and if their crops will be successful, but what about a corrupt government having such a large amount of insight and the opportunity of acting on it before its even happened.

    Reply
  3. Nidhi Patel

    Big Data is deemed to be the next big thing after cloud computing in the transition to an online computing infrastructure. It generates value from the storage and processing of very large quantities of digital information that can’t be analyzed with the average computing techniques. Retailers are starting to build up vast databases of recorded customer activity. Public use of social media is also creating large quantities of digital material that may be mined to generate valuable insights. Big data is also important due to the advancement in nanotechnology, synthetic biology and genome sequencing. Volume is big data’s greatest challenge as well as its greatest opportunity. Storing and processing vast quantities of digital information offers tremendous possibilities for a wider range in activities, however traditional solutions are not capable of handling such tasks. Velocity, rate, is also increasing beyond capacity of their IT systems to store and process. Big Data requires new technology solutions. It is an opportunity to find insight in new and emerging types of data, to make businesses more agile, and to answer questions that, were beyond reach in the past. It is a way to gain more knowledge about the technology to come in the future. Big Data also holds the promise of giving enterprises deeper insight into their customers, partners and business that they may not have even thought to ask. It is important because it has the potential to improve analytical insight as well as allowing the creation of new products and services that were previously impossible. Big Data is useful and helpful to all.

    Reply
  4. Krysten Lumsden

    Big data is about the vast amount of data being generated these days. It is also about the increasingly high speed at which data is required and finally about the variety of data that is required that goes beyond the traditional types of data being handled by computers, in particular databases. Big data is expected to be the next big thing in computing, an understandably so.

    It was very interesting to note that despite the fears about invasion of privacy due to increasing amounts of information being collected by governments and corporations, in reality this articles suggest that lots of that data is not currently being processed. Further, in so far as it is being processed, that data is not being processed in a way that yields much useful analytics. That said, the article suggest that this is changing and that new Big Data systems including cloud computing will allow data mining to become much more sophisticated and capable.

    The article suggest that there are many positive uses of the data mined from Big Data, but it almost feels to me like one of those sci-fi novels/movies/tv-shows where the ‘mad’ scientist creates some mind blowing technology with very good intentions to improve social welfare only that the technology ends up doing the opposite. Consider when such technology it issued by criminals or when it reveals some unintended purpose that can be used for ‘evil’. This makes me also think about the whole ‘Big Brother’ concept and the expected advancement in Artificial Intelligence. Will Big Data become so big that it could be used as weapon of mass destruction or even as a way to control, police and oppress the masses? Imagine if hackers were to infiltrate the security of Big Data systems. Imagine the havoc they could create in society. What about corrupt governments and corporations who have access to Big Data. There seems a lot of good to come from Big Data, but what about the potential for abuse and misuse. Instead of glorious future, we could instead create a very scary one.

    Reply
  5. Katrin Saballa

    Big Data is a term for very large quantities of data that cannot be analyzed with traditional computing techniques. It is important because the quantity of computer data being generated today is a lot. Databases from different sectors in different parts of the world are growing bigger and bigger. Big Data allows us to generate this huge amounts of data. It is important because without it, most traditional computing techniques would not be capable of handling vast quantities of digital information and interpreting it so that the data would be useful. Retail companies use Big Data to keep track of purchases, stocked products, customer information and etc. Social media also uses this to keep track of their users and it also allows them to monitor each user’s activity.

    Big Data can be used anywhere that requires handling of a large amount of data. A university is an example of this. A university needs to keep records of their students, faculty, and other staff members. They also need to have a record of every class that a student has taken, what classes a professor had or will teach, student schedules, exam schedules, and a whole lot more. These are all data that needs to be collected and stored, as well as be accessible when they need it. For example, the months or the weeks leading up to graduation can be hectic. The university needs to be able to easily access a student’s list of courses to see if he or she qualifies for graduation or if he or she is missing a class. Without Big Data, all these processes can take a very long time. With Big Data, we can compile, manage, and process data more efficiently.

    Reply
  6. Gordon Entz

    Big Data presents endless opportunities but also has a number of challenges. Collecting the information is not the challenging part. What is actually a challenge is getting something being able to interpret the data and come up with useful information. As pointed out in the video a large amount of the information is lost as data exhaust. Figuring out ways to increase the efficiency is difficult task. Big data has great potential and there is a great deal of money to be made by making use of this data. Sports teams can use data to get a competitive advantage against their opponent. Marketers everywhere can use this information to help them be more specific and and how to market better to their target audience.one of the other problems is that the amount of information collected is too great and companies are running out of space to store the data. So volume and velocity present problems the potential is great and once the efficiency in interpreting the information is increased the data will be a great asset to any business or organization. if the government is willing to invest 200 million in big data that gives an indication of the importance they see in big data. Big data is the future of IT and will not only save an abundance on efficiency in the future by using big data, they will also be able to make a great deal of money as a result of big data. The only problem will be to figure out how to not lose the majority of information due to data exhaust.

    Reply
  7. Ben Lo

    What is BIG data? BIG data captures, stores and processes a very large number of digital information which cannot be analyzed by using the traditional methods that have been used in the past for computing. BIG data can be described using 3 V’s, volume, velocity, and variety. Volume is the biggest challenge for big data due to the amount of databases that are needed for everyday life. Databases are compiled through multiple sources, for example, day to day business such as Walmart and Target collect consumer information daily and store this information in their databases. Another example of high volume of data is through social media, every day more and more information or content is created on websites like Facebook, Youtube, Linkin, Twitter and Soundcloud. Velocity is also another key issue due to the increased popularity of streaming websites like Youbtube and Netflix. Consumers are expecting fast high quality videos and music which is why velocity is a key issue in big data. Information and data needs to be transfer at a speed that is acceptable to the consumers. Variety is also another issue in big data. There are many different types of data that need to be analyzed and processed, for example Facebook has to process many data types like photos, videos, storing chat data and many more. Big data needs to be able to process many types of data effectively and efficiently to be able to function to maximum potential. This is important because with the increasing amount of data that needs to be captured, processed, and stored is increasing everyday it is important to understand the important factors that will affect this process.

    Reply
    1. Nadine

      I find it very disconcerting that a company can obtain so much Information that a company could obtain on you simply by purchasing a product. I have signed up for various cards in order to obtain bonus points, I was not aware that my address, DOB, and such would be for sale to other companies. This article has really put piqued my interest in the amount of information a company could get on your purchases. This habit stuff is really eye opening, as I was reading this article I thought about some of my shopping habits and the focus on pregnant women was so spot on. I lived in Calgary when I had my last child who just turned 15, I recall at the time that the Real Canadian Super store had just opened and we went shopping there. I liked to go to that store as what Target offered as one stop shopping, the RCSS offered the same convenience. My thought at the time was the convenience of being able to go to one store with an infant in tow and be able to buy food, toiletries, diapers, milk, baby clothes and everything I needed in one stop. Prior to my pregnancy, I shopped at various stores for my necessities depending on what coupons were in my mailbox or what flyers came in the mail, but I was only able to that without a child in tow. After my baby was born, I did not have time to go to various stores to purchase my items. Kind of a scary thought that a company can garner that much information on a person…

      Reply
  8. Kayla Rothe

    Big Data allows for organizations to gather a great amount of information by gathering, processing and storing massive amounts of data that is analyzed in non-traditional ways. This Big Data than previous forms of data because it is able to come in at high volumes, velocity, and varieties that often businesses are receiving too much information to manage once.
    I think that all industries can benefit from using Big Data. Since such a variety of information can be received from around the world, Big Data can benefit numerous industries. However, I believe that for Big Data to add the greatest value, it must be analyzed and implemented in a efficient and timely manner.

    Reply
  9. Tolu Adepoju

    Big data is the exponential growth and availability of data. It is used compel data on consumers that could be used to interpret information. Big data is an effective resource to make faster and better decisions to create new products and service. Big data is compelled into three sectors which are velocity, volume and variety. I believe that the use of data a crucially tool that is really helpful when it comes to healthcare matter. Applications like the storing and retrieval of a patient’s file or records makes work process more efficient. Big data reduces the cost of gathering more information on customers. It makes it easier for sellers to know what products to produces to serval market demographics and time to offers those products. Government and healthcare systems have started to invest greatly in the use of this tool. Although, there are some drawbacks such as data exhaust. In the article an example was given where video captured by hospital is deleted within weeks. I think as time goes on more institution will use big data as a tool to effectively render products and service to customers. Nowadays it is very easy to gather information on a person just by checking social media platform such as Twitter, Facebook, or Instagram. People are quick to post there information out there today. I believe big data should be use by industries in an ethical manner, to find out with to effectively provide customers with right services and lower costs for customers not to track the customers every move.

    Reply
  10. Kahn Daviduck

    Volume, Velocity, and Variety, the three big Vs of Big Data. The amount of data that corporations can now store is practically limitless. They can now analyze our buying habits, market trends and forecast what consumers or potential consumers are going to do. Big Data can be collected almost anywhere as information is all over the internet. The amount of information that these companies process is phenomenal. All of this information can make a company much more efficient, profitable and more sustainable in an uncertain economy. Of course all major companies are probably using this tactic so it just further increases the competition in the market place. If the use of all of this personal information does improve companies’ services and reduce cost I would gladly encourage the use of these techniques. If costs of goods remain high and corporations use this data only to benefit themselves this may upset many people. There are already many ethical dilemmas associated with Big Data because the practices used to acquire this information often involves just taking data without asking clients for their consent and then possibly selling this information. All in all the use of Big Data is an overwhelming concept with so many variables it is quite impressive that these corporations can get so much meaning from all this data. As technology advances and more and more of our information is accessible on the internet, companies will also have more access to this information and be able to make even more detailed decisions about our buying habits or even day to day routines and will adjust their marketing strategies accordingly.

    Reply
  11. Beili Bai

    Big data is a tool which has high volume, high velocity, and variety to collect different data in our daily life. This article introduces the 3V’s of Big Data, the Volume, Velocity, and Variety. It can store huge data for the company. These data helps a company to processed and analyzed. The Real-time or nearly real-time information can make the company more efficiently. Big data has many different kinds of form. It can be collected in different positions of our daily life. Big data is used to sets or combine datas which volume, variability, and velocity make them difficult to be captured. It is also used to manage, processed and analyzed these data sets. For business, big data can make the company more efficiently. These companies are ablie to gain more profit and more complete understanding of their business. According to the analyzed of big data, companies can increase their sales, lower their costs, and get better customer service.

    Reply
  12. Beili Bai

    Big data is a tool which has high volume , high velocity and variety to collect different data in our daily life. This article introduce the 3V’s of Big Data, the Volume, Velocity,and Variety. It can store huge data for the company. These data helps a company to processed and analyzed. The Real-time or nearly real-time information can make the company more efficiently. Big data has many different kinds of form. It can be collected in different positions of our daily life. Big data is used to sets or combine datas which volume, variability, and velocity make them difficult to be captured. It is also used to manage, processed and analyzed these data sets. For business, big data can make the company more efficiently. These companies are ablie to gain more profit and more complete understanding of their business. According to the analyzed of big data, companies can increase their sales, lower their costs, and get better customer service.

    Reply
  13. Kim Sikhosana

    Big data stores and processes large quantities of digital information that cannot otherwise be processed by traditional computers. The characteristics and challenges of the big data are described by the three Vs; volume, velocity and variety. Volume is the greatest challenge, yet carries alot of potential because of the large quantity of information being received. Velocity describes the fast rate that data is being shared or required. As we become more dependent on technology and as the more advanced our smart phones get, we will be relying on large quantities of data. Variety are the different types of data that we are using.
    Big data is an important part of the process to becoming more technologically dependent.

    Reply
  14. Fangbo He

    Big Data is a huge revenue generator for business, and the big data is going to become a vital part of any business strategy. The big data is important because it related how to develop a decent industry. Big data is the basis for business intelligence, which is about taking all that information and turning it into knowledge to drive better business decisions. Whether it’s data about retail consumers or homebuyers, it’s all the same game.This article also emphasized that the big data from three Vs, which is volum, velocity and verity. poeple who are going to collect perfect data need know how this three Vs work definitely. The big data has made a lots of value in many industries. Such as hospital, airport, police office, restaurant, shopping centre, and so on. Overall, the big data can help business meke better understand for their market from the outside in and across many disciplines including public relations, customer service, marketing, advertising, research, product innovation, and sales. I think the big data will also important in the future.

    Reply
  15. Wancong Lu

    Big Data create value and benefit from very large information in storage and processing, which definition as the three Vs: volume, velocity and variety. Volume is one of the biggest thing in Big Data because when Big data processing the large amount of digital information, it can give some possibilities for activities. for example, analyzing customer behaviour, disease, and climate. Velocity is the data must be timely manage due to the unprecedented speed, and Variety is all types of formats of data, such as photographs, audio, video, 3D models, complex simulations and location data, etc.
    Big Data is important because it can save money, increase efficiency and quality. The recent report in the article, “the US healthcare sector could achieve $300 billion in efficiency and quality savings every year by leveraging Big Data”, “Across Europe, they also estimate that using Big Data could save at least $149 billion in government administration costs per year.” That is a lot of money for every governments. The most beneficial part is in every manufacturing firms. Firms can reduce their production time, and increase their products quality.
    Using Big Data, people can find the useful resources by what they need instead of a bunch of information. Not only that save us money, but also save us a lot of time.
    Although Big Data’s positive implications seems like outweigh the negative implications, there are still some negative impact. People worry about their personal information in everywhere in the internet, and the fact is yes. That is an invasion of privacy.

    Reply
  16. Joalee Mann

    Big data is the term referred to by the vast amounts of information being collected and analyzed by non-traditional computing. It is characterized by volume, velocity and variety. It is important because in all organizations, both public and private, data about customers, competitors and other external factors can be available to them to remain competitive. It has the capabilities of reducing administrative expenses, and increasing overall efficiency when it is being utilized. This all presents challenged for organizations that are not already investing in big data to be able to compete in the future.

    The importance of this can be seen in industries such as health care. In Canada where the large majority of health care is public and already prone to disadvantages of inefficiencies, big data could be the answer to being able to eliminate redundancies and wait times. Think of a trip to a walk-in clinic: you are asked for your healthcare number, insurance information, your family physician, what medications you are on, and your recent medical history. With big data, that information about you should already be available from your previous visits with other healthcare providers, and pharmacies. There really is no excuse for these inefficient processes well computing systems are available to host big data.

    Of course, the downside being that many people do not agree that information about them should be collected to that great of an extent and that it is an invasion of privacy. The challenges moving forward with organizations and big data will not only be about utilization, but also about privacy protection.

    Reply
  17. Bola Fowosere

    The mainstream definition of Big Data includes three Vs, which are volume, velocity and variety. Big Data refers to an extremely large data sets that reveal patterns and trends of human behaviors and interactions. In other words, it is a massive volume of structured and unstructured data that is difficult to process or interpret using traditional database and software techniques. Product transaction information and financial records are some examples of Big Data.
    Nonetheless, Big Data can also be defined as a large collection of data from traditional and digital sources inside and outside a firm. While government benefit from using Big Data, large firms and most organizations also benefit from using this technology to handle enormous amount of data and storage facilities. Not only does it allow them to use a great source of data to analyze their customers’ buying behavior, but also have a better and deeper insight into their customers, partners and business.
    While it serves as a cost reduction, time reduction as well as a means of generating product ideas and development for firms, the ability to use information proficiently and intelligently is what matters the most. In other words, the use of Big Data is not just about acquiring large amount of data, but being able to understand how and what those data could be used for.
    Although this could be challenging as it becomes difficult to seek the most valuable and important piece of information from a large volume of data, it’s worth understanding what, when and how to use the information intelligently and effectively to make faster and smarter business decisions.

    Reply
  18. Ajuot

    When the television was invented, the so called “the moving image” that is the human image was unidentifiable or hardly identified, but as the time goes on, it becomes clear and clearer. So, like the tv, the Big Data may be challenging now, but as the time goes on, it would not be something of the big issue to the companies as the motive of capitalization and profit making would motivate them to come up with resources that are needed to address the challenges of the Big Data.

    Reply
  19. Arth Patel

    Big data generates value from the storage and processing of very large quantities of digital information that cannot be analyzed with traditional computing techniques. Big data gives the enterprises a deeper insight into their customers, partners, and business. The technology is very new, many companies are adapting to it, and it is growing exponentially because of its use in e-commerce. Companies are building large sets of databases of recorded customer activity. Departments such as logistics, financial services, healthcare, and many others also gathering large sets of databases and generate value from it. Big data is also generated by an expanding internet of things. The three characteristics and challenges of big data is described by volume, velocity, and variety.

    Volume is the biggest challenge of big data and also the greatest opportunity. Storing, interlinking, and processing large quantities of information provides wide range of activities, but the traditional computing solutions are not capable of handling the wide range of activities. Velocity raises issues as well because people want real time data right at their fingertips (mobile phone, tablet, etc.), but the rate at which data is flowing into organizations is beyond the capacity of their IT systems to store and process them. The variety of data that many companies are called on to process are becoming more dense and diverse. Most of the big data sources are unstructured, and hence not easy to categorize them, which makes them difficult to process with traditional computing power.

    Reply
  20. Rui. Zhang

    Big data is a huge set of data with characteristics of “4V” (Volume, Velocity, Variety and Value). However, just because of these four characteristics of big data, it cannot be defined as data set simply. The Big Data is a kind of information property which needs to be analyzed with data processing model to enhance the decision-making, insight of opportunities and process optimization. The meaning of the Big Data technology is not to “keep” the massive data or information but to process and analyze those information.
    The world that we live in is full of information. Nowadays, computers are not only used for sending email or go on the internet, but also used for create a new world and decrease the distance between people. Thus, here comes big data. Big data means that something has a lot information which normal software tool can not store it, and it is becoming more and more popular. For example, twitter, facebook even youtube are all using big data, but these are only common things that some people may use it in everyday’s lives. People are also using big data for more complicated things, such as vision recognition and scientific research. People like to use big data because it can store lots of information. It is easy to see that big data is all over in our lives, although we may never hear it before.

    Reply
  21. Hongyu Xu

    After reading this article, I have a brief understanding of the big data. In my opinion, the big data is a growing number of digital data that are collected from the internet or daily life. It is important not only to business but also to the whole society. There are lots of ways to collect data to the database. It can be collected from the commerce, health care, financial service and many other ways from our daily life. For examples, the retailer may collect their customers’ information and build up a big database. The bank also can collect the clients information when people opening an account or applying for a credit card.
    This article also defined the big data from three Vs, which is VOLUME, VELOCITY AND VARIETY.
    The volume is the number of the data. It is the most challenge thing for the big data. Many factors contribute to the increasing of data. And also the data streaming at a really quick speed which requires us to collect them with a high velocity. This is another challenge for organizations.
    Third characteristic is variety. Today’s data are in many different kinds of format like text message, video, pictures, voice message and so on. This require organizations familiar with all kinds of format of data and can recognize them really quick.
    As we can see from the article, the big data has become the second important big thing is the IT world just after the Internet. We should also be aware of the exploration and data exhaust as the number of big data are getting bigger and bigger.

    Reply
  22. Glenn ZoBell

    Big data is the ability for systems to receive, analyze and store extremely large sets of data to help understand consumer patterns. It is very important because until recently, technology could not compute the volume, velocity and variety of data efficiently so that it could be used to create productivity.
    I think big data can be a benefit to every single business and industry but that it definitely has larger value in a few areas. Big data could be a huge benefit for governments to understand relationships and behaviors or individuals or groups to understand and predict human behavior. This could allow governments to become much more optimal in every area. This could be a huge benefit for political parties and the economy. Farmers could also greatly benefit from this data by being able to predict seasons and when to plant/harvest. The last area I feel would have benefit the most from big data would be large enterprises or corporations who could analyze consumer behavior to create more profit. Stock markets could also use big data to predict increases or decreases in stock values. Big data could be a huge advantage for all these industries because they would have the largest sources of data. Although private businesses could also benefit from being able to analyze consumer buyer behavior, their data population would not be near as big and might not even require big data to analyze and store their consumer data.

    Reply
    1. Nathan Watmough

      As information technology increases, our limits decrease. Now companies are beginning to have access to literally a ton of data and more importantly are beginning to have systems in place for the interpretation of this data to make it useful. Who knows what the limitations to BIG DATA could be? My uncle works for direct energy and he has explained to me the immensity of all the data they receive for meter reads on an hourly basis. I still don’t really understand it and can’t explain it either but what I do know is it is a ton of data that needs to be interpreted and deciphered mostly by humans. He told me that there are systems being developed that could make his and his department’s jobs obsolete. Something interesting he told me though was that this didn’t mean he is going to become a homeless bum, it just means the company will be able to focus their resources (i.e. the labor of my uncle) into other areas in order to become even more efficient. For me at least it is hard to get my head around it but all I know is that it is amazing

      Reply

Leave a reply to Tamara First Charger Cancel reply