Big Data Explained

Description: Big Data is the next big thing in computing. This video explains Big Data characteristics, technologies and opportunities.

Source: http://www.explainingcomputers.com

Due to the issues raised by its volume, velocity and variety, Big Data requires new technology solutions. Currently leading the field is an open-source project from Apache called Hadoop. This is developing a software library for reliable, scalable, distributed computing systems capable of handling the Big Data deluge, and provides the first viable platform for Big Data analytics. Hadoop is already used by most Big Data pioneers. For example, LinkedIn currently uses Hadoop to generate over 100 billion personalized recommendations every week.

What Hadoop does is to distribute the storage and processing of large data sets across groups or “clusters” of server computers using a simple programming model. The number of servers in a cluster can also be scaled easily as requirements dictate, from maybe 50 machines to perhaps 2000 or more. Whereas traditional large-scale computing solutions rely on expensive server hardware with a high fault tolerance, Hadoop detects and compensates for hardware failures or other system problems at the application level. This allows a high level of service continuity to be delivered from clusters of individual server computers, each of which may be prone to failure. Processing vast quantities of data across large, lower-cost distributed computing infrastructures therefore becomes a viable proposition.     READ REST OF STORY 

Questions for discussion:

1.  What is Big Data and why is it important?

2.  What potential applications do you see for Big Data and in what industries will this add the greatest value?

Advertisements

19 thoughts on “Big Data Explained

  1. Paul Hruby

    Big data refers to vast, insane amounts of data characterized by the three v’s; volume, velocity, and variety. The share mass quantities of data available give us (humanity) both the greatest opportunity as well as the greatest challenge (how are we supposed to manage so much data??) Companies like hadoop are outsourcing the processing of this big data to multiple computers in a network which allows each to sort through a given part, sort of in the same manner as torrenting files which allows parts of a file to be downloaded from multiple peers at one time resulting in each computer giving a small share of the larger file. The speed in which this data comes in is also a problem in that by the time we’re done computing a given huge amount of data there’s likely already as much new data available to be processed and companies just aren’t fully utilizing the quantities of data available and leaving them half processed. Also there is now an incredible amount of variety out now, you can not just get documental information but also photos, videos, 3d models, etc. the amount of data has never been greater. We are coming up with other means like quantum computers (which i believe have to be insanely cold – like 2 degrees kelvin – not quite a practical computer to have around at this point). But this data is so important because we’re in the age of information and we can see the benefits in programs like linked in for marketing, but also many other applications. I read somewhere that they’ve made a temperature and humidity censor which is about a square centimetre in surface area and aprox. 1mm thick. If they put this into cellphones we’d be able to instantly get the temperature and humidity of the immediate area around us, the most accurate current weather report ever. If this information from millions of cellphone users was then relayed and processed properly at weather stations, we’d definitely be able to come up with 1000’s of times more accurate long term weather predictions. This would work similarly to how google is now so easily able to predict traffic flows from such a large network of users, this stuff is ridiculous! I also think we’d be able to predict what flues are coming out and prevent a lot of sickness as well, I can’t wait to see where we’re at in the next 20 years!

    Reply
  2. Amber Siemens

    Big Data is the next big thing in computing. Big data generates value from very large data sets that can not be analyzed by traditional corporation techniques. Big data is based on Volume, Velocity and Variety. It is important because the amount of computer data generated on earth is growing so fast, and greatly for many different reasons. The importance of Big Data is based on how companies utilize the information within the data program. Potential applications that can be created with analyzing this data could add to more revenue within the company. If they are able to break down their categories of profit, and see what is generating the most revenue, their company can be extremely successful. For example, Firestone would print out their reports, and see what are the most popular entrees on the menu, and they would keep those, and when they would re vamp their menu, they would get rid of the entrees that don’t sell to the percentage they should for maximum profit. They also have a Red Box Rewards system, which tells them how often their guests are coming in, how many people they are bringing, what they are ordering each time. This example, touches on how companies are starting to embark on analyzing big data.
    I believe the consumer industry, the banking/financial industry, and the health care system would benefit from it vastly. From retail, food, using credit cards, to electronics, they are the industry that has patterns that are very easy to follow. They see what people buy, whether its on a daily basis, or a few times a year. Credit card companies can analyze consumers statements and know what they spend their money on in a month. A lot of consumers live a life of pattern, consistency, where they spend and do the same things each month. With the health care system, now that we are taking photos of the patient and what they are doing to that patient, and having files with information, the data is getting bigger and bigger to work with. With Big Data, they are able to hold the data and be able to properly retrieve the information at the proper time, to the best benefit.

    Reply
  3. ...

    Big Data has three characters: volume, variety and velocity data. Data is complex and big. It is not hard to find the data through computer. And that is because it is usually collected by using computer. It could be very useful for people to select what they need. Big Data basically means a large quantity of data. The volume includes the production of the data, conduct data, and how to save data.
    Big Data may be veracity. Since the data is too much, sometimes they may not be very reliable to use.
    Just like this “A revolution that will transform how we live, work and think.” Big data make our life and work more easily. Data should collect in an overall way of date, not just the partial of it.
    In IT Company, I think the big data is quite useful and helpful. The person who works for IT companies, they can get the information just by checking out the big database.
    Another example, big data may have greatest value for chemical company. People worked in chemical company they can check the big data about their sales records to compare the previous sales or their competitor’s sales. In this way, they have a sense of how they are doing and how to improve that.
    Data has the greatest value for research. When people do research work, the data could be the most important factor to use. Through the big data, research people can get some rules of special related results at the end to use for doing research.

    Reply
  4. Jessica Larsen

    Big data is characterized as being high volume, high velocity, and high variety data that could be very useful to the organizations that hold the data if they could only process it all. Big data is about creating value from very large sets of data that traditional computing techniques cannot analyze. As many sectors are capturing more and more data in the hopes of gaining added value from it, the quantity of big data is continually growing. This ends up creating such a large volume of data that traditional systems cannot handle it all, as well as such a high velocity of data flowing into organizations that it exceeds the capacity that their IT systems can handle, and an increasing variety of data types that makes generating value from data much more difficult. Because of these factors most data that companies gather is simply ignored or cannot be processed to find useful information since they cannot handle all of the data that they gather. To gather the information so it does not just pass through the company as data exhaust, technologies such as Hadoop have to be made to find useful information in large data sets. Hadoop distills large data sets into smaller sections before traditional data analysis solutions are applied to the data to gather value from large data sets. Big data is important because it provides a chance to find greater value in high volume, high velocity, and high variety data that traditional methods cannot find value in. The analysis of big data allows organizations the opportunity to find more value in data that can provide insights into their customers, competitors, and their own business.
    Some applications that I see for big data include being able to have more customized marketing strategies for each customer based on the data collected on what everyone purchases, and the government being able to predict things like changes in the economy that cause recessions and inflation. The government would also have more information about each citizen that they could use. Though whether this would be a good or bad thing is debateable.

    Reply
  5. Stacey Ridler

    Big Data is extremely important as individuals and businesses can utilize the vast amount of information and data to make knowledgeable and strategic decisions. Big Data refers to the volume, velocity, and variety of information that is produced and collected from year to year. This influx of information is becoming so large that current systems are not running at full potential in storing, retrieving, and analyzing the data. Many companies were only collecting information and not utilizing the potential of knowledge data carries. Big Data is the next wave of innovation in computer technology and this boat of information is already here. From the video clip an operating system has already been launched (hadoop) and one of the main known connections was LinkedIn. Information and data that is being stored, distributed, and produced is more then just transactions and storing personnel files. There are now videos, blogs, profiles, pictures, and much more. Each and everyday people and businesses add to their evolving trends of technology and interaction. Utilizing the trends of businesses and people might speak to certain advantages and create connections we have never seen before. Big Data is important to those who want to catch the new wave of computer technology and to advance in their decision making power. To understand individuals and businesses on more then one level of communication and to track patterns could produce more definite projections of future markets and spending habits.

    In my opinion Big Data will open up new applications on technology service fronts. Companies producing services like hadoop. Services that will begin to fulfill the potential that the information and data hold. Businesses are beginning to realize that we are creatures of habit and flow from one idea and like to the next. Understanding and being able to organize, collect, and analyze such data could prove quite beneficial for retail companies as to our buying trends and similar interests. From previous experience in working for RBC the company had started introducing an ‘Offers and Opportunities’ tab in one of our transnational processing systems. This was created from spending and savings patterns of RBC clients and were top selling points for employees to achieve. In my opinion retail industries, banks, governments, and health care industries would all benefit greatly from being able to efficiently and effectively analyzing such information. Taking out the human error of ‘cold calling’ and ‘door to door visiting’ or recalling from memory. I believe the retail industry would benefit the most from such information because the greatest amount of information generated is consumer patterns.

    Reply
  6. Jericka Versikaitis

    Big Data is a form of digital storage that sometimes referred to as the next big thing because of its potential to build information databases. The amount of information it is able to process is significantly larger than any normal computing system. The information that Big Data is able to collect can come from almost anywhere or anything and Big Data makes it possible to be able to eventually analyze. The fact that Big Data can collect insane amounts of data and have it for analyzing purposes can be a negative impact on our privacy or what little privacy we do have left in our every day lives. Being able to collect data from the Internet opens doors such as collecting data from every possible Internet site and not just the big names like Facebook and Youtube to see what ads suit that viewer best. Eventually it will start applying to emails and for example track what information most e-mail contain. In making Internet data all track-able but an outside source, such as the government, it is no telling where this could go. This information could be a positive for larger corporations and the government as well. Being able to store that much data would help their business immensely. With patterns it could help predict and prevent future catastrophes. I think there should be a way to regulate the data being collected on what it can actually be used for.

    Reply
  7. Dani Rasmussen

    Big data is a modern technology that acts as storage for vast and intricate databases. Big data is the next big thing in processing because it offers features that the outdated technology cannot keep up with. Businesses are building massive databases trying to capture more data and create vast quantities of digital material as well as new scientific advancement is beginning to reply on enormous quantities of data. Big data is often characterized by three characteristics known as volume, velocity and variety. Volume offers the greatest opportunity because it can hold such a larger amount of capacity then the older technologies can, velocity because it allows for a more seamless flow of data, and variety of data types because it allows for different data such as photos, video and audio, 3D models, simulations and location data available to users. This helps business learn to understand people better, because prior to this the majority of data that is captured is not being processed in the proper ways. Big data is extremely imperative for all industries in the future because databases are continuously becoming larger and more complex and big data offers storage to all of these data sets. It will be able to potentially assist business to predict future events such as farmers could potentially be able to predict the weather better or crop failures and even the government could possibly prepare for future pandemics. It is also estimated that the use of big data could save industries billions of dollars a year on administrative fees.

    Reply
  8. Rémi Cizeron

    Big Data are given so large that it is difficult to treat. They will have to focus the analysis on what they are really looking. These data come from different sectors such as retailers, logistic, financial, social, media etc.
    Big data becomes a real tool for many institutions.

    Big data generate a really impressive added value for those who know how to use. Because they are so important that it is impossible to analyze it with traditional management tools.

    The big data is growing up exponentially. There has indeed in fact more and more transactions or connected people for example. This is explained by an expanding high use of different actors such as professional or civilians.

    Finally, these data characterize us and they evolve at the same time as us. Although, even if these data are complicated to analyze, they can be easily purchased and used. It can be frightening because these data can be used for bad things. It’s a real disadvantage. It must know how to use good effect.
    You still have used these data can be very useful for many professionals and can save a lot of money.

    In my opinion these big data can be very helpful in many different sectors. Even if we know that a very large portion of these data are not used we have to use it. They can also permit a lot of innovation so we have to take a lot of benefits.
    But I think that the utilization of those data must be control and regulate because of the negative effects.

    Reply
  9. Janine Hawkins

    Big data is concerned with storing and analyzing large amounts of data. It is characterized by volume, velocity and variety. With the growing amount of internet use big data technologies are becoming vital for companies to keep track of their data. It is no longer possible for companies to store their data on a computer because of the sheer amount of data that companies need to have access to. Networks have created a space for companies to stash more information but this isn’t a permanent solution. Companies in many industries are continuing to increase the amount of data they store but they have failed to develop organizational systems for this data. For companies to remain successful they need to be able to access information to make business decisions. Big data technologies can help business to organize their data. This will make retrieval of information easier and will ultimately save the company time and resources.
    These big data technologies should be developed so they don’t only provide a storage space for data but they should also have functions to help organize, retrieve, shrink and delete data. Shrinking data is vital in this world of ever growing data collection. For example, emails can be archived so they take up less room but they still contain all the information necessary. Big data technologies should be able to archive things like files, videos and photos until they need to be retrieved. Additionally, technologies should be able to delete data that isn’t necessary. For example, if a company sells an area of its business the program should delete the data pertaining to that area after a year. Or if a patient dies, their medical records could be deleted after a year. All of these features will help companies to manage the vast amounts of data they must keep.

    Reply
  10. Dani Rasmussen

    Big data is a modern technology that acts as storage for vast and intricate databases. Businesses are building massive databases trying to capture more data and create vast quantities of digital material as well as new scientific advancement is beginning to reply on enormous quantities of data. Big data is often characterized by volume, velocity and variety. This helps business learn to understand people better, because prior to this the majority of data that is captured is not being processed in the proper ways. Big data will be a huge positive for industries in the future because it will be able to potentially assist business to predict future events such as farmers and bad weather or even the government for pandemics.

    Reply
  11. mina omidian

    Big data is a term for a group of data so vast and complex that it becomes difficult to process using a regular database management tools. It is the next big thing in computing. It is a type of storage device that is up to date with todays technology. Big data can be classified to help with the three v’s. Volume the amount of data coming in is so large that it is hard to find the proper storage. The velocity at which the data is being transferred it quite difficult to keep up with and the variety of data is never ending. Big data has a way of helping solve all of these problems. With big data people can now start processing large amounts of data without the stresses of running out of space, speed, variety. Big data can help with photographs, simulations audio and video, 3D models and much more. where as your traditional data helped keep track of financial records, small documents and personal files.
    I can see that big data will be useful in many ways. for example for airports to have projected flight plans and keep records of how much fuel, food, etc. is being used, so they know how much to use for future flights. it will help corporations with marking,HR, accounting and any other part of a company. marketing will use big data for helping them find the proper group that they want to appeal to and have all the photographs necessary to help accomplish their goals

    Reply
  12. Shawn

    Big Data is a result of this ever changing technological world. Constant innovation and the need for bigger and better have spawned Big Data. It can be defined as a term for collecting data sets so large and intricate that it is very difficult to process via database or traditional processing. Big Data is extremely important because it allows storing and transferring larger files of vital data. Until it came about data was derived from small sets of data which would take longer hours to analyze, interpret and relay. Now that big data has been created it is possible to relieve one large data set which makes analyzing and relaying the data much easier. This has allowed researchers to spot business trends, combat crime, prevent disease, determine roadway conditions and etc. The value of Big Data is extremely important and very crucial to the development and efficiency of information systems. Hospitals will be able to send and receive larger data sets which will increase their efficiency in deciphering each set which could lead to more lives saved and to me this is important. Knowledge is everywhere and with the internet and many other applications it can be spread pretty quickly. However the kind of knowledge that catches murderers, saves lives, prevents stock crashes and prevents diseases needs to be researched, analyzed and built this is not the kind of data you can send in a quick email. This is Big Data and the long run benefits outweigh any short run struggles immensely.

    Reply
  13. yuxuan.hou

    as the video mentioned that Big data is the term for a collection of data sets so large and complex . it is a new kind of data storage that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, duration, storage,search, sharing, transfer, analysis and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to “spot business trends, determine quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions. there is no doubt that big data is very useful in our society especially in widely business world. big data has a huge opportunity to leads a new business operate mode. for my personal opinion. the most valuable industry with big data is that logistics communication fields which need lots of data analysis and collection.

    Reply
  14. Xiao Yang

    Big data is the next big thing in computing. It produces values for very large data set that can not be analyzed via using traditional computing techniques. There are 3 characteristics of big data which are volume, velocity and variety. The volume of large data presents challenges and opportunities; it helps organizations understand people and allocate resource more effectively. The velocity refers to the speed of information delivering to organizations. Additionally, the types of data are increasingly diverse. The traditional data are documents, finances, stock records, personnel files. But today’s big data include photographs, audio and video, 3D models and simulations. In addition, many organizations ignore large quantity of valuable information. The leading big data technology is Hadoop. And it also provides platform for big data analytic.

    As far as I am concerned, an organization such as a hospital would benefit greatly from big data. Because we can know huge information about patients and medicines. The more data we can get, the more information we can analyze. Big data is a good approach to reduce inaccuracies. After all, using big data is a growing trend. It can not only to increase efficiency but also can save money and time. Big Data will be useful in almost all industries and companies, such as Amazon, Ebay and Linkden.

    Reply
  15. Bailey

    Big Data is a term that means a massive amount of data information being generated through computers all over. It’s the next big thing in computing; it generates value from very large databases that cannot be analyzed with the normal classic computer techniques. It is important because almost everywhere that produces a product or service is using a database in one way or another. Therefor the quantity of data on computers is vast. The three characteristics of big data are velocity, volume, and variety. Because databases are growing so much the volume in big data helps to store it, the velocity helps it flow better, and the variety of data types helps store different data like photos, audio, video, locations etc..
    I feel that every industry will benefit from Big Data. They all use databases to analyze where their customers live, come from and what they buy and etc.. the data sets of each industry is very large and with big data it will be helpful to store all the information they need. Big data helps accurately analyze every aspect of data, it is mind blowing how much money it can also help save for some of the top industries in the world.

    Reply
  16. mogounn ornella cleone

    Big data is the next technological challenge which generates value from a large quantity of digital information, that which cannot be accurately analyzed with the use of the traditional computing techniques.Most traditional computer hardware solutions are incapable of handling the growing amount of digital information being generated that which big data does.This technological improvement offers a vast opportunity to organizations to better understand the people as well as the market place in which it is operating, so as to generate business intelligence.
    Big data offers great enterprises the opportunity to improve on their management skills with regards to their suppliers,customers and business partners.I think big data will have a great influence on geographic applications such as windows which could help them to accurately forecast bad or good weather. I also belief that in time, big data might have a great impact on the government such as helping the government to protect and plan civil unrest, and could as well aide in reducing government administration cost by a great amount
    .In addition to this, health centers could use it in diagnosing disease and planning health care services what the traditional computing solutions are increasingly not capable of handling.Big Data has the potential to improve analytical insight, as well as allowing the creation of new products and services that were previously impossible.
    social media applications such as Google, Amazon and Facebook have already demonstrated how Big Data can permit the delivery of highly personalised search results, advertising, and product recommendations. In time, Big Data may also help farmers to accurately forecast crop failures.

    Reply
  17. Adam Houtekamer

    Big Data is a term that refers to the vast amount of digital information being generated through computers in the world today. Often referred to as the next big thing because of its potential in building information databases. However the problems associated with Big Data come in to play when considering the sheer size of the information being collected, the rate that it is being produced, and the different types there are to collect. Many companies today leave their Big Data uncollected or unused because of their technologies ability to do so. However this is a huge mistake as the benefits of taking advantage of Big Data far outweigh the cost of entry. The importance of Big Data is its ability to build vast databases that can be used in a number of different ways – companies can develop a sense of their customers interests, sports teams can analyze a wealth of information on their opponents and medical centers can access information on their patients with ease. Capturing and harnessing these vast amounts of data can be wildly advantageous across many different industries.
    Big Data has the potential to change the way marketers operate. Many examples of its use can be seen today with advertisements on Youtube, Facebook and Google. Both are free to use services that generate a majority of their revenue through advertisements. They both use the searching trends of individual users to strategically place, which ads show up on which machines. They will chose adds that are more likely to get clicked on by the people looking at them. Perhaps suggesting that those not safe for work “pop ups” on the side of your Facebook aren’t just there for no reason. Big Data has a large impact on these businesses because its use allows them consistent and profitable revenue streams to become the giants they are today. However, without collecting information about what millions of individuals are looking up they would not be able to do so.

    Reply
  18. Simone Olmstead

    Big Data is an amazing new form of data storage and is the next big thing in technology. Nothing like it has been seen before and the opportunities that Big Data provides and it’s uses can not be done with traditional technology. It is important because so many industries and professional companies are capturing more data because of Big Data. Organizations can now scan huge amounts of data that is currently being missed because the organizations systems are old, outdated and primitive and it’s hard to do with traditional computing techniques. This new technology will help organizations, corporations,the health care sector, and service sector to provide high levels of service.
    I thought it was astonishing the opportunities and doors that this technology opens up. For instance the example given about how it could help farmers to predict weather patterns thus helping to prevent any lost crops or the government using it to predict any civil unrest or pandemics; think of the lives this technology could save. Awesome! Also in health care reducing the loan of $300 billion. I think this technology has many uses all the way from the little guy doing audio, video or simulations all the way up to governments.
    With any powerful technology I firmly believe that it could also have a negative side. There are always people who can manipulate powerful technology to benefit their pocket books and take advantage of things. So as people become more technologically savvy with Big Data I think it will reveal it’s positives and negatives.

    Reply
  19. Jodi Berry

    Big Data is a form of digital storage, that processes large amount of information that is too large for regular computing systems. Before this technology data of this kind was not being processed. This offers vast new opportunities that were not open before this technology. Big Data can offers a way to analyze the vast amount of information being presented in the world today. Because of the internet of things, data can be made from almost any device and this presented a challenge, but with Big Data it is becoming a possibility to analyze and understand the data out in the world.
    I believe that Big Data will be useful in the corporate world. Corporations will use big data to research about their consumers, and use it to further personalize their marketing. If a corporation can look back on the who, what, where, when, why, and how of their consumers purchases they can open up new doors to how they will sell you the next item.
    I think two big issues that will arise from Big Data will be the government and education. We have already seen the US government willingness to obtain information on the world, and if they have Big Data on their side the effects might be scary. As well education has become so integrated into the internet today that everything a student does is on the web. Because the majority of students spend their free time after school on the internet, this will create a profile of each student all day. This can also apply to work places that commonly use internet. Some peoples entire lives maybe profiled under Big Data.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s