Big Data explained

Description: Big Data is the next big thing in computing. This video explains Big Data characteristics, technologies and opportunities.

Source: http://www.explainingcomputers.com

 

Due to the issues raised by its volume, velocity and variety, Big Data requires new technology solutions. Currently leading the field is an open-source project from Apache called Hadoop. This is developing a software library for reliable, scalable, distributed computing systems capable of handling the Big Data deluge, and provides the first viable platform for Big Data analytics. Hadoop is already used by most Big Data pioneers. For example, LinkedIn currently uses Hadoop to generate over 100 billion personalized recommendations every week.

What Hadoop does is to distribute the storage and processing of large data sets across groups or “clusters” of server computers using a simple programming model. The number of servers in a cluster can also be scaled easily as requirements dictate, from maybe 50 machines to perhaps 2000 or more. Whereas traditional large-scale computing solutions rely on expensive server hardware with a high fault tolerance, Hadoop detects and compensates for hardware failures or other system problems at the application level. This allows a high level of service continuity to be delivered from clusters of individual server computers, each of which may be prone to failure. Processing vast quantities of data across large, lower-cost distributed computing infrastructures therefore becomes a viable proposition.     READ REST OF STORY 

Questions for discussion:

  1.  What is Big Data and why is it important?
  2. What potential applications do you see for Big Data and in what industries will this add the greatest value?
Advertisements

70 thoughts on “Big Data explained

  1. Lamin

    Big data as the name suggest is the accumulation of a vast amount of data through the use of technology, that encompasses almost every spheres of human activity. It would have been unthinkable to gather, manage and process the amount of data that is collected daily. However, there are some big concerns about the three Vs mentioned in the article. The volume of data could be so huge that when a system goes down, an untold proportion damage could be done to the users of the data.
    Another concern is varieties of different data, which pose a big challenge to users. Sometimes, the speed or velocity with which this huge data are processed could also be a challenge especially for places where high speed internet is not available.
    The advent of big data has also brought in a huge resource requirement to facilitate its use.
    As people are doing away with traditional method of data collection, new innovations are almost the order of the day, thereby creating more job opportunities in the Information technology.

    Reply
  2. Ashley Nelson

    Big Data, as referred to in the video as “ the next big thing in computing”. This innovation is not only important for today but it will be even more important in the years to come as technology and the internet grows. As things like social media becomes more prominent in businesses along with personal lives these mass amounts of data will only get larger. More people are uploading, downloading and searching than ever before and lots of things now demand real time results, especially with the new technologies and innovations that are becoming connected to the Internet. Hadoop, a developing software library for reliable computing system is capable of handling the Big Data. This application allows these mass amounts of data being uploaded to become organized information, which cannot be organized now with the traditional technologies eliminating the fact that we are essentially ignoring and deleting large amounts of important data. Hadoop is currently being used for LinkedIn, one of the number one sites for the business world and it seems to be an advantage for the site. Many things would benefit from Big Data. Going back to the real time topic many companies want and need real time results and having that results in tons of data. It would help marketing firms through the social media because they would be able to store more data on you and connect your personality and interest to other relatable sites as well as more common everyday things such as security. Many companies delete security footage after a short period of time because they simply do not have the room to store it. With Big Data they would be able to have more organized data and eliminate this issues.

    Reply
    1. chitra

      The big data is said to be the next big thing in computing techniques and generates values from very large database sets that cannot be analysed with traditional computing techniques. The big data is very useful in today’s world but it is going to benefit many next generations as the usage of computer and internet is getting bigger day by day. it is very useful as retailers are recording customer activities, organizations working in logistics, financial services and healthcare are capturing more and more data. public social media is collecting and creating vast quantity of digital material.many social big data sources are complex in structure and that is the reason why they are not easy to categorized and being processed. many companies most of the data which surrounds the company is not processed at all and it is often ignored. Hadoop is developing software library for reliable computing system is capable of handling the big data. linked in uses big data to gather over 100 billion personalized recommendations every week. hadoop distributes storage ans processing of larger data sets across groups or clusters on server computers. its two main features are distributed file system which commits high bandwidth cluster based storage and data processing framework map reduce. big data is very beneficial to companies dealing with really large amounts of data. another good thing about big data is they are easily available, e.g for the organizations which cannot afford internal database they can use cloud based data sets available on internet.e.g amazon website hosts of online data set which contains government and healthcare information.big companies like IBM and operating systems like oracle provides facts about how big data is going to help companies better understand their customers.

      Reply
  3. Morgan

    As we continue to advance and integrate technology into more and more aspects of our everyday lives, we also end up creating more data than ever anticipated. Big data is so important because of the potential uses, applications of the data, and the value we can create from this data. Big data is generated on the internet, e-commerce, logistics, finance, healthcare, basically any form of technology we use today is creating data. Our biggest problem is whether we can process and use this data or if it will go to waste. The greatest challenges, and opportunities arise from the three V’s (Volume, Velocity, and Variety). Volume is the amount of data we create, it poses a huge issue as massive volumes of data place stain on any system attempting to acquire and retain this data; however, this data can also be processed and used to create marketing plans or predict consumer behavior in e-commerce for example. Velocity is the rate at which data is create, and transmitted to organizations. Many organizations struggle to keep up with this data as they are not equipped to handle the speed that is needed to keep up with big data. The third issue with big data is Variety, Variety poses an issue to organizations because processing and correctly organizing big data is quite complex, the difficulty is amplified with the Volume and velocity that the data is created at. I think that any industry that uses data for any reason will benefit from further advances in the collection, transmission, and organization/processing of big data. All of these factors will facilitate organizations to become for precise, organized, productive and effective in their particular industry.

    Reply
  4. Ayesha S

    Big data is a huge data sets that is used to analyze and reveal patterns of human interactions/ behaviour. This information is gathered throughout individual lives which is very easy to do since technology is such an enormous part of daily activity. We are on the internet every single day. Even though we might not realize it we are being watched when are buying and selling items, browsing pictures, and having conversations with other people online. We are watched by companies around the world so they can gather data to interpret if are well off as their potential customers. We are being monitored every single day with purchases, medical leads, our work information (payroll, hours, position etc.) and much more. In fact, there is so much information that it is quite difficult for it to even be sort out in categories easily. There has been a rise in the number of companies who are helping out by gathering and organizing the data for it to be of use to help business target consumers that are beneficial for them. Because of this step, there will be an enhanced improvement in advertisement and social marketing. Another application I see that will be truly benefit is Health Services. Although, our medical care as Canadians is one of the best in the entire world, it can be enhanced more when information is sorted properly and accessible to help other patients. It can track people who are in a nearby setting who can be of help to another individual.

    Reply
  5. Anish Singh

    Big data, as the term suggests itself, is huge amount of data which was almost impossible to maintain and process until now. With emerging Big Data technologies like Hadoop and with the help of cloud computing technologies we are now able to process large data sets parallelly after distributing them into different nodes/clusters for individual processing, thus achieving efficiency. This processing power may be further enhanced with quantum computing in the future when the data can be processed even faster and with the current trends we are sure that the size of data we need to maintain will keep expanding.
    Hadoop serves as a big advantage over traditional computing as it processes data by distributing and provides a high level of tolerance for hardware failure. Hadoop can be implemented on traditional data centers or cloud storage too which provides an additional cost advantage enabling small companies to afford these services. Facebook is a spectacular corporate user of Hadoop and we are all aware of the massive databases Facebook maintains, thanks to BigData for the billions of personalized recommendations we get everyday!
    As we make a shift from traditional computing and begin implementing faster and efficient data processing technologies like Hadoop, we can imagine to have a more simplified world ahead. The application of BigData is almost in all major industries from retailing to mining and from health care to agriculture, research and almost everywhere. In a recent PwC’s report, it was revealed that 80% of the CEO’s consider data mining and analysis to be most strategically important technologies and that tells the story how simplified the world becomes when we are able to process unstructured data and gain valuable information or gain what we actually want to know to make that decision.
    During the General Elections in India in 2014, a political party achieved a highly successful win using big data analysis and no wonder BigData was responsible for Mr. Obama’s successful re-election in 2012. Many governments and companies are making huge investments in developing these technologies as they can see a potential which is going to change the way companies work, bring down costs, improve speculation and decision making abilities. The benefits of big data would be a win over the disadvantages like data privacy issues while we are also confident on developing better technologies to prevent such data breaches.

    Reply
  6. Xuan Wen

    Big data is a important thing in computing an d generates value from very large data-sets, which cannot be analyzed with traditional computing techniques. Big data capture more data and public social media is creating vast quantities of digital material. Big data is also being generate by an expanding Internet at things, and scientific advancement starting to generate a mole on my own vast quantities of data. Bid data characterized using the 3vs volume, velocity, and variety. The volume as big data could help many organizations to understand people better and allocate resources more effectively. Velocity can create some issues with the rate at which stage was flowing into many organizations. Also, the database today is become more variety such as photographs, audio, video, 3d models, simulations, and location data, which is not easy for traditional computing techniques to categorize them. Today the leading big data technology is “hadoop” which is an open source software for reliable scalable distributed computing and provides the first viable platform the big data analytics. “Hadoop” also distributed to storage in processing have large datasets across groups or clusters. “Hadoop” detection compensates the hardware failures or the system problems at the application-level.
      In real life, big data may help farmers to accurately forecast bad weather. Government also can use big data to predict and plan to civil unrest or pandemics. Overall, int this generation, big data is significant creation of human-being because of it can create customer database. Also, big data can help Health Care, logistics and financial to capture more data.

    Reply
  7. Shiqi Wang

    Big data is a popular term used to describe the exponential growth and availability of data, both structured and unstructured. And big data may be as important to business – and society – as the Internet has become. There are three Vs of big data: volume, velocity and variety. For example, unstructured data streaming in from social media. Increasing amounts of sensor and machine-to-machine data being collected. In the past, excessive data volume was a storage issue; RFID tags, sensors and smart metering are driving the need to deal with torrents of data in near-real time. Reacting quickly enough to deal with data velocity is a challenge for most organizations; unstructured text documents, email, video, audio, stock ticker data and financial transactions. Managing, merging and governing different varieties of data is something many organizations still grapple with.
    There are some examples which customers using big data to benefit their business. Across consumer and B2B industries, it’s an unpleasant fact of life: Every day around the world, criminals are busily at work trying to defraud companies through a constantly evolving portfolio of schemes and strategies. As the volume and sophistication of these schemes increases, many organizations are turning to powerful analytics to sift through massive data volumes and uncover hidden patterns, trends and suspicious events that can indicate criminal fraud. In the past few years, it’s been anything but smooth sailing for financial services firms that have struggled to effectively manage their extensive consumer home loan portfolios. An industry wide failure to properly assess the latent risks lurking in thousands of substandard loans led to billions of dollars of losses, thanks to record-setting write-downs.

    Reply
  8. Mohammad Ashraf

    Big Data generates value from storage and processing of a large amount of information. It is unique because normal computers do not have the ability to perform such tasks yet. In a fast changing society, our ways of working are changing and generating value from information is becoming a trend in the business. This useful information can be external and internal information for any organization that is willing to use information to gain a competitive advantage. There is a very large amount of data that needs to be generated and stored. Since there is so much data coming in, companies need to figure out which data is most useful for them.

    Big Data is using a very effective tool, which gives organization an advantage in the marketplace. Big data uses the “Hadoop” application to effectively process large amount of data and create a summary. The summary that is created can be defined as “Business Intelligence”, which provides the organization with information that can be used to satisfy current customers and gain a wider market. Being a regular user of the Internet, I see organizations make use of Big data. It is very apparent, as I see ads on Facebook banner’s, which meet my interests. I see advertisements of my favorite designers when I am on websites such as Facebook, YouTube, Google, etc.

    I can’t pick a specific industry that will find using Big Data beneficial, because I believe every business will gain something from using this type of technology. The use of creating value with the use of external data is becoming more apparent and widely used. I believe that those organizations that choose not to use this type of technology, will eventually fall behind their competitors.

    Reply
  9. Jaime Zachery

    Big Data is data collected from companies on their customer base and provides information on what services are being utilized, what products are being purchased, as well as information on consumer behavior. The amount of data collected is very large and is not able to be analyzed with traditional computing techniques. Big Data is important for businesses to analyze their business strategies and brainstorm new plans.

    Big Data will add value to any industry as it provides useful information. Big Data can be used to analyze product/service promotions, efficiency of employees, and provide knowledge on their consumer base.

    Reply
  10. Ryan.kozole@uleth.ca

    In society today, Big Data is vital nowadays for businesses to intake extremely large amounts of data to try and gain an upper hand on other businesses competing with them.

    Some of the companies with smaller IT systems in place, especially in retail and marketing, are much more disadvantaged to the bigger retail corporations due to the more capacity in the 3 V’s: Volume, Velocity and Variety. This is major downfall due to the fact that the internet as a whole is expanding at a rapid rate, and is the biggest source for information in world. With bigger companies being able to process at a faster rate, hold more information and and carry a bigger variety of information due to a larger IT system, many of the bigger corporations in the world have an increased knowledge on possible information such as buyer wants and trends quicker than the smaller competitors in this sector. I believe that in this sector especially, many smaller corporations will need to focus on bettering Information intake for the future in order to compete and stay in business with some of the biggest corporations in the world. Otherwise, this will lead to a loss in business due to a lack of knowledge from the smaller corporations.

    Big Data obviously has a big impact even on other businesses that are not in the retail and marketing sector, but because of the demand to keep customers and find many new trends and popular items, the 3 V’s are essential to achieve success in getting to customers needs before other competitors try to do so.

    Reply
  11. Ann

    When we are speaking of Big Data it can be explained as when the data for a company has become too vast and to fast it outweighs how a company can manage it. According to the video it is “the next big thing in computing and generates value from very large data sets that cannot be analyzed with traditional computing techniques.” According to the video Big Data can be characterized into three groups which are called the three V’s. These three groups are Volume, Velocity and Variety.
    Volume is the mass of data that continues to grow at a rapid rate which can help companies to better understand people and allocate resources. According to the video companies are trying to figure out how to manage this volume but are unfortunately not there yet due to its mass.
    Velocity is the speed the speed in which data is received within a company’s system. Users are always demanding more data if the data is too fast for the companies systems. It is useful when the data received is faster than the data that is being processed, this happens in real time.
    Lastly we have variety which is the different types of data types that are available and the diversity of it is growing very fast.
    This is important because it is useful to people such as retailers who will be able to create customer data bases. Health care, logistics and financial will be able to capture more data, as well as social media like Facebook and twitter. Helps the scientific community access large quantities of data. Also without Big Data a lot of important information is getting neglected. Big Data can give companies more knowledge on their customers, partners and business.

    Reply
  12. melissawhitegrass

    Big Data is a way for the huge amount of data that is being collected on the internet to be analyzed and generated value that can not be done with the technology we are using right now. This is important because the amount of information being collected by the internet is huge and is growing rapidly so there needs to be a way to keep up with the data and analyzing the data in real time so it can benefit a business. This also helping technology to improve and expand so that we can keep up with how much data is coming in and the very large amounts and extract the information we want as fast as possible.

    Big Data has the greatest impact on marketing research in my opinion. Industries such as credit card companies, customer loyalty programs, distribution centres, retail/shopping and pharmaceutical would benefit from this the most, the needs of their clients are changing and expanding rapidly so making products to suit this needs to be fast, accurate, and pinpointed to what market they want to get into. Companies that specialize in analyzing Big Data would definitely have an opportunity and if efficient they can have millions of clientele who want that data in real time. This can be an exploding industry with billions of dollars being made if the company can make this work.

    Reply
    1. Maryana

      Big Data consists of 3V’s: volume, velocity and variety. Due to the issues raised by these 3V’s Big Data requires new technology solutions. There is so much information that companies or their computer systems cannot handle and therefore it gets deleted and lost. This is known as “ data exhaust”, a large proportion of data that is gathered but is not processed. Some of this information can be a great source for different issues, and an answer to a lot of questions. By gathering all these information that most of computer systems cannot process, we can find a lot of answers and we can improve our health and government system.
      Hadoop, an open-source project from Apache, is a developing software library for reliable computing system capable of handling the Big Data. This means that all the data that was lost or deleted before, is now well taken care of. This gives us an opportunity to combine all the data, analyze it and process valuable information. Hadoop is being already used, and it has been so far extremely helpful and successful technological innovation. Also, this project may help farmers to accurately forecast bad weather and crop failures. As well as, governments may use Big Data to predict and plan necessary actions in case of civil war or pandemics.
      Therefore, Big Data is extremely important to our society, especially with the rates we are going in this 21st century. Tons and tons of valuable information is getting passed from one resource to another every single day, and with the Big Data technology innovation we are able to analyze it and use it in different fields.

      Reply
    2. Jacob

      Big Data is a term used to describe the large data sets in today’s digital world that cannot be analysed using traditional computing techniques. Big Data consists of the “3 V’s”: Volume, Velocity, and Variety, each comes with its own opportunities and challenges. The challenges the 3v’s can force companies to commit “data exhaust”, which means they begin to ignore very large quantities of potentially valuable information. Thus further Big Data technologies are needed to provide solutions to the problems of Big Data. Currently leading the field in this regard, is an open-sourced project called Hadoop. Hadoop uses what they call “clusters” of server computers (using a simple programming model) to become a viable platform for Big Data analytics. A good example is their client “LinkedIn” which currently uses Hadoops platform to generate 100 billion personalized recommendations for its users every week. That being said, most companies don’t have the funds to afford the platform, but with cloud technology it is becoming more and more available.
      Big Data is important as it has unbelievable ability to change the business and social landscape of how business and governments make decisions. In many sectors, the efforts to exploit the advantages from analysing are already in full swing. Industries such as retailers attempting to predict consumer behaviour, financials and logistics companies using it to find the big picture for demand and economic data, and finally healthcare trying to compile medical data from all over the word to try and gain a deeper insight into the needs in the field.

      Reply
  13. Jaeger

    Big data is extremely large data sets that need to be access computationally to reveal critical information to a company or organizations success. In a world ruled by technology and information the more information you have access and can analyze the better off you will be. Big data can provide organizations with competitive advantages over there competition as well as drastically increase their efficiency. Although big data will provide numerous benefits it doesn’t come without challenges. Due to issues raised by the three V’s (Volume, Velocity, Variety) big data requires new and expensive technology. Leading the way in developing this technology is Apache with Hadoop, software capable of handling these large sets of data. As it would not be feasible for most companies to implement the Hadoop solution, Google and Amazon have begin to offer cloud options which is both cheaper and the data would not need to be stored on the organizations own computers.

    Like the article states “Big data is the next big thing in computing” as its applications are endless with the ability to process data types that the traditional approach was incapable of doing, we open ourselves out to countless new opportunities. The industries that this will add value to include agriculture, health care, government, and private enterprises. As efficiency is an important factor in all organizations big data will add great value to nearly every industry.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s