Moving from “Big Data” to “Rich Data”

The Gartner Hype Cycle is real. This is especially true in the web data market, which I believe has been entrenched in the “trough of disillusionment” for the last 5 years.

Web scraping has been around for decades, and companies providing web scraping services have been around for the last 8–10 years. Unfortunately, during that time, no one has really cracked the nut as far as making web data truly consumable. That’s because web data is still just “big data”. It’s not yet become “rich data”.

Techradar posted a great article today about the differences between rich data and big data. Here are some key insights:

“It’s like the difference between crude and refined oil,” says Dr. Rado Kotorov, Chief Innovation Officer at Information Builders. “Combining data provides new context and new use cases for the data. For example, combining social media data with transactional data can provide insight into purchases and thus lead to product innovation.”

“Rich data can be used to answer different kinds of questions that would previously have been difficult,” says Southard Jones at cloud business intelligence and analytics company Birst. “Linking up multiple sources of information can help see things in new ways or across the whole process, rather than just one team’s responsibility.”

Essentially, the key is combining data from different sources to provide more insight and context. This is why web scraping alone is not good enough to make web data valuable. Web scraping provides raw big data from single sources. A business needs to provide more structure and combine data from multiple sources together to make it more valuable to the business’ goals.

As businesses begin to realize the limitations of big data pulled from raw web scraping, they’ll see that it’s actually rich web data that they’re after.

You can connect with us and learn more about our business, people, product, and property APIs and datasets by selecting one of the options below.