Stratifyd 3.0 Data Fusion Confronts Big Data Variety
Like the other Big Data “V’s”, dealing with data variety is costly for businesses. Data comes in unlimited flavors such as text, numerical, geolocations, temporal - from enterprise to online - and it is time consuming to link the data together for analysis. Nearly 80% of the development efforts on Big Data projects are devoted to data integration.
Data fusion is the process of linking data from disparate sources in order to provide a more complete dataset for analysis. Insights derived from fused data can oftentimes be more meaningful than insights from singular dataset analyses.
The extract, transform, and load (“ETL”) process is exhausting. Data integration tools from IBM, Oracle, SAS, and Microsoft are specifically designed to alleviate this business pain. These tools are robust; however, few solutions on the market combine data fusion of structured and unstructured data. Even fewer solutions have data fusion, analytics, and visualization in one platform.
Stratifyd integrates data using full outer joins based on a primary key in real-time. Benefits of this method include:
- Preservation of metadata
- No schemas required
- Data transformation not required
- Newer data overrides older data
- Conflict ignorance
A practical application of data fusion merges review data from multiple sites and surveys together and establishes relationships with enterprise data. For example, a company like Marriott, Yum Brands, or Anytime Fitness could merge millions of reviews scattered across Yelp, Google, TripAdviser, and ConsumerAffairs and fuse with their own internal data for each franchise location in the world. As a result, parent companies can quickly discover where, when, and how to focus their resources to improve their customer retention and satisfaction.
We look forward to developing a Hidden Markov Model, which will relate scenarios to outcomes across datasets to help users find the most likely driving factors of their KPIs. Stratifyd’s future developments will also allow for mediating and decision based conflict resolutions and/or conflict avoidance.