Architecting the Data Management Platform Success Story

Authored by Deepak Kushwaha, Program Manager

The Data Management Platform (DMP) can be visualized as a data warehouse. DMP is a software that houses audience and campaign data from various kinds of information sources. DMPs talk to demand side platforms (DSPs) and help marketers unify audience and performance data across different sources. Get introduced  to DMP by reading our blog, ‘An Introduction to Data Management Platform (DMP)’.

According to a report by Digital Journal, the DMP market is expected to grow at 15% CAGR growth and reach USD $3 billion by the year 2023. DMPs have use cases that span industries that include eCommerce, Auto, Retail, Real Estate, Travel and Hospitality, and Financial products. With extensive use cases the DMP should see a 100% adoption rate, however, despite various off-the-shelf solutions in the market, DMP implementations are riddled with challenges. For success, CIO’s must overcome these challenges:

Challenges:

1. Audience performance measurability across devices and channels, including real-time and offline.
2. Scale and effectiveness with onboarding first-party, second-party and third-party data audiences
3. Ability to syndicate custom audiences at scale across channels, including offline
4. Lack of complete view of consumers in platforms such as Facebook
5. Accuracy in cross-device linkages to consistently and accurately identify consumers
6. Audience Data Loss after Integration with DSP and Ad Serving Platform
7. Data pixels provided by DMP are too large to be implemented in Third-Party partners

Good architecture design is the key to overcoming these challenges, when stitching together with the data from various sources that work together to develop a 360-degree view of the audiences. Here is a six-stage architecture for a custom DMP.

1. Data Sources Identification – Identifying all the data sources is the first step. These sources can be first/second or a third party such as WebApp, Mobile App, CRM, Partner Data, SAP, Retail Transactions, Google AdWords, Social sites such as FB, etc.
2. Data Aggregation – Data Aggregation, is the second and one the critical steps, performed by existing solutions such as Google Tag Manager or Tealium.  Only the script provided by Tag Manager (GTM/ Tealium) is used in a client’s codebase, which extracts the complete required analytics data. The Tag Manager internally configures all the other analytics tags.
3. Data Collection and Processing – This step utilizes external systems (Kochawa, AppBoy, Google AdWords, FB Analytics, AppFlyer) for fetching and processing data. A simple way to gather cross channel/device data. Tags have specific functions and should be reviewed and selected based on requirements. These tags are applied to capture user actions data performed on the data source (website or App).
4. The DMP – The complete set of data from step 3 is cleaned, classified, stitched, processed and analyzed before storing in the DMP. At this stage, the DMP is a data storage unit. Choosing optimum cloud storage such as S3, AWS EMR, RDS, Redshift, Apache Hadoop, Hive is imperative; and based on volume of data, performance or other nonfunctional parameters.
5. Data Analytics – Nearly the last step, data analytics forms the base for adding machine learning algorithms (based on R/Python/SQL/Spark) for building a customer profile, look-a-alike models, segmentation and custom audiences. Offline provisions are made based on the clients’ needs and the volume of data to be made available.
6. Dashboard/Publish Data – In the final stage, the dashboard contains all the reports.  All real-time and offline data is shown through a secure Rest API’s to another 3rd party, DSP’s, Ad service platform. Email notification services are also integrated with DMP to campaign client ads/coupons/offers to the targeted custom audience etc.

From a review of the requirements to careful selection of technology used, the role of an architect is crucial to the successful DMP implementation. DMP’s help brands gain an edge by making their data more portable and easier to integrate with DSPs. It provides a competitive advantage by leveraging millions of data profiles from consumers; performing business analytics; and creating custom segments and audiences.

Infogain helps Data Management executives and CXO’s to achieve their business objectives. We invite you to connect with us at info@infogain.com.