• Calender

    April 2015
    M T W T F S S
    « Mar   May »
     12345
    6789101112
    13141516171819
    20212223242526
    27282930  
  • Contact

    Send us press releases, events and new product information for publication..

    Email: nchuppala@ outlook.com

WPP’s Gain Theory launches globally; key hub in Bangalore

 

BestMediaInfo Bureau | Delhi | April 14, 2015

Jason Harrison named Worldwide CEO. Global operations to be supported by Worldwide COO & CEO – EMEA, Manjiry Tamhane & Worldwide CSO & CEO – APAC, Sunder Muthuraman

Gain-Theory-logoGain Theory, a WPP company, has been launched as a marketing foresight consultancy that brings together data, analytics, technology solutions and consumer-insight capabilities. It combines WPP’s intellectual capital in media, marketing, data and technology to create a consultancy that will help brands make smarter, faster, predictive business decisions.

The consultancy will be led by Worldwide CEO Jason Harrison. With key hubs in New York, London, and Bangalore, the global operations will be supported by Worldwide Chief Operating Officer and CEO – EMEA, Manjiry Tamhane and Worldwide Chief Strategy Officer and CEO – APAC, Sunder Muthuraman.

A recent independent qualitative research study highlighted that marketers feel swamped by the sheer volume of data and technology solutions in the marketplace. Marketers also highlighted the need for a partner that would help them navigate this landscape, offering the insight and intelligence required to integrate, predict, plan and model marketing decisions effectively.

Gain Theory will address this by providing solutions that tackle a set of pain points faced by marketing and insight professionals today in achieving the desired ROI from their marketing activities. These pain points include:

  • Difficulty discerning actionable information from an expanding set of data and technology
  • Confusion around terminology and jargon
  • Multiple answers to a single business question
  • The need for faster, smarter predictive insights

“At Gain Theory, our goal is to give clients the confidence to make the best marketing decisions now and in the future,” said Jason Harrison. “Gain Theory’s predictive analytics and global team of specialists help simplify the process, distilling complex data and providing a holistic perspective to improve marketing results. I’m honoured to lead such a dynamic team of smart problem solvers.”

The Gain Theory team comprises 200 marketing effectiveness consultants, analysts, data experts and engineers. This team offers insight-backed recommendations so that brands can adjust their marketing programs for maximum business impact.

Sunder Muthuraman said, “At Gain Theory, we will offer marketers through bespoke analytical solutions. Our solutions will bring intellectual capital in marketing analytics, big data, technology, media and customer relationship management to drive successful marketing decisions. Our goal is to create a new, independent and unbiased consultancy that will help marketers on the journey from data to outcomes and make smarter, faster predictive marketing decisions.”

He added that Gain Theory service would include a number of new products that would be launched shortly – marketing ROI management platforms, visual analytics platforms, customer engagement management services, marketing forecasting and more.

http://www.mandmglobal.com/downloads/MMG14_IntMedia_Digital.pdf

What MapReduce can’t do

We discuss here a large class of big data problems where MapReduce can’t be used – not in a straightforward way at least – and we propose a rather simple analytic, statistical solution.

MapReduce is a technique that splits big data sets into many smaller ones, process each small data set separately (but simultaneously) on different servers or computers, then gather and aggregate the results of all the sub-processes to produce the final answer. Such a distributed architecture allows you to process big data sets 1,000 times faster than traditional (non-distributed) designs, if you use 1,000 servers and split the main process into 1,000 sub-processes.

MapReduce works very well in contexts where variables or observations are processed one by one. For instance, you analyze 1 terabyte of text data, and you want to compute the frequencies of all keywords found in your data. You can divide the 1 terabyte into 1,000 data sets, each 1 gigabyte. Now you produce 1,000 keyword frequency tables (one for each subset) and aggregate them to produce a final table.

However, when you need to process variables or data sets jointly, that is 2 by 2 or or 3 by 3, MapReduce offers no benefit over non-distributed architectures. One must come with a more sophisticated solution.

The Problem

Let’s say that your data set consists of n observations and k variables. For instance, the k variables represent k different stock symbols or indices (say k=10,000) and the n observations represent stock price signals (up / down) measured at n different times. You want to find very high correlations (ideally with time lags to be able to make a profit) – e.g. if Google is up today, Facebook is up tomorrow.

You have to compute k * (k-1) /2 correlations to solve this problem, despite the fact that you only have k=10,000 stock symbols. You can not spit your 10,000 stock symbols in 1,000 clusters, each containing 10 stock symbols, then use MapReduce. The vast majority of the correlations that you have to compute will involve a stock symbol in one cluster, and another one in another cluster (because you have far more correlations to compute than you have clusters). These cross-clusters computations makes MapReduce useless in this case. The same issue arises if you replace the word “correlation” by any other function, say f, computed on two variables, rather than one. This is why I claim that we are dealing here with a large class of problems where MapReduce can’t help. I’ll discuss another example (keyword taxonomy) later in this article.

Three Solutions

Here I propose three solutions:

1. Sampling

Instead of computing all cross-correlations, just compute a fraction of them: select m random pairs of variables, say m = 0.001 * k * (k-1) / 2, and compute correlations for these m pairs only. A smart strategy consists of starting with a very small fraction of all possible pairs, and increase the number of pairs until the highest (most significant) correlations barely grow anymore. Or you may use a simulated-annealing approach to decide with variables to keep, which ones to add, to form new pairs, after computing correlations on (say) 1,000 randomly selected seed pairs (of variables).

I’ll soon publish an article that shows how approximate solutions (a local optimum) to a problem, requiring a million time less computer resources than finding the global optimum, yield very good approximations with an error often smaller than the background noise found in any data set. In another paper, I will describe a semi-combinatorial strategy to handle not only 2×2 combinations (as in this correlation issue), but 3×3, 4×4 etc. to find very high quality multivariate vectors (in terms of predictive power) in the context of statistical scoring or fraud detection.

2. Binning

If you can bin your variables in a way that makes sense, and if n is small (say=5), then you can pre-compute all potential correlations and save them in a lookup table. In our example, variables are already binned: we are dealing with signals (up or down) rather than actual, continuous metrics such as price deltas. With n=5, there are at most 512 potential pairs of value. An example of such a pair is {(up, up, down, up, down), (up, up, up,down, down)} where the first 5 values correspond to a particular stock, and the last 5 values to another stock. It is thus easy to pre-compute all 512 correlations. You will still have to browse all k * (l-1) / 2 pairs of stocks to solve you problem, but now it’s much faster: for each pair you get the correlation from the lookup table – no computation required, only accessing a value in a hash table or an array with 512 cells.

Note that with binary variables, the mathematical formula for correlation simplifies significantly, and using the simplified formula on all pairs migh be faster than using lookup tables to access 512 pre-computed correlations. However, the principle works regardless as to whether you compute a correlation, or much more complicated function f.

3. Classical data reduction

Traditional reduction techniques can also be used: forward or backward step-wise techniques where (in turn) you add or remove one variable at a time (or maybe two). The variable added is chosen to maximize the resulting entropy, and conversely for variables being removed. Entropy can be measured in various ways. In a nutshell, if you have two data subsets (from the same large data set),

  • A set A with 100 variables, which is 1.23 GB when compressed, 
  • A set B with 500 variables, including the 100 variables from set A, which is 1.25 GB when compressed

Then you can say that the extra 400 variables (e.g. stocks symbols) in set B don’t bring any extra predictive power and can be ignored. Or in other words, the lift obtained with the set B is so small that it’s probably smaller than the noise inherent to these stock price signals.

Note: An interesting solution consists of using a combination of the three previous strategies. Also, be careful to make sure that the high correlations found are not an artifact caused by the “curse of big data” (see reference article below for details).

Another example where MapReduce is of no use

Building a keyword taxonomy:

Step 1:

You gather tons of keywords over the Internet with a web crawler (crawling Wikipedia or DMOZ directories), and compute the frequencies for each keyword, and for each “keyword pair”. A “keyword pair” is two keywords found on a same web page, or close to each other on a same web page. Also by keyword, I mean stuff like “California insurance”, so a keyword usually contains more than one token, but rarely more than three. With all the frequencies, you can create a table (typically containing many million keywords, even after keyword cleaning), where each entry is a pair of keywords and 3 numbers, e.g.

A=”California insurance”, B=”home insurance”, x=543, y=998, z=11

where

  • x is the number of occurrences of keyword A in all the web pages that you crawled
  • y is the number of occurrences of keyword B in all the web pages that you crawled
  • z is the number of occurences where A and B form a pair (e.g. they are found on a same page)

This “keyword pair” table can indeed be very easily and efficiently built using MapReduce. Note that the vast majority of keywords A and B do not form a “keyword pair”, in other words, z=0. So by ignoring these null entries, your “keyword pair” table is still manageable, and might contain as little as 50 million entries.

Step 2:

To create a taxonomy, you want to put these keywords into similar clusters. One way to do it is to compute a dissimilarity d(A,B) between two keywords A, B. For instances d(A, B) = z / SQRT(x * y), although other choices are possible. The higher d(A, B), the closer keywords A and B are to each other. Now the big problem is to perform clustering – any kind of clustering, e.g. hierarchical – on the “keyword pair” table, using any kind of dissimilarity. This problem, just like the correlation problem, can not be split into sub-problems (followed by a merging step) using MapReduce. Why? Which solution would you propose in this case?

GRAPH DESIGN TIP- Determine If Grid Lines Are Required

Excerpt from Stephen Few paper

Grid lines in graphs are mostly a vestige of the old days when graphs had to be drawn by hand on grid paper. Today, with computer-generated graphs, grid lines are only useful when one of the following conditions exists:

• Values cannot be interpreted with the necessary degree of accuracy

• Subset of points in multiple related scatter plots must be compared

Bear in mind that it is not the purpose of a graph to communicate data with a high degree of quantitative accuracy, which is handled better by a table. Graphs display patterns and relationships. If a bit more accuracy than can be easily discerned is necessary, however, you may include grid lines, but when you do, you should subdue them visually, making them just barely visible enough to do the job.

 

When you are using multiple related scatter plots and wish to make it easy for folks to compare the same subset of values in two or more graphs, a subtle matrix of vertical and horizontal grid lines neatly divides the graphs into sections, making it easy to isolate particular ranges of values, as shown in figure

Grid

Forrester Wave™: Big Data Hadoop Solutions, Q1 ’14

BigData

Core Components of Hadoop Ecosystem

Corecomponents

Google-Twitter hookup rumours pushes up babble blog site’s shares

On the edge of another blabbergasm? Really?

+Comment Google is once again rumoured to be mulling over a potential takeover bid of micro-blabbing site Twitter.

The latest speculative guff pushed shares in Twitter up nearly four per cent by the close of play on Tuesday as Wall Street appeared to respond favourably to the scuttlebutt.

That rise in shares was the best performance the Dick Costolo-run firm, which was founded in 2006, had seen for six months. Twitter’s market capitalisation currently stands at $33.7bn.

Googlitter rumours have swilled around the interwebs for years. Yet such a hookup has remained firmly in the realms of fantasy. Could the Chocolate Factory be weighing up its options given its withdrawal from its own dog’s dinner of a “network thingy”, Google+?

Possibly. Commentators are wrongly wedding the concept of a must-have social network to Google’s colossal ad biz strategy.

Comment

As I’ve said before, Google+ has now largely served its purpose for the Larry Page-run multinational: it allowed the firm to slurp up millions of IDs by clamping the service to established Google properties, such as Gmail.

Twitter, then, wouldn’t be a replacement for Google+. In fact, I struggle to see how the profit-lite micro-blogging site would be folded into Google’s empire. It’s simply the wrong fit.

The latest rumours arguably hint at a different type of disruption among investors. So should @dickc be watching his back? ®