Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

April 27 2012

Visualization of the Week: The living city

This week's visualization comes from Interactive Things, a design and technology studio based in Zurich, Switzerland. The company has been working on a project called "Ville Vivante" — the living city.

Here's how the coordinators describe the project:

"Based on the premise that the mobile phone has become the center of our everyday communication and main source of information, the City of Geneva, in cooperation with the Lift conference, decided to take the challenge to visualize the digital traces created by our mobile phones ... the goal of the project was to visualize the urban flow of everyday life and make it visible to passersby at the local train station."

Screenshot from the "Ville Vivante" visualization. See additional screens and learn more about the project.

In the Interactive Things' blog post, the company talks about some of the design decisions that went into making visualizations that were aimed at the public and installed in public places.

The data comes from the Swiss telecommunications company Swisscom, and the dataset used for the visualization is based on one week's worth of mobile data usage.

Found a great visualization? Tell us about it

This post is part of an ongoing series exploring visualizations. We're always looking for leads, so please drop a line if there's a visualization you think we should know about.

Fluent Conference: JavaScript & Beyond — Explore the changing worlds of JavaScript & HTML5 at the O'Reilly Fluent Conference (May 29 - 31 in San Francisco, Calif.).

Save 20% on registration with the code RADAR20

More Visualizations:

January 06 2012

Commerce Weekly: Yahoo's new CEO has data focus

As the payments world roused itself from its holiday hiatus, here are some of the items that caught my eye.

Former PayPal chief brings data focus to Yahoo CEO position

YahooScott Thompson's move from leading eBay's PayPal division to becoming CEO of Yahoo received ample coverage in this light news week. The most interesting aspect to me was this former chief technology officer's focus on the importance of data to Yahoo's success. While past CEOs have focused on advertising, the company's role in the media landscape and alliances with U.S. and Chinese companies, Thompson showed his tech-centered origins in an interview with Ad Age:

At PayPal, we were able to create an unbelievably compelling business because we used data to understand risk and fraud better than anyone on earth. And that was the secret sauce. We had more data than anyone else, better tools and models, and super smart people who were challenged by the problem. It doesn't seem glamorous, but that was the reason.

Fast Company emphasized Thompson's background as PayPal's CTO and made clear to its lay-business audience that when he's talking about data, he's not just talking about a better dashboard to understand advertising opportunities. He's talking about the "big data" opportunity, tapping into large datasets produced by the transactions and interactions of Yahoo's 700 million members around the world.

From E.B. Boyd's Fast Company post:

Every day, those 700 million souls log in to the Yahoo universe and start making their way around its sites, moving from story to story to story to story — effectively giving Yahoo a media mogul's dream: the largest petri dish in the world to understand what sorts of content appeal to which sorts of people and what sorts of things will make them likely to consume more and more.

Of course, this is hardly news to Yahoo's data engineers or the big data community, but it will be interesting to see what effect a data-savvy CEO will have on Yahoo's prospects.

X.commerce harnesses the technologies of eBay, PayPal and Magento to create the first end-to-end multi-channel commerce technology platform. Our vision is to enable merchants of every size, service providers and developers to thrive in a marketplace where in-store, online, mobile and social selling are all mission critical to business success. Learn more at

Flurry: More than one billion apps downloaded in 2011's final week

While most retailers focus on the crucial weeks leading up to the holidays, the week between Christmas and New Year's Day — when customers are off work playing with their newly received devices — is more important for app developers. In fact, Flurry reports that this particular week was the largest ever for iOS and Android device activations and app downloads.

Flurry estimates that more than 20 million iOS and Android devices were activated, and 1.2 billion applications were downloaded on the two platforms. Christmas day itself was the biggest day ever for downloads: Flurry estimates that 242 million apps were downloaded while happy recipients explored their new toys.

Flurry also predicted that Apple's App Store will have delivered more than 10 billion apps in 2011 — more than twice the number downloaded in 2008, 2009 and 2010 combined.

EBay's mobile VP goes shopping with Robert Scoble

Just before the holiday, we reported on the "Watch with eBay" feature in eBay's iPad app, which offers viewers a sort of real-time catalog, proffering goods related to the program they're viewing on TV. Robert Scoble has an interesting follow-up interview with Steve Yankovich, eBay's vice president of mobile. Yankovich dropped by Scoble's home office with the app to show him how it works, and he revealed a new feature that identifies fabric patterns in clothing and taps related clothing items in eBay's inventories.

Posters on Scoble's related Google+ thread were more fascinated (or irritated) by Yankovich's comments that even though Android devices are dominating the market, the iOS platform is still more important from a commerce perspective.

Got news?

News tips and suggestions are always welcome, so please send them along.


Sponsored post

August 10 2011

T-Mobile challenges churn with data

For T-Mobile USA, Inc., big data is federated and multi-dimensional. The company has overcome challenges from a disparate IT infrastructure to enable regional marketing campaigns, more advanced churn management, and an integrated single-screen "Quick View" for customer care. Using its data integration architecture, T-Mobile USA can begin to manage "data zones" that are virtualized from the physical storage and network infrastructure.

With 33.63 million customers at the end of the first quarter of 2011 and US$4.63 billion in service revenues that quarter, T-Mobile USA manages a complex data architecture that has been cobbled together through the combination of VoiceStream Wireless (created in 1994), Omnipoint Communications (acquired in 2000) and Powertel (merged with VoiceStream Wireless in 2001 by new parent company Deutsche Telekom AG).

The recently announced AT&T agreement to acquire T-Mobile USA kicked off a regulatory review process that is expected to last approximately 12 months. If completed, the acquisition would create the largest wireless carrier in the United States, with nearly 130 million customers. Until then, AT&T and T-Mobile USA remain separate companies and continue to operate independently.

Information management architecture

As T-Mobile USA awaits the next stage of its corporate history, integration architecture manager Sean Hickey and his colleagues manage data flows across a federated, disparate infrastructure. To enable T-Mobile's more than 33 million U.S. customers to "stick together," as the company says in its marketing tagline, a lot of subscriber and network data has to come together among multiple databases and source systems.

T-Mobile Information Management Architecture and Source Systems
T-Mobile Information Management Architecture and Source Systems (click to enlarge).

Previously, many IT systems were very point specific, stove-piped and not scalable. Some systems began as start-up projects that are now still running seven or eight years later, long after they no longer meet a good return on investment (ROI) standard. Staff that knew the original data models and schema no longer work there.

To integrate data across its disparate federated architecture, T-Mobile USA uses Informatica PowerCenter. (Disclosure: Informatica is a client of my company, Zettaforce.). T-Mobile runs PowerCenter version 8.6.1, is a 9.1 beta customer, and plans to upgrade to version 9.1 in the fourth quarter of this year. Data modeling tools include CA ERwin and Embarcadero ER/Studio. To identify data relationships in its complex IT environment, T-Mobile USA uses Informatica PowerCenter Data Profiling and IBM Exeros Discovery
Data Architecture (now part of IBM InfoSphere Discovery).

This data integration layer powers multiple key business drivers, including regional marketing campaigns, churn management and customer care. Longer term projects — such as adoption of self-service BI and automatically provisioned virtual data marts for business analysts — are on hold pending the acquisition.

Virtual data zones

Backed by this data integration layer, the T-Mobile USA architecture team introduced the concept of virtual "data zones". Each data zone comprises data subjects, and is tied to one or more business objectives. These zones virtualize data applications from the physical data storage and network. From a data architecture perspective, the data zone approach helps pinpoint where there are complex systems to maintain, shadow IT, redundant feeds, differences in data definitions or incompatible data. This approach also helps highlight where business rules are embedded all over, leading to duplicate or inconsistent business rules, versus more centralized rule management.

T-Mobile Data Zones
T-Mobile Data Zones (click to enlarge).

T-Mobile USA adopted SAP BusinessObjects Strategic Workforce Planning, the first SAP application to use SAP HANA in-memory computing to provide real-time insights and simulation capabilities. According to Sean Hickey, T-Mobile USA has been very pleased so far with pilot tests of the HANA-enabled in-database analytics.

Legacy systems do present constraints with management of specific data subjects. For example, T-Mobile USA would like to archive off historical subscriber records that are more than seven years old, which is the cut-off date for regulatory-required retention. However, with the bottom-up growth of the the company's data architecture, it is difficult to carve out old data. The call date was not necessarily part of the partition key. Accordingly, with how data is segmented, T-Mobile USA continues to store subscriber records and other information dating back to 1999.

T-Mobile Data Zones
T-Mobile Data Subjects (click to enlarge).

Regional marketing campaigns

Each data zone is associated with one or more strategic business objectives. For marketing, a couple years ago T-Mobile did a fairly aggressive U.S. reorganization to become a more regional-oriented organization. T-Mobile used to do U.S. national marketing campaigns but has moved to a decentralized model that involves geography, demographics and call usage patterns to perform cross-sell and upsell campaigns by region, with assistance from third-party marketing partners for outsourced analytics. T-Mobile now has more than 20 regional districts across the United States, with a local head who is responsible for sales, marketing and operations in that district.

Northern California VP and GM Rich Garwood added about 30 staff in new regional jobs to take over functions previously handled by T-Mobile USA headquarters in Bellevue, Wash., and will for the first time make a concerted effort to market to small business owners in Northern California. "It's exciting for us as employees. We really have local ownership of what the results are", Garwood told the San Francisco Business Times.

T-Mobile Data Zones
T-Mobile Business Objectives Associated with Each Data Zone (click to enlarge).

SAS Marketing Automation gathers 300 attributes, including campaigns, take rates and dispositions. Before, T-Mobile did national campaigns, with a kind of "shoot and see what sticks" approach. Now, T-Mobile's regions can run targeted campaigns specific to customer demographics and customer segmentation. This requires pulling in more than 20 different sources of data. Deep data mining operations cover billions of rows a day.

For analytics reporting, T-Mobile USA uses SAP Business Objects including Crystal Reports. Finance and accounting department staff still tend to download data into Excel spreadsheets. As part of the company's data security enforcement, every employee and contractor is required to use a T-Mobile-supplied computer with hard drive encryption. Power users can access the Teradata system directly with Teradata SQL for data mining.

Churn management

T-Mobile USA has begun using a "tribe" calling circle model — with multi-graphs akin to social network analysis — to predict propensity of churns and mitigate the potential impact of "tribe leaders" who have high influence in large, well-connected groups of fellow subscribers. An influential tribe "leader" who switches to a competitor's service can kick off "contagious churn," where that leader's friends, family or co-workers also switch.

In the past, wireless service providers calculated net present value (NPV) by estimating a subscriber's lifetime spend on services and products. Now, part of the NPV calculation measures the level of influence and size of a subscriber's tribe.

As noted by Ken King, marketing manager for the communications industry at SAS: "In North America, we increasingly work with service providers that are keen to examine not only segmentation, churn and customer lifetime value but new things like social networking impact on their brands, or the relationships between customers so they can recognize group leaders and their influence on others in terms of buying products or switching to competitors."

Churn management at T-Mobile USA begins with an Amdocs subscriber billing system and financial data stored in a Teradata enterprise data warehouse (EDW). "The heart of the company is the billing system", said Sean Hickey.

However, some key data for churn management is not captured in the billing system. Non-billable events can be very important for marketing. Raw call data gathered from cell towers and switches, supplied by Ericsson and other system vendors, can show the number of dropped calls for each subscriber and the percent of a subscriber's total calls that drop. T-Mobile USA loads call data into IBM Netezza systems from a series of flat files.

T-Mobile engineering uses this data for drop call analysis. They can look at drop calls for specific phone numbers. For example, if a T-Mobile customer moves to a new home in a location where cell towers provide only limited coverage, T-Mobile marketing can proactively offer the subscriber a new cell phone that could improve reception, or a free femtocell that connects to the subscriber's home broadband network. Customer demographic data, however, is not stored in the Netezza systems — that's stored by T-Mobile IT in its Teradata enterprise data warehouse (TED).

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science -- from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 30% on registration with the code STN11RAD

The Teradata EDW sends out extracts to T-Mobile USA's SQL servers and Oracle servers. The Teradata, Oracle and Microsoft SQL Server databases are fed by dozens of source systems, including Seibel, the Billing Portal, Epiphany, Sales Operations and Cash Applications. "Shadow IT" data warehouses include revenue assurance, cost benefits analysis (CBA), business operations and sales credits.

The T-Mobile USA information management team has targeted multiple data marts and shadow IT warehouses to incorporate into the Teradata enterprise data warehouse, pending funded projects to add those to the EDW. In this respect, T-Mobile USA is similar to many other Fortune 500 organizations, which balance an EDW vision with the constraints of budgeting, legacy systems and acquisition integration, and therefore manage a hybrid information management architecture combining an EDW and data federation.

Data delivery is really across the board. It takes a day for information to be batch loaded from retail stores and web sales. It used to then take a second data for analysis. The combination of Informatica PowerCenter and SAP Business Objects Explorer enables the T-Mobile USA channel management team to run reports within seconds rather than an hour or a day. "It's a pretty cool platform," said Hickey. Future steps may target speeding up the data acquisition.

T-Mobile USA continues to innovate for churn management. To better identify the multi-faceted reasons behind customer turnover, T-Mobile USA ran a proof of concept (PoC) with EMC Greenplum, with a storage capacity of roughly 1 petabyte, including data from cell towers, call records, clickstreams and social networks. Following the PoC, T-Mobile USA decided to work with an outsourced service provider, which uses Apache Hadoop to store and process multi-dimensional data. Sentiment analysis predicts triggers and indicators of what customer actions are going to be, which helps T-Mobile proactively respond.

Informatica's newly announced PowerCenter version 9.1 includes connectivity for Hadoop Distributed File System (HDFS), to load or extract data, as explained by Informatica solution evangelist Julianna DeLua. Customers can use Informatica data quality and other transformation tools either pre- or post-writing the data into HDFS.

Single-screen Quick View for customer care

Backed by this data integration architecture, T-Mobile USA just rolled out Quick View as part of an upgrade of its customer care system. With Quick View, agents and retail store associates can view multiple key indicators including the customer segmentation value on one screen. Before, call center agents and retail store associates had to look at multiple screens, which is problematic while talking live with a customer.

Quick View pops up with offers specific to that customer, such as a new phone or a new service plan. Subscribers with a high value may be sent automatically to care agents specially trained on handling high-value customers. T-Mobile USA plans to extend Quick View to third-party retailer partners such as Best Buy that sell T-Mobile phones and services in their retail stores.

More integration

In addition to empowering innovations in regional marketing campaigns, churn management and customer care, data integration will take on even more significance if the AT&T acquisition of T-Mobile USA is approved next year. An approved acquisition would kick off a host of new integration initiatives between the two companies.


July 22 2011

Visualization of the Week: Mobile data redraws the map

New technologies may blur boundaries, but in many ways geography still dominates how our relationships are formed. Researchers at AT&T Labs Research, IBM Research, and MIT SENSEable City Laborator have analyzed mobile phone and SMS data to demonstrate how place continues to play an important role in our relationships.

Here's how the "The Connected States of America" project put mobile data to use:

Using millions of anonymized records of cell phone data, researchers were able to map the communities that people form themselves through personal interactions. The cell phone data included both calls and texts and was collected over a single month from residential and business users.

The analysis and accompanying visualization highlight the communities that are regional but that also stretch beyond official borders and across county and state lines. Some of the findings:

  • California, Illinois, and New Jersey are split on a north-south basis.
  • Pennsylvania is divided east-west.
  • Some communities merge several states: Louisiana-Mississippi, Alabama-Georgia, New England.
  • Some communication communities, such as Texas, actually match state borders quite closely.

The Connected States of America
Click to see the original graphic from "The Connected States of America" project. You can also check out more visuals and an interactive map.

Found a great visualization? Tell us about it

This post is part of an ongoing series exploring visualizations. We're always looking for leads, so please drop a line if there's a visualization you think we should know about.

Android Open, being held October 9-11 in San Francisco, is a big-tent meeting ground for app and game developers, carriers, chip manufacturers, content creators, OEMs, researchers, entrepreneurs, VCs, and business leaders.

Save 20% on registration with the code AN11RAD


Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.
No Soup for you

Don't be the product, buy the product!

YES, I want to SOUP ●UP for ...