Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

November 08 2011

When good feedback leaves a bad impression

http://assets.en.oreilly.com/1/eventprovider/1/_@user_4490.jpgIf a teacher is prone to hyperbole — lots of "greats!" and "excellents!" and "A+++" grades — it's natural for a student to perceive a mere "good" as an undesirable response. According to Panagiotis Ipeirotis, associate professor at New York University, the same perception applies to online reviews.

In a recent interview, Ipeirotis touched on the the negative impact of good-enough reviews and a host of other data-related topics. Highlights from the interview (below) included:

  • Sentiment analysis is a commonly used tool for measuring what people are saying about a particular company or brand, but it has issues. "The problem with sentiment analysis," said Ipeirotis, "is that it tends to be rather generic, and it's not customized to the context in which people read." Ipeirotis pointed to Amazon as a good example here, where customer feedback about a merchant that says "good packaging" might initially appear as positive sentiment, but "good" feedback can have a negative effect on sales. "People tend to exaggerate a lot on Amazon. 'Excellent seller.' 'Super-duper service.' 'Lightning-fast delivery.' So when someone says 'good packaging,' it's perceived as, 'that's all you've got?'" [Discussed at the 0:42 mark.]
  • Ipeirotis suggested that people should challenge the initial conclusions they make from data. "Every time that something seems to confirm your intuition too much, I think it's good to ask for feedback." [Discussed at 2:24.]
  • Ipeirotis has done considerable research on Amazon's Mechanical Turk (MTurk) platform. He described MTurk as "an interesting example of a market that started with the wrong design." Amazon thought that its cloud-based labor service would be "yet another of its cloud services." But a market that "involves people who are strategic and responding to incentives," said Ipeirotis, "is very different than a market for CPUs and so on." Because Amazon didn't take this into consideration early on, the service has faced spam and reputation issues. Ipeirotis pointed to the site's use of anonymity as an example: Anonymity was supposed to protect privacy, but it's actually hurt some of the people who are good at what they do because anonymity is often associated with spammers. [Discussed at 2:55.]

The full interview is available in the following video:

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.

Save 20% on registration with the code RADAR20

Some quotes from this interview were edited and condensed for clarity.

Related:

May 04 2011

Trading on sentiment

Numbers on boardComputers don't get emotionally invested in financial trades, but they do take feelings seriously.

Case in point: The financial trading dashboard managed by Thomson Reuters uses sentiment analysis data from Lexalytics to track news on 20,000 stocks and thousands of commodities. The Lexalytics system parses text from multiple sources, looking for keywords, tone, relevance and freshness. The resulting textual analysis (the meaning of the text) and sentiment analysis (the emotions in the text) is then incorporated into widely used algorithmic trading systems.

Mark Thompson, CEO of McKinley Software (the parent company of Lexalytics), told me more about this emotion-to-data conversion. "Our financial engine is something we developed over an 8-year period, and the main partner for that is Thomson Reuters," Thompson said. "The Thomson Reuters news passes through our black box and we kick out scores based on 80 different variables for all of the articles."

Algorithmic trading is automated trading where trading software takes various inputs, or "trading signals," and uses them to decide what trades to make. Trades are executed in a matter of milliseconds and there is no human intervention. In 2009, algorithmic trading accounted for more than 25 percent of all shares traded on the buy side. No human being can read the latest financial news fast enough to contribute to those buy or sell decisions. That's where sentiment analysis comes in.

"By scoring the news, within milliseconds we get a very accurate view of what's being said about a particular stock or sector," Thompson said. "Thomson Reuters sells that output to trading houses who then plug this data into their algorithmic trading models. We have found that we can predict stock market movements. We provide an extra layer of richness that trading staff haven't been able to get their hands on. Otherwise, you are just doing very two-dimensional quant processing."

Rochester Cahan, VP of Global Equity Quantitative Strategy at Deutsche Bank, has been experimenting with the Thomson Reuters system. Cahan told me that he has seen significant improvements in trading performance when the text analysis and sentiment scores are used as trading inputs. In addition, the scores are uncorrelated with existing trading signals — in other words, they provide new information to the trading system.

The most positive sentiment levels (e.g. Apple releases the iPad to universal acclaim) are not necessarily the most useful for trading. The stock price reacts very quickly so it's difficult to take advantage of the information. However, Cahan said stocks with moderate positivity tend to be overlooked by the market and can make for good buys.

I asked Thompson about the limitations of the sentiment analysis technology. He explained that even human beings don't agree on the sentiment of an article more than about 85% of the time. "The problem with our kind of engine is trying to get above 85% accuracy," Thompson said. "Beyond that level, you get a diminishing return and you need more human intervention. This leaves the human analyst to pass different types of judgements."

The competitive edge may be lost if all trading systems use sentiment analysis, but Thompson thinks there is some distance to go before we get to that point. "Everyone has a slightly different way of composing the model and using the news, and there are always advances in the technology," he said. "But there will come a point when sentiment becomes an ordinary part of the trading mix."

Photo: ABOVE by Lyfetime, on Flickr



Related:


Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl