From Valence to Emotions: How Coarse versus Fine-Grained Online Sentiment can predict Real-World Outcomes

Table of Contents

List of Abbreviations

List of Figures

List of Tables

1 Introduction

2 Structure of Book

3 The Need of Automated Prediction Using Online Sentiments

4 What are the Different Prediction and Sentiment Detection Approaches and Techniques based on User-Generated-Content?
4.1 User Generated Content and its Technical Background
4.1.1 Social Media vs. Web
4.1.2 Online Community
4.1.3 Social Networking Service
4.1.4 Weblog
4.1.5 Review Site
4.2 Online Word-of-Mouth
4.2.1 Appearance of Online Word-of-Mouth
4.2.1.1 Scale Rating
4.2.1.2 Tweets
4.2.1.3 Review Texts
4.2.1.4 Blog Posts
4.2.2 Forms of Online Sentiments
4.2.2.1 Volume
4.2.2.2 Valence
4.2.2.3 Emotions
4.3 Sentiment Classification
4.3.1 Machine Learning Techniques
4.3.1.1 Naïve Bayes
4.3.1.2 Maximum Entropy
4.3.1.3 Support Vector Machines
4.3.2 Semantic Orientation Approach
4.3.2.1 Pointwise Mutual Information and Information Retrieval
4.3.2.2 Latent Semantic Analysis

5 How Consistent are Prediction Results Based on Online Sentiments?
5.1 Predictive Power of Online Sentiments
5.1.1 Stock Markets
5.1.1.1 Predictive Sources
5.1.1.2 Methods and Findings
5.1.1.3 Consistency
5.1.1.4 Limitations
5.1.2 Sales Volume
5.1.2.1 Predictive Sources
5.1.2.2 Methods and Findings
5.1.2.3 Consistency
5.1.2.4 Limitations
5.1.3 Box Office Revenues
5.1.3.1 Predictive Sources
5.1.3.2 Methods and Findings
5.1.3.3 Consistency
5.1.3.4 Limitations

6 Do Fine-Grained Sentiments Generate New Insights and Better Prediction Results Than Coarse Sentiments?

7 Conclusion

8 Managerial Implications

Bibliography

List of Abbreviations

illustration not visible in this excerpt

List of Figures

Figure 1: Research Framework

Figure 2: Seven Functional Building Blocks of Social Media

Figure 3: Functional Building Blocks of Online Communities

Figure 4: Functional Building Blocks of Social Networking Services

Figure 5: Functional Building Blocks of Weblogs

Figure 6: Amazon’s 5-Star Scale Example

Figure 7: Amazon`s Review Text Example

Figure 8: Hyperplane in a Binary Classification Problem

Figure 9: Corresponding Compressed Row Vectors

List of Tables

Table 1: Selected Studies on Stock Market Prediction

Table 2: Selected Studies on Sales Volume Prediction

Table 3: Selected Studies on Box Office Revenue Prediction

1 Introduction

Social media have undergone a huge growth in the past ten years. Consumers are no longer using social media only as sources of information, but they actively create and share content of thoughts, opinions, and experiences, which is also known as user-generated content (UGC). Opinions, thoughts and experiences are usually stated numerically in form of ratings (such as star-scales) or as textual content in form of product reviews, in forums, or weblogs. All of this content has the power to continuously affect trends in all areas, from technologies and politics to entertainment and lifestyle (Bothos, Apostolou, and Mentzas, 2010 p. 50). In fact, Liu et al (2007, p. 607) argue that UGC can influence product sales, which is why the understanding of opinions and sentiments expressed online is very important.

Given the unstructured and huge amount of UGC, which is created online day by day, quantifying the information it contains is difficult. Innovative methodologies have been developed by a number of organizations, ranging from finance to information science, to filter and structure UGC in order to better understand consumer behavior or predict certain human-related events. Prediction techniques are based on various automatic text sentiment detection tools such as machine learning techniques or linguistic approaches, and an increasing number of research has been devoted to examining prediction using social media based on online sentiments. For instance, Das and Chen (2007) tried to predict stock returns by gathering stock message board data, and Tirunillai and Tellis (2012) focused on online product reviews to predict the same. Further, frequently explored prediction subjects are sales ranks (e.g., Chevalier and Mayzlin 2006; Moe and Trusov 2011) and box office revenues (e.g., Dellarocas, Zhang, and Awad; Liu 2006). In the field of marketing, most research has been conducted on social media and UGC literature.

This work contributes to the current research in several ways. Based on an extensive literature review, this book seeks to shed light on the unstructured vastness of research dealing with sentiments in UGC and its predictive power on real-world outcomes. The aim of this work is to structure, classify, and compare existing research. Identifying and comparing different approaches of sentiment identification and classification used in past and current research seems to be overdue. Literature that captures the full picture of different approaches of prediction based on online sentiments is not up to date.

As a contribution to further research in the field of prediction using sentiments from social media, this work aims to answer the following three specific research questions. First, what are the different prediction and sentiment detection approaches and techniques based on UGC sentiments? As there are plenty of different approaches using miscellaneous techniques, such as the abovementioned machine learning techniques or linguistic approaches, this question aims to support a structured overview of sentiment detection, which is essential for the prediction of real-world outcomes. Second, how consistent are the prediction results based on online sentiments? This question focuses on a deeper analysis of research results and their differences. Third, do fine-grained sentiments generate new insights and better prediction results than coarse sentiments do? The significance of this question lies in that fact that simple classification of review texts into either positive or negative ones may not provide a comprehensive measurement of online sentiments, as Liu et al. (2007, p. 607) argued.

2 Structure of Book

After introducing the need of automatic prediction using online sentiments in Chapter 3, this work first discusses the theoretical background of UGC in order to answer the first research question. In this context, Chapter 4 includes the introduction of specific designs of UGC platforms categorized according to the seven functional building blocks of Kietzmann et al. (2011), an introduction into Online Word-of-Mouth (OWOM) and its different forms, and, lastly, a discussion of relevant types of online sentiments. Before answering the second research question, different sentiment classification techniques from the field of machine learning techniques as well as linguistic approaches are described to complete the theoretical background. With regard to existent research literature on prediction using online sentiments, Duan, Gu, and Whinston (2008, p. 1008) and Dhar and Chang (2009, p. 301) identified mixed results in past research studies. This book takes account of this finding by answering the second research question in Chapter 5. To do so, past and current research literature is classified into their prediction subjects – stock market prediction, sales volume prediction, and box office revenue prediction. For each identified prediction category, the used predictive sources and prediction methods and findings are discussed. Furthermore, consistencies and limitations within and among the analyzed literature are identified. Finally, Chapter 6 investigates possible improvements by applying fine-grained online sentiments instead of coarse sentiments. For this purpose, empirical literature results using fine-grained online sentiments and theory-based literature on fine-grained sentiment classification is analyzed. Chapter 7 provides a conclusion and Chapter 8 some managerial implications.

Figure 1: Research Framework

illustration not visible in this excerpt

3 The Need of Automated Prediction Using Online Sentiments

The boom of social media and with it the growth in UGC opens up the possibility of access to opinions, feelings, ideas, or user preferences stated online. This vast amount of information is mostly generated voluntarily and is freely accessible. Prediction based on human agents using social media, especially experts, seems to be more precise in some cases (Pang, Lee, and Vaithyanathan 2002, p. 79), but there are clear advantages for predicting the future using IT-based techniques to analyze and process social media data and online sentiments.

One reason is that humans tend to overvalue small probabilities and undervalue high probabilities, as stated in psychology and economics literature (Wolfers and Zitzewitz 2004, p. 117). This disparity may lead to poor and biased personal predictions of future events. Moreover, as Wolfers and Zitzewitz (2004, p. 118) point out, desires and interests influence people’s decisions and therefore they do not judge objectively. Using automated prediction methods avoids such behavioral biases and leads to an objective prediction (Bothos, Apostolou, and Mentzas 2010, p. 56). Another advantage is the cost-efficiency of prediction using IT systems (Godes and Mayzlin 2004, p. 548, 558). For example, reading every single online review and handling the information contained therein manually would be a very costly and time-consuming task with the additional risk of interpretational mistakes because of human-based biases (Chevalier and Mayzlin 2006, p. 348). Automatic prediction can also handle greater volumes of data on the one hand and process them more quickly on the other hand (Mishne and de Rijke 2006, p. 1). Finally, researchers have shown that automatically generated prediction can outperform human-produced predictions (Pang, Lee, and Vaithyanathan 2002, p. 83; Zhang and Varadarajan 2006, p. 51). Researchers have recognized these advantages, which have led to increased research and effort to improve automated prediction using UGC.

4 What are the Different Prediction and Sentiment Detection Approaches and Techniques based on User-Generated-Content?

In order to align the research framework introduced here, this chapter examines the different prediction approaches and techniques and gives an introduction to the technical background of online sentiments, their appearance and forms using a top-down approach.

Due to the fact that online sentiments emerge through UGC, at first fundamental background knowledge on UGC and its technical concept will be presented. Second, OWOM, a special form of UGC, will be explained and characterized. Thirdly, different forms of online sentiments that appear within OWOM will be illustrated. Finally, technical and mathematical sentiment classification and detection tools will be introduced.

4.1 User Generated Content and its Technical Background

Many researchers have developed miscellaneous approaches to automatically predict real-world outcomes using UGC. Relevant to these techniques is a sound footing of available online data that is created by online users. There are three distinct patterns that define UGC according to an OECD (2007) study. First, UGC must be published over the Internet and be available to the public. E-mails and instant messages are not publicly available, which is why they are not classified as UGC. Second, the content has to comprehend some creative effort. This avoids replication of existing content. Third, UGC has to be created by non-professionals with a non-professional intention. (OECD 2007, p. 9) The last condition excludes content with a commercial purpose. In summary, UGC is understood as online published content created by private users with a non-commercial intention.

The following chapter will introduce the basic concepts of Web 2.0 and social media applications, their characteristics as well as the different forms of online data they contain.

4.1.1 Social Media vs. Web 2.0

The development of Web 2.0 has led to a flood of data while providing the technical basis for various platforms to communicate, to express, or to rate subjects online. Web 2.0 defines applications and services that use the World Wide Web to enable UGC without the need to download any software. Compared to the first generation (Web 1.0), the content of Web 2.0 is mainly created by its users (Kaplan and Haenlein 2010, p. 61). This includes writing texts, evaluating and commenting on other articles, posting and sorting pictures and videos, or creating and fostering a social network (Alpar, Blaschke and Keßler 2007, p. 4-7).

To distinguish the characteristics of social media and the seemingly interchangeable related concept of Web 2.0 is a challenging task. Social media and Web 2.0 are very similar in nature, which causes confusion among researchers and managers (Kaplan and Haenlein 2010, p. 60). To draw a line between the blurred boundaries, the coherent definitions of Kietzmann et al. (2011) and Kaplan and Haenlein (2010), that are based on the historic development of each term, are used. As stated above, Web 2.0 comprises mainly the development of the technical aspect. Hence, Web 2.0 can be seen as an ideological and technical platform for social media, while UGC is created and exchanged on social media (Kaplan and Haenlein 2010, p. 61). According to Kietzmann et al. (2011), social media consist of seven functional blocks: identity, conversations, sharing, presence, relationships, reputation, and groups (Kietzmann et al. 2011, p. 243-248). Figure 2 illustrates the seven blocks of social media where each block represents a specific facet of social media.

Figure 2: Seven Functional Building Blocks of Social Media

illustration not visible in this excerpt

According to Kietzmann et al. (2011, p. 243)

Due to the difference in social media applications, not all seven blocks have to be present at the same time in social media activity, as the examples in the following subchapters will demonstrate.

The identity block corresponds to the degree users reveal their identity within a social media application. This may include information about name, age, and gender as well as personal behavioral information like hobbies or interests (Kietzmann et al. 2011, p. 243). Users can share identity-related information intentionally or unintentionally in form of subjective information. This information can be contained in online postings or comments expressing the user’s thoughts and feelings (Kaplan and Haenlein 2011, p. 62).

The degree of communication of users among each other is represented by the conversation block. Each social media application has a different focus on communication and user interaction. Users can express, talk, discuss, and connect online. Due to the fact that communication data may contain individual opinions, thoughts, and feelings, the rich data of online conversation is of high relevance to researchers and firms (Kietzmann et al. 2011, p. 244). Either to use conversation data to gain new insights into customer needs and behavior or to predict real-world outcomes, conversation data is an extensive and valuable source.

The third block, sharing, represents the exchanging, distributing, and receiving of online content in a social media application. Depending on the platform, users can share movies, pictures, music, or texts in order to build relationships with each other (Kietzmann et al. 2011, p. 245).

Presence refers to the extent of information about a user’s accessibility. The information ranges from knowing if another user is online or available in the virtual world to detailed location information where a user actually stays in the real world. The presence block therefore closes the gap between online and real world due to its possible connectivity on the move (Kietzmann et al. 2011, p. 245).

In connection with the conversation block, Kaplan and Haenlein (2010) explain that a higher level of social presence is likely to make online conversations more influential, which shows a direct connection between different blocks of social media.

Building relationships is the fifth block of social media. It refers to the way and intensity in which users connect and relate to other users online. Relating to other users can be built on common interests or associations and will lead to conversation, sharing objects of sociality, or following each other online. Depending on the social media platform and the specific value of identity, online relationships can be more or less intensive (Kietzmann et al. 2011, p. 246).

Another block is reputation, which refers to the standing of each user in a social media setting. Considering the standing of other users as well as one’s own standing, different technical methods exist to create reputation. For instance, click rates, star rankings, or likes and dislikes can give information about a user’s trustworthiness and reputation (Kietzmann et al. 2011, p. 247).

The last functional building block of social media that Kietzmann et al. (2011) define is groups. Groups represent the extent to which users can form subgroups or communities within a social media setting. Groups can exist as closed groups with restricted access or as publicly accessible groups. Furthermore, individuals can group their online contacts to control the online content that is shared within a social media setting (p. 247-248).

The seven blocks of social media explain and define in a very concrete manner the diversity of a social media setting. Subsequently, specific social media applications or platforms will be introduced and characterized.

4.1.2 Online Community

Current research literature has established plenty of definitions to specify online communities, due to their varying social and technical structures, with more or less fuzzy boundaries. To be able to differentiate clearly between the following social media applications, a precise definition of online communities will be used. According to Preece (2000, p. 10), online communities consist of a socially interacting group of people coming together for a shared purpose online. Their members often share common interests, values, and characteristics and they keep to agreed rules concerning the community membership. Hence, they are governed by the communities’ individual norms and policies. With a deeper focus on the communication process, Bagozzi and Dholakia (2002, p. 3) define online communities as “mediated social spaces […] that allow groups to form and be sustained primarily through ongoing communication processes.” Thus, members of online communities have the opportunity to constantly access and comment on the opinions of socially relevant peers (Miller, Fabian, and Lin 2009, p. 305). Moreover, online communities underlie a constantly dynamic process and evolve and change over time (De Souza and Preece 2004, p. 580).

Online communities focus on interest-related topics such as technical, social, or economic interests, ranging from expert knowledge forums to shared interests Web sites (Mühlenbeck and Skibicki 2007, p. 15-18). Furthermore, the wide range of online forums includes file-sharing communities and consumer communities. Within file-sharing communities, users are able to upload and download media data such as movies, music, and pictures (e.g., Flickr and YouTube). Other users are able to comment on and rate the uploaded content, which reflects the community character (Walsh, Kilian, and Hass 2011, p. 11). Consumer communities serve as exchange platforms for consumer insights on particular products or services (e.g., Ciao and E-pinion). Community members can express their experiences and recommend or advise against products or services (Walsh, Kilian, and Hass 2011, p. 11).

To bring the understanding of online communities, within this work, in line with the seven functional building blocks of social media, sharing, conversations, groups, and reputation are the individual blocks that characterize online communities most.

Figure 3: Functional Building Blocks of Online Communities

illustration not visible in this excerpt

According to Kietzmann et al. (2011, p. 248)

4.1.3 Social Networking Service

In general, a social network comprises a group of persons or organizations and the relationships between each of them. They exist in every area such as economics, politics, science, and general public (Bommes and Tacke 2006, p. 34). The members of such a social structure can be from different social entities and range from individual persons, political groups to families or organizations. The connections within these social networks are built on specific relationships, interests, or interactions and are characterized by, for example, information exchange or emotional closeness (Hollstein 2006, p. 14).

With the advent of social media, social networking services (SNSs) applications started to enable users connecting to friends or colleagues by providing features of convenience. Through creating a shareable profile containing personal information such as birthday, hobbies, preferences or photos, videos and audio files, users can express themselves. Furthermore, they can invite friends to get access to their profiles and in some SNSs they are able to send mails and instant messages or post messages on other profiles inside the bounded system of the SNS (Kaplan and Haenlein 2010, p. 63). In comparison with online communities, SNSs allow much more self-expression in form of a personal profile and commonly other users are able to see friends to whom a user is connected. SNSs therefore allow conveying real-life networks online and building new connections based on interests and activities (Ahn et al. 2007, p. 835). SNSs differ with regard to the member target group. LinkedIn or Xing are SNSs focusing on business contacts only. Hence, personal profiles and information in these domains are mainly focused on education, work experiences and interests. Facebook and Google+, in contrast, are leisure networks used for connecting with friends or acquaintances. Accordingly, most personal information on these SNSs is private interest oriented (Cyganski and Hass 2011, p. 83). Besides business- and leisure-oriented SNSs, other SNSs differ according to their user base, interest focus or features such as dating- or classmate networks (Boyd and Ellison 2008, p. 214). The access to a SNS can be open or restricted. In case of restricted access, users need an invitation from another existing user of this SNS. In case of LinkedIn, for example, users need an invitation because LinkedIn is an “invitation only” network.

According to the seven functional building blocks of social media, SNSs comprise mainly of the following elements: relationships, conversations, identity, and reputation.

Figure 4: Functional Building Blocks of Social Networking Services

illustration not visible in this excerpt

According to Kietzmann et al. (2011, p. 248)

4.1.4 Weblog

Open Diary was the first website in the mid-1990s uniting online diary writers; with its foundation, the term “weblog” was introduced, which is now commonly referred to as the shorter expression “blog” (Kaplan and Haenlein 2010, p. 60). The diary characteristic is still typical of today’s blogs. Blogs are online journals which list texts, pictures, videos, or all of them in reverse chronological order (OECD 2007, p. 36; Liu et al. 2007, p. 607). Users (bloggers) can either choose between running a blog on an own server, which requires installation of software (e.g., wordpress), or they can use a blog hosting service, such as myspace.com or livejournal.com, which avoids software issues because the blog application is provided online (OECD 2007, p. 36). The freely accessible software or online application for blogs makes it an easy-to-use tool for publishing texts, pictures, or videos online (Walsh, Kilian, and Hass 2011, p. 10). Due to its diary character, the content of blogs is updated frequently and usually strongly related to the blogger’s personal life or interests (Balog, Mishne, and de Rijke 2006, p. 207). Furthermore, the structure and design of blogs is simple, and readers are able to comment on blog entries (posts), which distinguishes blogs from regular Web pages (Walsh, Kilian, and Hass 2011, p. 11). The topics range from conventional subjects (e.g., holidays, movies, sports, products, food, etc.) to special and detailed issues (Liu et al. 2007, p. 607). Hence, the authors of blog articles often express their moods, thoughts, and feelings in these posts, which leads to a very subjective spread of words. Within the blogosphere, which describes the interconnection of all blogs, bloggers can follow other blogs and link their articles, which creates a co-working atmosphere among bloggers (Walsh, Kilian, and Hass 2011, p. 11).

Today, blogs are not only private diaries anymore, where individual bloggers talk about their lives and experiences. Among the authors are ordinary people as well as professionals and celebrities (Kietzmann et al. 2011, p. 242). Industries have realized the power of blogs, because of its fast spread of words among Internet users. Mainstream media adopts blog content, and some blogs are also able to influence industries because of their strong and huge readership (Walsh, Kilian, and Hass 2011, p. 11).

Along the definition of the seven building blocks of social media, and the definition of weblogs, sharing, conversations, relationships, and reputation are the four characterizing blocks of weblogs.

Figure 5: Functional Building Blocks of Weblogs

illustration not visible in this excerpt

According to Kietzmann et al. (2011, p. 248)

4.1.5 Review Site

Even though review sites cannot be seen as a social media application by itself, they have to be considered in this context, because review sites are a valuable data source to gain online sentiments for predicting real-world outcomes. Review ratings appear in a variety of forms – they either are embedded into commercial Web pages (e.g., Amazon.com, ebay.com) or appear as exclusive review Web sites specialized in professional or user reviews (e.g., Epinions.com, Cnet.com) (Dave, Lawrence, and Penncock 2003, p. 519; Dellarocas 2003, p. 1408). The area of product or service reviews is complex and includes, for example, car-, electronic-, book-, or movie reviews. On commercial Web sites, the review is directly linked to the product that can be purchased online, whereby on solely review Web sites reviews are sorted by product type and products cannot be purchased directly. Online reviews take the form of either a numerical or graphical rating scale (e.g., 5-star rating), a free text, or a combination of both (Luo and Zhang 2011, p. 13; Dave, Lawrence, and Penncock 2003, p. 521). Through online user ratings, users can recommend or advise against products by posting their experiences and opinions, thereby supporting and influencing other users’ purchase decisions (OECD 2007, p. 35-40).

Because review Web sites cannot be seen as a social media application, a definition according to the seven functional building blocks of social media is not reasonable.

4.2 Online Word-of-Mouth

By analyzing the purchase behavior of different household goods, Katz and Lazarsfeld (1955) were the first researchers who found evidence that word-of-mouth (WOM) is the most influential source for consumers to switch a brand, compared to several other means of advertising such as newspapers or radio. WOM is a communication process conveying details about and experiences with a product or service among consumers. Because of the sender’s independence of the market, WOM is considered as more trustworthy, compared to commercial advertisements or sales persons’ consulting service (Brown, Broderick, and Lee 2007, p. 4; Jansen et al. 2009, p. 2169; Liu 2006, p. 74). Using social media applications, WOM is no longer spread only among friends, colleagues, and family members “offline” in a face-to-face manner. Consumers now exchange experiences, recommendations, and knowledge with strangers online by generating OWOM. Review sites, blogs, or social networks, for example, are used to gain and create product or service experiences. Hence, Dhar, and Chang (2009, p. 303) value OWOM generated by consumers as “the truest form of word of mouth .” Furthermore, OWOM allows exchanging information anonymously or confidentially, which is why it is difficult to control. Thus, OWOM is very important for corporations and organizations with regard to brand management (Jansen et al. 2009, p. 2169).

In the following, different forms of OWOM and its technical appearance will be introduced and explained.

4.2.1 Appearance of Online Word-of-Mouth

OWOM appears in different forms based on different social media applications on the Internet. Users are able to share their product or service experience in a graphical way or can give advice to other consumers in form of a free text. But not only product experiences are shared online. Trading advice for stock traders can be found online within social media applications as well as cooking or car maintaining instructions.

In the following, the most common, and in the upcoming analysis of empirical studies used, OWOM sources are described to give a basic understanding of today’s OWOM appearance.

4.2.1.1 Scale Rating

Scale rating gives online users the opportunity to rate a specific product or service in an easy, fast, and very short way. In form of a graphical star-scale (e.g., Amazon.com, ebay.com) or numeric scale (e.g., Pitchforkmedia.com), users can value their product or service experiences. Commonly, the scale ranges from 1 to 5 stars, but also 1 to 10 stars can be found online, whereby the less stars are given, the worst is the product or service experience of the consumer. Scale ratings indicate the valence of a review, that is, whether it is positive, negative, or neutral, and they can be interpreted by users easily and without much effort (Dhar and Chang 2009, p. 303; Chevalier and Mayzlin 2006, p. 346; Moe and Trusov 2011, p. 445-446). Figure 6 shows a typical 5-star scale of Amazon.com.

Figure 6: Amazon’s 5-Star Scale Example

illustration not visible in this excerpt

Screenshot of 5-star scale on www.amazon.com^[1] (Assessed: August 1st 2012)

4.2.1.2 Tweets

In contrast to scale ratings, where only a positive, negative, or neutral opinion can be expressed, Twitter, the most popular online microblogging application, allows users to express their thoughts, feelings, and opinions with short comments (tweets). Tweets have a limited length of 140 characters, including hyperlinks, and are sent to connected friends (followers) and the public via instant messages, Web, cell phones, or e-mail through a microblogging service like Twitter (Jansen et al. 2009, p. 2170). As a special type of weblogs, microblogs also allow users to create profiles and share interests, thoughts, and feelings in form of tweets. Furthermore, users can connect with other users, celebrities, or companies by following their messages or news postings and communicating with them (retweet) (Zhang, Fuehres, and Gloor 2011, p. 55). The significance of tweets for business, marketing, and research purposes is suggested by the number of tweets that are posted each day. By June 2011, 200 million tweets were posted each day on Twitter containing personal thoughts, feelings, opinions, and emotions as well as professionally created content with a commercial purpose (Twitter 2011). The brevity of the messages, in particular, leads to a higher posting frequency compared with blog posts. Furthermore, the high availability of posts and the flexibility of posting from almost anywhere makes Twitter unique among all OWOM sources (Jansen et al. 2009, p. 2170). However, as Go, Bhayani, and Huang (2009, p. 2) found, especially this flexibility leads to more misspellings and usage of slang compared with other OWOM sources. Furthermore, they identified an average tweet length of 14 words or 78 characters.

The election of the German Federal President is a good example of the attention that is paid to tweets and their high flexibility and availability. During the election in 2010, a member of the German Parliament posted the final result on Twitter before the German Parliament announced it to the public officially. Within seconds, media stations spread the news they received via Twitter without making use of their usual news sources.

Also researchers from different disciplines are paying more and more attention to Twitter and its tweets as a valuable source of data (Jansen et al. 2009; Bollen, Mao, and Zeng 2010; Zhang, Fuehres, and Gloor 2011).

4.2.1.3 Review Texts

Review texts appear in a growing number within different forms of social media applications as well as Web sites (Dellarocas 2003, p. 1408; Tang, Tan, and Cheng 2009, p. 10761). Web pages like epinions.com or consumerreports.org focus solely on consumer reviews ranging from electronics, cars, and appliances to travel and music reviews. Amazon.com, the famous online retailer, combines its products with reviews written by consumers. Online review texts reflect experiences, thoughts, and advice on products or services written by the customers. According to Moe and Trusov (2011, p. 444), online reviews may help exchange experiences and facilitate purchase decisions for undecided consumers and they can be seen as a “sales assistant”. As mentioned above, review texts can appear as combination with a scale rating or solely as text. Two review formats can be found online. First, a structured format where users have to describe pros and cons of a product separately, followed by a summary, which is requested on cnet.com and epinions.com. Second, consumers can formulate review texts without any restrictions regarding text length or guidelines for content, which is used by Amazon.com (Liu, Hu, and Cheng 2005, p. 343). Thus, they can express their opinions, feelings, thoughts, and experience in depth. Furthermore, in some cases, pictures and videos can be uploaded to supplement the free text (e.g., Amazon.com). In some cases, review texts can be evaluated in form of comments (e.g., epinions.com) or ratings (e.g., Amazon.com) by the readership. Due to a vast number of online reviews, the evaluation of reviews allows a fast and uncomplicated search for useful and qualitative reviews. Review texts are a very strong source for OWOM with a big influence on other users. Figure 7 shows a typical review text taken from Amazon.com.

Figure 7: Amazon`s Review Text Example

illustration not visible in this excerpt

Screenshot of a review text on www.amazon.com^[2] (Assessed: August 1st 2012)

4.2.1.4 Blog Posts

Mishne and Glance (2005, p. 155) describe blog posts as the “voice of the public” because of their comprehensive subjects and discussions that can include “a wide range of opinions and commentary about products.” Blogs therefore represent the public opinion of millions of customers, as the following example shows (Liu et al. 2007, p. 607). In August 2005, the famous US blogger Jeff Jarvis posted on his blog buzzmachine.com his disappointment with the product quality and customer service of US-based computer hardware manufacturer Dell. His post was shared and spread through the Web like a bushfire and received over 700 comments.^[3] This negative OWOM was especially bad advertisement for Dell and shows the power of OWOM (Mishne and Glance 2005, p. 155).

In a more general way, blog posts can include OWOM, but they are not especially focused on sharing experiences and giving advice like review texts or review sites. Blogs much more focus on special interests and comprehend general issues regarding these interests. Still, due to the expression of the bloggers’ opinions, blog posts are a valuable source for OWOM (Liu et al. 2007, p. 607).

4.2.2 Forms of Online Sentiments

Classifying the huge amount of data, which is generated by online users every day, is a challenging and nearly impossible task. Classification or clustering methods based on subjects or keywords neglect the expressed sentiments within OWOM (Feng et al. 2011, p. 281). The above-introduced sources of OWOM unify the existence of sentiments reflected by each post, tweet, or article due to stated thoughts and feelings (Mishne and Glance 2005, p. 155). Sentiments can express the “overall opinion towards the subject matter – for example whether a product review is positive or negative.” (Pang, Lee, and Vaithyanathan 2002, p. 79) In addition to sentiment classification in terms of a positive, negative, or neutral attitude, researches even try to identify and classify attitudes in a finer-grained manner using emotions such as worried, happy, or anxious. In which form and to what extent sentiments appear online will be illustrated in the following. At this juncture, we focus on the forms that are used most within sentiment classification literature.

4.2.2.1 Volume

Although volume is not a sentiment, it is used in the context of sentiment-mining regularly. In this context, volume relates to the appearance frequency of a specific rating or post and shows the degree of attention a product or service receives (Tirunillai and Tellis 2012, p. 202). In case of ratings or reviews for example, volume counts the number of ratings or postings which contain the same sentiment or opinion (Moe and Trusov 2011, p. 445; Duan, Gu, and Whinston, p. 1008), or, as Liu (2006, p. 75) states: “Volume measures the total amount of WOM interactions.” Even though other measures exist, such as dispersion, intensity, or duration, only volume will be introduced here because it is a more important measure compared with the other ones (Liu 2006, p. 76). In line with the title of this book, volume can be seen as a coarse measurement.

4.2.2.2 Valence

Besides volume, valence is one of the most important measures of OWOM denoting whether an OWOM message is positive, negative, or neutral (Liu 2006, p. 75). Since valence can be expressed through opinions, feelings, and thoughts, different methods and techniques exist to identify valence within OWOM. Those techniques will be introduced in the following subchapter. The easiest and fastest way to extract the valence from OWOM is numerical or graphical ratings, such as 5-star ratings (Chevalier and Mayzlin 2006, p. 345; Forman, Ghose, and Wiesenfeld 2008, p. 293). Because valence measures are either positive, negative, or neutral, valence cannot measure the intensity. Therefore, within this book, valence is understood as a coarse measurement of online sentiment.

4.2.2.3 Emotions

Blog posts, tweets, and other forms of UGC and OWOM include much more than only a positive, negative, or neutral sentiment. Since users express feelings, thoughts, and opinions, complex emotions, like joy, surprise, or anxiety, and moods, such as happy, sad, or angry, can be identified in their written content (Feng et al. 2011, p. 284). Furthermore, blog hosting services like livejournal.com allow users to state their mood in addition to their free text, using non-verbal emotional expressions such as predefined smilies or by entering a free text stating their mood (Mishne 2005, p. 2). The challenging task is to detect hidden emotions within written documents, because non-verbal cues are missing such as sounds, gestures, and facial expressions (Feng et al. 2011, p. 284; Hancock, Landrigan, and Silver 2007, p. 929). Depending on the source of OWOM, more or less emotional content can be expressed. Tweets may not have such an emotional volume, due to their limited length, compared with blog posts or review texts. This may lead to interpretation and classification problems, because emotions are very complex and words expressing emotions can have multiple emotional meanings (Feng et al., p. 284; Mishne 2005, p. 1). Emotions are fine-grained online sentiments, which are hard to detect and classify within texts, because they depend on the textual context. In the subsequent chapter, technical methods to detect and classify valence and emotions will be defined.

4.3 Sentiment Classification

Until now, the development and sources of online sentiments have been introduced. In order to predict real-world outcomes, sentiment-containing data has to be automatically identified and classified using information systems. The easiest way to determine the attitude of a text is the bag-of-word approach. With this model, the frequency with which a word appears within a text is measured, whereas stop words, such as “the,” “a,” or “to” are removed. Although it is a quite simplistic model, relatively good results can be achieved (Pang, Lee, and Vaithyanathan 2002, p. 79; Schumaker and Chen 2006, p. 5-12). The bag-of-words model is used very often in combination with more sophisticated approaches. Distinguishing between subjective and objective information is a more challenging task done by analyzing words and complex sentences (Tang, Tan, and Chen 2009, p. 10761). In the following example, the sentiment of the review is easy to understand for humans, but for computational classification the negative meaning of apparently positive sentiments is hard to detect:

“This Film should be brilliant. It sounds like a great plot, the actors are first grade, and the supporting cast is good as well, and Stallone is attempting to deliver a good performance. However, it can`t hold up.” (Pang, Lee, and Vaithyanathan 2002, p. 81)

Sentiment classification classifies attitudes and subjective opinions which appear within texts, posts, or tweets into either binary or multi-class (Liu, Hu, and Cheng 2005, p. 342; Tang, Tan, and Cheng 2009, p. 10761). Binary classification divides sentiments into two groups, such as positive or negative, whereas multi-class classification allows fine-grained classification into manifold groups, like emotions (e.g., happy, sad, afraid, satisfied) (Tang, Tan, and Cheng 2009, p. 10761).

Classification models can use either labeled data or lexicons. Lexical classification is based on background knowledge. For this purpose, lexicons that contain defined words regarding their valences or emotional meanings are used (Melville, Gryc, and Lawrence 2009, p. 2). For instance, WordNet is a famous lexical database (Miller 1995). The easiest way to deploy lexicon data is a simple count of occurrence of defined words within a document. Classifying the document as either positive or negative happens according to the most frequent positive or negative words as defined in the lexicon. Furthermore, the lexical database can be used in combination with more sophisticated approaches. (Melville, Gryc, and Lawrence 2009, p. 2)

In the following, technical approaches will be introduced to identify and classify online sentiments.

4.3.1 Machine Learning Techniques

Machine learning is a process of artificially generated knowledge through training experience. Software is being trained with the help of a sample set of data to understand and recognize behavioral patterns. Based on this trained knowledge, intelligent decision can be automatically made on new, unknown data with the aim to reduce classification errors (Dua and Du 2011, p. 6). Two different kinds of learning methods exist: supervised and unsupervised learning. Supervised learning is determined by a given classification. The output classification is already known and the learning model is trained to recognize the already defined output and depict it at minimum cost. Unsupervised learning techniques do not dictate the desired output. In this specific case, natural clusters within the sample data are recognized according to a given cost function (Dua and Du 2011, p. 7). The accuracy of machine learning strongly depends on the type of training experience as well as the performance evaluation metrics and the strength of the problem definition. Hence, machine learning can only be evaluated empirically by comparing the accuracy of the classification with the accuracy of the training data classification (Dua and Du 2011, p. 7).

Subsequently, three different standard algorithms for learning techniques which have been used in recent research on sentiment classification will be described.

4.3.1.1 Naïve Bayes

As the name already suggests, the Naïve Bayes (NB) classifier is a statistical classifier using probabilities based on Bayes’ rule. NB is a supervised machine-learning algorithm using training data to learn categorization according to it. Words, bigrams, and trigrams can be identified, as well as part of speech structures. Word identification means categorization of single words within a sentence; bigram is the categorization of every consecutive two-word pair; and n-gram is the same with just n-words. In this context Part-Of-Speech can identify and distinguish different word senses depending on the context of the sentence (Pang, Lee, and Vaithyanathan 2002, p. 84). In the following, the NB algorithm will be introduced according to Das and Chen (2007, p. 1378); Go, Bhayani, and Huang (2009, p. 3); Pang, Lee, and Vaithyanathan (2002, p. 81-83); and Tang, Tan, and Cheng (2009, p. 10762).

describes the Bayes’ rule with as the given text document and as the class. Furthermore, denotes the a priori likelihood of the event c and denotes the a priori likelihood of the event d. Accordingly, denotes the conditional probability of event c in case of event d. This means that in text document

words will be analyzed whether they belong to class or not. The NB classifier is defined as follows:

The NB classifier contains which is a set of attributes appearing within a document. Consequently, is the number of attributes within a document. To avoid zero-probabilities, the smoothing term of is also embodied in the NB classifier. After training the classifier, the main task of the NB algorithm is to calculate the most likely classification by combining every possible hypothesis.

Using the NB algorithm not always leads to a satisfying result due to the specific underlying. Researchers have conducted many studies using the NB algorithm with many different settings and various successes. Pang, Lee, and Vaithyanathan (2002, p. 84) found that NB leads to better results by embedding only the presence of a feature rather than the feature frequency, which is in direct opposition to the findings of Nigram, Lafferty, and McCallum (1999) and therefore illustrates the sensitivity of the NB regarding its underlying. Today, the NB algorithm is usually used for spam filters.

4.3.1.2 Maximum Entropy

According to Go, Bhayani, and Hunag (2009, p. 3), Nigam, Lafferty, and McCallum (1999, p. 61-65), and Pang, Lee, and Vaithyanathan (2002, p. 82), Maximum Entropy (ME) is a classification method based on weighting the features instead of counting them like NB. Different features are weighted regarding their entropy.

Estimating constraints such as in the following ME exponential form ensures a unique distribution with maximum entropy:

with as feature-weight parameter and as normalizing factor to guarantee a suitable probability:

A high value of suggests a strong indicative power of for class . Applying ME to a specific domain, for example sentiment classification, demands selection of a set of features for setting the constraints. denotes a feature/class-function for each feature and class and is defined as follows:

For example, if the specific bigram “very good” appears in the document and the document’s sentiment is hypothesized as positive, the feature/class-function will return 1. Because ME does not make an independence assumption for its features, unlike NB, bigrams and phrases can be added without worrying about feature overlapping. Compared with NB, this fact is an advantage of the ME method. Moreover, Nigam, Lafferty, and McCallum (1999) and Pang, Lee, and Vaithyanathan (2002) have shown that ME models in some cases outperform NB methods.

4.3.1.3 Support Vector Machines

The third machine learning technique introduced here is a large margin classifier, compared with NB and ME, which are probabilistic classifiers (Pang, Lee, and Vaithyanathan 2002, p. 82). Support Vector Machines (SVM) belong to the group of supervised learning methods and can be defined accordingly to Pang, Lee, and Vaithyanathan (2002) and Joachims (1999, 2001):

Based on a binary classification problem, SVM are universal learners that try to find a maximum sized hyperplane between two classes. Maximum sized hyperplane in this case means “maximum Euclidean distance to the closest training examples” (Joachims 2001, p. 128). This hyperplane is represented by the vector and the closest examples to the hyperplane are called support vectors. Figure 8 illustrates a binary, linear classification with the hyperplane (), the support vectors (circles), and the maximum distance (). The two classes are separated into positive (+) and negative (-). In case of a non-linear dataset, the vector will be converted into a multidimensional space to classify the data, using a kernel function. After separation, the vector will be reduced to a two-dimensional vector, as shown in Figure 8 on the right side.

Figure 8: Hyperplane in a Binary Classification Problem

illustration not visible in this excerpt

Linear classification Non linear classification

According to Joachims (2001, p. 129)

In a binary classification problem, two different classes will be distinguished, for example , where 1 stands for “positive” and -1 for “negative.” The classes of document are represented by:

In this solution, and are support vectors because they contribute to vector .

4.3.2 Semantic Orientation Approach

Semantic orientation approaches are unsupervised learning algorithms which focus on the semantic meaning of words, sentences, or documents (Ye, Zhang, and Law 2009, p. 6528). They are able to identify the semantic polarity of words and need no training dataset; hence, they belong to the group of unsupervised learning algorithms (Turney 2001, p. 491; Ye, Zhang, and Law 2009, p. 6528). In the following, two unsupervised learning algorithms for semantic orientation identification will be introduced.

4.3.2.1 Pointwise Mutual Information and Information Retrieval

Pointwise Mutual Information and Information Retrieval (PMI-IR) is an algorithm that calculates the degree of the association between two terms using Pointwise Mutual Information (PMI) (Turney 2003, p. 316). Furthermore, the PMI is able to use Information Retrieval (IR) by issuing queries to online search engines, for example. The number of matching documents (hits) of the query will be noted (hitcounts). Thus, the PMI does not need any kind of training data because of its access to a huge database, the Web. Additionally, the PMI-IR algorithm can handle huge data sources (Turney 2001, p 495; Turney and Littman 2003, p. 316).

According to Mishne (2005), Turney (2002, 2003), and Turney and Littman (2003), PMI between two terms can be defined as follows:

The numerator expression describes the probability of the co-occurrence of and . In case of statistical independence of and , the probability of co-occurrence is given by the expression denoted in the denominator . The ratio of and measures the dimension of statistical dependence between and . With the log term, the amount of information that is represented by the occurrence of one term, when observing the other term, is represented.

The Semantic Orientation (SO) of a phrase using PMI can be calculated simply by subtracting:

If is positive, then the phrase is more related to the SO of . In case of a negative value for , the semantic orientation of the phrase is more associated with (Turney 2002, p. 419, Turney and Littman 2003, p. 315-318).

As mentioned above, the PMI-IR embeds search queries by using hitcounts to estimate the PMI; thus, the equation of the PMI becomes:

In order to estimate the total PMI of a text, the individual PMIs can be summarized (Mishne 2005, p. 4).

4.3.2.2 Latent Semantic Analysis

Deewester et al. (1990) were the first researchers who used the Latent Semantic Analysis (LSA) method to analyze the statistical relationship among words. LSA is a semantic orientation approach to calculate the semantic association between words using a Singular Value Decomposition (Turney 2001, p. 495; Turney and Littman 2003, p. 318). Key issue of LSA is the sorting of words and documents according to their appearance frequency into a reduced dimensionality space, the latent semantic space, based on a Singular Value Decomposition (Hofmann 1999, p. 50; Turney 2001, p. 496). Using a reduced latent space representation eases the detection and classification of similarities between words, phrases, or documents, compared with the original representation (Hofmann 1999, p. 50). Hofmann (1999, p. 50) states that one main advantage of this method is “that documents which share frequently co-occurring terms will have a similar representation in the latent space, even if they have no terms in common.” This enables the identification of topic-related content even if different documents do not use the same words, which also can be seen as some kind of noise reduction (Hofmann 1999, p. 50).

The similarity of words, phrases, or documents is measured graphically by determining the cosine of the angle between the corresponding simplified row vectors (Turney 2001, p. 496; Turney and Littman 2003, p. 318). The smaller the angle, the higher the semantic similarity of words, documents, or phrases. Figure 9 shows the graphical depiction of the compressed row vectors, where dots represent fitting documents and terms and the triangle and square represent non-fitting terms and documents. The dotted line illustrates all fitting terms and documents within a specific angle. One can see that the smaller the cosine of the angle, the closer the relation of terms and documents.

Figure 9: Corresponding Compressed Row Vectors

illustration not visible in this excerpt

According to Deerwester et al. (1990, p. 397)

5 How Consistent are Prediction Results Based on Online Sentiments?

After introducing the technical background for predicting real-world outcomes using online sentiments, in this section existing research will be compared, classified, and evaluated according to research findings. Structured by the prediction subjects, different classification approaches and prediction techniques will be compared to answer the research question of how consistent prediction results based on online sentiments are. Before analyzing current empirical research, potential prediction subjects will be introduced briefly.

5.1 Predictive Power of Online Sentiments

In recent years, researchers have discovered a growing number of prediction subjects by employing social media or UGC as predictive source. Obviously, the most famous subjects, on which most of the research has been carried, are stock market development (e.g., Bollen, Mao, and Zeng 2011; Das and Chen 2007; Tirunillai and Tellis 2012; Wysocki 1999), sales volume (e.g., Chevalier and Mayzlin 2006; Dhar and Chang 2009; Gruhl et al. 2005; Moe and Trusov 2011), and box office revenues (e.g., Dellarocas, Zhang, and Awad 2007; Liu 2006; Liu et al. 2007; Mishne and Glance 2005). Furthermore, some researchers have tried to predict the results of political elections (Tumasjan et al. 2010), the spread of epidemic diseases (Lampos and Cristianini 2010), or people’s visits to specific travel destinations (Choi and Varian 2009). More or less every conceivable area is a worthwhile subject of predictions, as long as it is a human-related event and correlated to the public mood (Bollen, Mao, and Zeng 2011, p. 1). It would not make any sense to try to predict the weather using social media, because it is a non-human-related occurrence. In the following, the prediction subjects of stock markets, sales volume, and box office revenue will be analyzed by using a straightforward research structure.

First, the data sources of the different studies will be examined and introduced, followed by a short summary of the applied methods of sentiment classification and the results of each work. Afterwards, the consistency of the methods and results of the research papers evaluated will be compared and, finally, research limitations will be exposed.

5.1.1 Stock Markets

It is a known fact that people’s decision-making is influenced by their emotions. This also holds true for individuals’ investment decisions; hence, stock market prices are not only driven by new information, such as news, but also by the mood of individuals or the public (Bollen, Mao, and Zeng 2011, p. 1; Gilbert and Karahalios 2010, p. 58). In the following, current research on stock market prediction using online sentiments will be examined according to the process outlined in Chapter 5.1.

5.1.1.1 Predictive Sources

Sources and the appearance of UGC and OWOM have been introduced in general in Chapter 4. Below, the specific predictive sources that were used in empirical research dealing with the prediction of stock market-related content will be introduced. Wysocki (1999) carried out the first study that analyzed the predictive power of online stock message boards. By using a Web crawler program, he extracted more than a million messages of the 8,001 stocks that were listed on the Yahoo! Finance message boards between January 1998 and July 1998 (Wysocki 1999, p. 13-15). Considering the year of his study, Wysocki was one of the pioneers examining the impact of online stock message board activities on stock markets. In general, studies in the early 2000s mostly used online message boards because valuable sources like social media (e.g., social networks, blogs) had not emerged yet or had not experienced a strong user base at this point (Tirunillai and Tellis 2012, p. 201). The Yahoo! Finance message boards were also used as data source by Antweiler and Frank (2004) and Das and Chen (2007). Compared to Wysocki (1999), Antweiler and Frank (2004) also used RagingBull.com, an online message board solely focusing on finance content, as data source. With special, self-written software, the messages were downloaded and stored as simple text files (Antweiler an Frank 2004, p. 1261). Tumarkin and Whitelaw (2001, p. 41), who also used RagingBull.com as data source, describe message or bulletin boards as organized forums to exchange knowledge and experiences regarding a specific topic. They offer information retrieval and users are able to search through previous posts. Furthermore, in 1999 RaginBull.com offered a huge user base and historical data (Tumarkin and Whitelaw 2001, p. 43). Additionally, topic-based online forums as data source reduce the chance that postings are “off topic,” which reduces noise in general. The special configuration of RagingBull.com, for example, sorts messages by the ticker name of the stocks and members have to post their positioning regarding the valued stock with the help of predefined buttons, such as buy, hold, or sell (Tumarkin and Whitelaw 2001, p. 43). The huge number of members and the unique configuration make RagingBull.com a particularly valuable data source for academic research. TheLion.com, another stock message board, was used by Sabherwal, Sarkar, and Zhang (2008) as data source. In comparison to Yahoo! Finance and RagingBull.com, TheLion.com resembles a chat room with messages displayed in reverse chronological order. A ranking system on the main page of TheLion.com displays the 10 most discussed stocks and helps focusing on the stocks about which the most information is available (Sabherwal, Sarkar, and Zhang 2008, p. 424).

In a long-term study conducted for 16 years, from 1984 until 1999, Tetlock (2007) analyzed the predictive power of negative sentiment, published in a Wall Street Journal column. Because of its large readership of more than 2 million people, the column may influence many financial professionals as well as private investors (Tetlock 2007, p. 1140). Different from the abovementioned sources, the data is posted by journalists, but it also contains sentiments. In contrast to Tetlock (2007), Gilbert and Karahalios (2010) analyzed blog posts on 174 trading days, using LiveJournal.com. LiveJournal.com was chosen by Gilbert and Karahalios (2010, p. 59) because of its long history of coupling posts with moods and its established community. Moreover, it is one of the earliest blog platforms on the Internet.

In a very differentiated way, Luo and Zhang (2011) used four different digital user metrics in their study to predict future business financial performance of nine selected companies. Between August 2007 and July 2009, they analyzed the number of Web visits of specific company Web sites, the number of Google search queries, product review ratings, and blog sentiments (Luo and Zhang 2011, p. 6). Using the metric of Web visits, the amount of daily visits of a specific company Web site is recorded, as well as the number of pages browsed by each visitor. Google search queries are calculated on a weekly average basis; first, the search intensity of a firm-related keyword is measured and, second, the search instability of a firm-related keyword. Google search queries reflect the attention that is paid to a company, which stimulates the companies’ brand exposure. Product ratings are evaluated multidimensionally in form of the average rating score, the standard deviation of rating scores, and the volume of ratings, in each case with respect to a specific firm. Finally, weblogs are evaluated regarding their sentiments on a weekly basis. For that, a huge database searches thousands of sources of weblogs for content related to the analyzed firms. (Luo and Zhang 2011, p. 10-14) In comparison, Tirunillai and Tellis (2012) focused only on product ratings and product review sites. Like Luo and Zhang (2001), they assessed the average rating scores and the volume of reviews. Besides the variance, they measured the valence of the assessed reviews (Tirunillai and Tellis 2012, p. 202). The researchers used the three most popular review websites, Amazon.com, Epinions.com, and Yahoo! Shopping, in the period from June 2005 to January 2010 for their analysis. The authors preferred customer reviews because these reviews focus on the topic of product evaluations and the probability to drift off the topic is relatively low, compared to non-topic-based online forums or blogs. (Tirunillai and Tellis 2012, p. 201-202).

[...]

^[1] http://www.Amazon.com/Kindle-eReader-eBook-Reader-e-Reader-Special-Offers/dp/B0051QVESA/ref=sr_tr_sr_1?ie=UTF8&qid=1344162228&sr=8-1&keywords=kindle

^[2] http://www.Amazon.com/Samsung-UN46EH6000-46-Inch-1080p-HDTV/product-reviews/B0071O4EKU/ref=sr_1_cc_1_cm_cr_acr_txt?ie=UTF8&showViewpoints=1

^[3] www.buzzmachine.com/2005/08/17/dear-mr-dell/