Big Data and Weather Forecasting

In his article “How Big Data Can Boost Weather Forecasting” for Wired, author Steve Hamm discusses how Big Data analysis techniques are being used to more accurately predict weather patterns. As his first example, Hamm discusses how the Korean Metrological Administration (KMA) is working to upgrade its predictive systems in order to better prepare the Korean peninsula for storms that “[carry] dense clouds of yellow dust from China’s Gobi Desert that are sometimes loaded with heavy metals and carcinogens”. As part of their upgrades, the KMA is dramatically increasing the agency’s storage capabilities in hopes of more accurately forecast weather patterns through an increased ability to quickly analyze large amounts of information.

Such efforts to increase predictive capabilities are also being made in other parts of the world. Following the destruction caused by Hurricane Sandy, “leaders of the city of Hoboken…are considering building a wall around the city to keep the tidal Hudson River at bay”, but such efforts will be in vain if scientists are unable to predict how the changing climate will affect the river’s future depth and behavior. Due to the scale of the issue, IBM is assisting in researching more accurate predictive methods through a project called Deep Thunder, a “long-term weather analysis project”.

Currently, Deep Thunder has been used to accurately predict “the snowfall totals in New York City during the mammoth snowstorm” in February, and was also able to predict “when the snowfall would start and stop”. IBM is currently working to implement Deep Thunder in Rio de Janeiro for the 2016 Olympics, and provide attendees access to the predictive information through “iPad and cloud applications”. The accuracy and speed of Deep Thunder have great implications for the future of climate prediction; if the planet’s weather can be consistently be predicted, the damages and injuries caused by catastrophic weather could be greatly mitigated during future events.

The Rise and Benefits of Big Data

In The Wall Street Journal’s “Big Data is on the Rise, Bringing Big Questions”, author Ben Rooney explores the business significance of Big Data through his coverage of the Silicon Valley Comes to Oxford event and examination of the varied benefits of Big Data analysis. In particular, Rooney discusses that while many businesses are aware of the concept of Big Data, “only about 6% of companies have got beyond a pilot stage” of a Big Data initiative, “and 18% are still in one”. Dean of Oxford’s Said Business School Peter Tufano theorizes that while these companies are interested in Big Data analysis for the financial benefits, many are still not sure as to how Big Data can best serve their company’s needs.

Though some individual business have not yet determined exactly how Big Data can benefit them, “those companies that are able to use data effectively are more likely to win in the marketplace”. For example, in the field of personal location data, “some $100 billion of value can be created globally for service providers”. Additionally, the power of Big Data is not proven simply in profits; “John Aristotle Phillips, Chief Executive of Aristotle International” said that while “[the] election was not won because of Big Data…the use of data analytics had a material effect on outcomes” of the most recent presidential election.

In spite of these benefits, however, consumer-driven companies are not embracing Big Data as quickly as some would expect. “Andrew Grant, Chairman of Satalia, a U.K. university spinout that applies algorithms to optimize Big Data, suggests” that for many consumer-driven companies, “cultural obstacles are the biggest impediment” to Big Data implementation. Rooney also discusses that some consumers may find the predictive powers of Big Data to be off-putting; “[the] richest examples of Big Data are to understand consumer behavior and optimize your product for it”, and some customers prefer less targeted marketing tactics.

Chicago as the New Silicon Valley

In Amy Scott’s article “Creating the Next Silicon Valley in Chicago” for Marketplace, she discusses the possibility of an upcoming technological boom in Chicago. Specifically, she references the presence of the University of Illinois at Urbana Champaign as a potential source of talent for such a boom; the school itself is widely respected, but “its students are often drawn to other shores”.  Talent retention has been a concern for Chicago in the past, as “[founders] of Youtube, Paypal, and Yelp also studied there before heading west”. To help solve this issue, Chicago Mayor Rahm Emanuel “[wants] to draw the talent [to Chicago]”, and believes that “the companies will follow”.

To achieve this goal, Chicago is emphasizing the creation and maintenance spaces in which companies and new technologies can grow. The Merchandise Mart, “a massive building of showrooms and shops”, is becoming such a hub. Motorola Mobility now occupies space within the building, as do 100 startups sharing office space.

An issue that Chicago is facing, however, is the overall culture within the city; according to Margaret O’Mara of the University of Washington, Silicon Valley is unique in its “extraordinary tolerance or risk and failure”, and that such a culture doesn’t yet exist in Chicago. If that cultural impediment can be overcome, however, Chicago holds great potential for new companies. Lightbank, “a venture capital firm run by the co-founders of Groupon” has already begun changing the economic culture of tech startups in the city, having invested in 53 companies after only 18 months. If such trends continue, the potential technological success of Chicago would likely have far-reaching economic implications for the Midwest as a whole.

Voting Online and Computer Security

In his piece entitled “Why We Still Can’t Vote Online, and Why That May be a Good Thing”, Marketplace’s David Brancaccio responds to an article (“Why Can’t We Vote Online?” from the Verge) concerning the questions regarding and possibility of election voting being conducted online. The Verge’s article presents a thorough overview of the issue, taking into consideration the desires of American voters, political leaders, and the logistical hurdles that would have to be overcome for online voting to be a possibility.

Each article addresses some of the major hurdles in the concept of online voting. First, there seems to be an apparent “lack of enthusiasm when it comes to getting a working online voting system up and running” (Brancaccio). Though Dave Mason, “a former commissioner of the Federal Election Commission, said that ‘it’s just a matter of time until people demand that we vote on the internet’” (Sottek), that time is apparently not today; according to The Verge, there is very little in the way of citizen demand for online voting systems. Secondly, “any system for online voting would need governmental approval from jurisdictions across the country” (Brancaccio), which in itself presents a significant logistical issue.

However, the largest issue facing the implementation of online voting is that of security. Building a hack-proof system that would remain secure even on potentially compromised personal computers is a gargantuan task. Because of the importance of voting and the “stakes of [an]…election”, voting will most likely remain a pen-and-paper affair for now (Brancaccio).

The App Market and Location-Based Services

In Computer Weekly’s article “Apple iPad App Pushes the Location-Based Cloud”, author Adrian Bridgwater discusses how location-based service technology is currently being used in apps for Apple devices. In particular, he discusses how these technologies relate to the increasing prevalence of cloud computing-based resources and devices. Additionally, he discusses to main types of location-based services: “push” LBS, “pull” (also known as “query”) LBS, and a multi-user LBS service, which is the article’s primary focus.

Specifically, Bridgwater explores the benefits and potential pitfalls of an app called Find My Friends. Find My Friends allows users to view each other’s current locations using their iCloud Apple accounts. Additionally, the app can be used in more specific situations; rather than broadcasting one’s location to one’s friends at all times, Find My Friends allows users to “share [their] location with a group of friends for a limited time”. It can be used to monitor when others leave or arrive at a certain location, and can also distribute that information to one’s own contacts.

While Find My Friends certainly has great potential benefit for users, it also presents a new opportunity for developers to create “a new breed of apps that allows us to interact with each other based on where we are [and] what we’re doing”. However, such possibilities do come with potential privacy concerns. As location-based sharing requires the permission of each user to be fully utilized, individuals’ particular privacy settings could affect the efficacy of LBS-reliant applications. Nevertheless, the increased prevalence of cloud and LBS applications identifies an area of great potential within the apps market.

Crowdsourcing and Online Job Creation

Crowdsourcing is quickly becoming a powerful productivity tool in the modern market. In many fields, crowdsourcing solutions are ultimately cheaper and more efficient than traditional methods. This trend of increased efficiency and decreased cost is particularly noticeable in the realm of the voice-over; in an article from Marketplace, “Could Crowdsourcing Talent Online Create Jobs?”, author David Brancaccio explores the effect that crowdsourcing has had on the voice-over industry, and what these effects imply regarding the future of the market.

In particular, Brancaccio focuses on a company called VoiceBunny. VoiceBunny is an online service that allows clients and voice actors to more easily connect with one another; “client offers a script online and people who know how to read aloud offer their services”. Additionally, the software itself helps clients find the most suitable talent for their specific script.

Ultimately, this service is significantly cheaper for companies because of the lack of overhead costs. Rather than hiring voice over actors, perhaps renting studio space, and distributing the final recording themselves, companies are able to get a good-quality voiceover for “$11…plus a $2.20 service fee”. Because they work as independent contractors, the voice-over artists themselves are responsible for providing and maintaining their own equipment. Additionally, VoiceBunny offers podcasting services; through these services, clients can purchase spoken versions of their articles. These spoken versions can then be distributed by VoiceBunny or by the authors themselves.

Because of its cheapness and its ease of use, however, VoiceBunny does have interesting implications regarding the future of the voice-over market. Though many voice-over artists still receive the majority of their income “from bigger gigs outside of [the VoiceBunny] service”, many of the jobs available on VoiceBunny are ones that, in the past, would have gone to more local voice-recording artists. What was once a local market has become a globalized one and, consequently, more people are able to offer and be paid for their services. Indeed, services like VoiceBunny have the potential to change how the market as a whole operates; it is not inconceivable to imagine that hosting duties of local radio shows may one day be crowdsourced as well.

Crowdsourcing the Dictionary

In hopes of discovering and recording new and creative words, the staff at the UK’s Collins English Dictionary has begun crowdsourcing the entries in their dictionary. Collins, which added “crowdsourcing” to its own dictionary in 2009, is asking internet users to contribute their own words to this project. So far, there’s been a good response; “[in] the first two weeks of the initiative, there were 2,637 suggestions from more than 2,000 different users.”

While the Boston Globe’s article “Crowdsourcing the Dictionary” initially suggests that Collins’ project may initially seem more similar in nature to online dictionaries such as Urban Dictionary, it soon becomes clear that the concept of dictionaries incorporating user-submitted content is not in fact a new one; the Oxford English Dictionary, for example, put out its first call for user submissions in 1879. Rather, it is the ease of submission to and user influence on the dictionary that makes the Collins English Dictionary unique. While Merriam-Webster has a similar word submission project called Open Dictionary that has received “nearly 20,000 suggestions from users since…2005”, the Merriam-Webster editors use it simply as research inspiration. In contrast, the Collins English Dictionary aims integrate user submissions with their current online dictionary.

This project has interesting implications for future projects both within and outside of the lexical world. While soliciting user input is not a new business model for dictionary groups, the amount of moderated, user-generated content within Collins’ final product is unique. Though Collins’ model of user-based submissions makes improvements on past models, there is still much progress that could be made in terms of user interactivity and influence. However, such concessions are necessary if a specific level of quality is desired for the final product.

Big Data in Space

The article “Space: The Big Data Frontier” examines the implications that big data analysis will have on the progression of astronomical discovery and exploration. Specifically, the article discusses the issues that will face researchers should the funding for the Large Synoptic Survey Telescope (LSST) project be renewed. As budget renewal does seem likely, scientists must finalize and implement big data solutions soon.

 

The data problems that are facing the LSST team are massive; while Kirk Borne, the Chair of Information and Statistics for LSST, makes clear that the technicalities of data storage are not an issue, their challenge will be finding meaningful information in all of the data that the LSST will collect. It’s estimated that the LSST will collect “around ten petabytes [approximately ten million gigabytes] of data per year”.

 

Author Mari Silbey also addresses another issue that the LSST team will face – the issue of bandwidth. While the telescope will be in Chile, its data will be transported to Illinois each day, and “[while] sufficient bandwidth for moving big data around may be available in a few select geographic regions, it certainly doesn’t exist everywhere researchers reside.” However, public and private institutions are working towards resolving the bandwidth problems, and researchers are also working to improve algorithmic analysis in order to streamline the data that is to be transferred.

 

Though I’m not a formal student of astronomy, I’ve always found the topic interesting; ultimately, I believe that the wealth of data that the LSST project intends to provide will be invaluable in our understanding of space.  Additionally, I think that the utilization of big data analysis will be essential in the success of this project; the Sloan Digital Sky Survey “produced roughly twenty thousand academic papers”, and the LSST will produce an amount of data equivalent to the entire Sloan Digital Sky Survey every three days. Without big data analysis to help process this massive amount of information, much of the LSST’s discoveries would remain unexplored. With big data analytics, however, we will better be able to parse and utilize the discovered information to further our scientific understanding of space.

The Rise and Fall of Nokia

In this article from PandoDaily, Farhad Manjoo predicts that Nokia’s end as a company is near. Specifically, he claims that Nokia’s reliance on other companies’ software is the primary reason why Nokia is currently struggling; whereas successful mobile companies such as Google and Apple provide the software and, occasionally, the hardware for the massively popular Android and iPhone devices, “Nokia…always thought of itself as being in the device business. It made hardware, and it only cared about software to the extent that it needed code to run on those devices.” A memo sent to Nokia’s employees by recently appointed CEO Stephen Elop supports Manjoo’s claim. In it, Elop points out that while Nokia had continued to create solid hardware, they had failed to create entire device ecosystems.

In spite of Elop’s timely analysis of the issue, Nokia is nevertheless in decline. Referencing Apple’s 1990s similar decline in the face of Microsoft’s market ubiquity, Manjoo suggests that Apple was able to regain strength by “changing the rules of the game”, offering a solution that went beyond simply providing PCs. Nokia, however, isn’t showing the same type of innovation. In an attempt to quickly reverse their deterioration, Nokia has partnered with Windows in distributing their mobile OS. However, their success now relies on that of the Windows mobile OS, and Nokia/Windows Lumia phones aren’t selling as well as either company would hope. While Elop has hopes for the upcoming Windows 8 phones, Manjoo is not as optimistic regarding Nokia’s overall fate.

On a broader scale, however, this article is very telling as to how the device market has changed in the past years. Nokia’s decline suggests that it’s no longer economically feasible for IT production companies to produce only one element of a total product and subsequently expect to be both independent from other companies and financially successful. Rather, to succeed in the current business climate, companies need to consider providing combined IT solutions to consumers’ problems.

The Development of Data Visualization

Data visualization techniques can be used to further a user’s understanding of massive data sets, and can be used for nearly any information type. Many popular data visualization techniques are adapted from already well-known aesthetics such as pie charts, heat mapping technologies, and subway maps. Additionally, data visualization standards have been influenced by the concepts of proximity, similarity, and size; many designers group associated topics near to each other and use large or small graphical representations for frequently or rarely occurring data points respectively. Smashing Magazine’s article “Data Visualization: Modern Approaches” gives a brief overview of particularly compelling data visualizations, including ChalkLabs CTO Bruce Herr’s visualization of Wikipedia articles.

While data visualization techniques are undeniably effective for static data sets (such as a city’s population demographics at a specific time), they’re also extremely useful when representing live or variable sets of information. For example, OnISP’s live call map is a live visual representation of all of the calls being made or received by phones using OnISP’s VoIP service.

This type of visualization is particularly effective due to its activity; simply listing how many users a system has or how many connections are made per minute can be informative, but it’s not particularly attention grabbing. In contrast, a real-time map tracking each connection made through OnISP gives the user a more concrete example of the significance of the represented data points. The user is able to personally experience the data, subsequently assisting in his understanding and processing of the represented information.

The increased use of data visualization is particularly important for businesses to keep in mind when considering how best to communicate pertinent information from their Big Data sets. Data visualization makes it easier for users to understand large, complicated data sets, ultimately increasing user ease of use and efficiency.