{"id":3808,"date":"2013-11-07T20:38:20","date_gmt":"2013-11-08T01:38:20","guid":{"rendered":"http:\/\/jimgworld.com\/blog1\/?p=3808"},"modified":"2013-11-07T20:38:20","modified_gmt":"2013-11-08T01:38:20","slug":"big-data-blues","status":"publish","type":"post","link":"https:\/\/jimgworld.com\/blog1\/?p=3808","title":{"rendered":"Big Data Blues"},"content":{"rendered":"

I’m seeing a lot written these days about “Big Data”<\/a>. I’m not fully versed with it yet, but the concept seems to be grounded in the fact that many people today (especially younger people) put a whole lot of info about themselves on the Internet via social network sites, and through buying stuff on-line, banking on-line, and doing business with government agencies. And there are even more data sources, especially the smartphone: it monitors your calls, figures out your location with GPS locators, and otherwise snitches on what you are doing to some central command, like it or not. Furthermore, businesses (and even government agencies, albeit slowly) do more and more of their business via on-line mechanisms, exposing themselves to data collection. <\/p>\n

All of this has not gone unnoticed by the big communications companies and web site corporations (Google, Facebook, Twitter, etc.); they are compiling huge databases with extremely detailed info about the day to day doings of individuals and corporate entities. And it’s not like all of these databases with all kinds of private information about almost everyone get used only by the compiling company or agency; they put this info out there for sale, so that the info gets amalgamated into mega-databases. <\/p>\n

Obviously, a lot of people get upset about erosion of privacy due to these big databases. But a lot of young people don’t seem particularly worried; there seems to be a digital communalism amidst the Millennials, a hazy, idealistic belief on their part<\/a> that modern communications technology is creating a close-knit feeling of goodwill that is spreading to all corners of the human race. Well, good luck with that; I remember when people my age believed in “Woodstock Nation”. And then we were flung into the adult world, with jobs, mortgages, families, and lots of worries about getting by.<\/p>\n

But back to Big Data and digital idealism; a lot of people, not necessarily all idealistic 20-somethings, believe that Big Data can be used for good. It can help to see bad things coming such as epidemics, crime waves and economic bubbles \/ financial crises. My question is, just who is monitoring these databases for this; most governments or educational institutions aren’t going to pay for access to them (too expensive), so it will be mostly private corporations that detect the “bad waves”. Will they be nice enough to tap on the shoulders of the public agencies that could do something about it (or at least get ready to mop up the damage)? We shall see, this is all pretty new.<\/p>\n

Scientific American has been running a variety of articles in recent issues about Big Data issues. The November, 2013 issue had a good article on the dark side of Big Data (“How To Think About Privacy<\/a>” by Jaron Lanier). The author made a very interesting point about how Big Data analysis can cause trouble for society. He notes that “the traditional entertaining way to tell a cautionary science-fiction tale is to conjure an evil villain who becomes all-powerful.” E.g. we can imagine a huge international corporation that has more power than any government and thus starts planning and directing everyone\u2019s lives; or we can imagine a techno-facist strongman government playing the 1984 role of “big brother”. (Orwell\u2019s 1984<\/a> might finally be realized, about 40 years late). <\/p>\n

But, as Lanier points out, the more immediate danger from big analysis of big data seems to be coming from incompetent bungling on the part of business capitalists. Lanier gives two good examples of how businesses have used big data over the past decade or so to increase their profits, while inadvertently causing back-fire effects that eventually made things worse for these businesses; and at the same time caused a lot of social misery on the part of many everyday people. The first example regards the “massive statistical calculations that allowed American health insurance companies to avoid insuring high-risk customers.” It worked for a while as to increase the profitability of the insurance industry. But the managers and geeks who set up these computer routines did not ask what effect their new policies would eventually have on society and on their employers. They assumed that there would not be any “blowback”. <\/p>\n

But it turned out that society didn’t like the way that insurance companies were enforcing increasingly aggressive and preemptive “high-risk lifestyle \/ pre-existing condition ” limitations, not at all; and so eventually the political world responded. I.e., Obamacare came into being; and now the federal government is deeply involved with how every one of us will gain access to medical care, and how every health care provider and insurance company will provide insurance and care, and pay or be paid for it. And the jury is still out whether the cure itself is going to be worse than the disease.<\/p>\n

The second example is even bigger and even more obvious. Yes, the 2008 financial crisis and the Great Recession that followed it was triggered by a lot of smart analysts and sharp business managers applying lots of analytical fire-power to crunch lots of data. As a result, they came up with really innovative ways to squeeze a lot of profit out of the mortgage lending market. And darn if it didn’t work and work well for a decade or so. Furthermore, unlike the experience of health insurance companies with their high-risk customer limitations, the public also liked what was happening. Many families that could never expect to get the financing to buy their own homes were now in the real estate game. It seemed like a win-win. <\/p>\n

But eventually, certain actors in the system started adapting their behaviors to the new realities, while others didn’t; and so the economic feedback started to turn negative. Housing price increases couldn’t go on forever; at some point, individual buyers and financing entities started pulling back a bit. This little slow-down soon caused housing prices to peak and then even start declining. The business models and financing procedures in place did not anticipate this. And so that little tip set off a chain reaction that resulted in huge economic losses over the next 5 years (the so-called \u201cGDP output gap<\/a>\u201d; and we’re still not completely out of it — the American economy will not be back to its full production potential for at least another 3 years). People lost their homes and jobs, and many of them still haven’t regained the living standards that they had up thru 2008. All because of what turned out to be a short-sighted application of big data by big business.<\/p>\n

So what is the takeaway here? Are we threatened mostly by capitalism’s short-sighted and incompetent use of big data? These two stories seem to point in that direction. But then again, government is always a bit behind the private sector in terms of technology use. Big government is only getting started with big data. And Obamacare seems to be where it will first have a big role<\/a> in directing a major national program, because big data gathering and analysis is crucial to the agencies and mechanisms set up by the Affordable Care Act to control medical costs, along with the monitoring of compliance by insurance companies and health care providers with all of the complex regulations and provisions of the Act. <\/p>\n

Governments around the world do NOT have an unblemished track-record in terms of successfully carrying out major social programs. And many if not most of these failures are not due to evil totalitarian schemes such as fascism or corrupted socialism, nor by pure incompetence in execution. Many of the big flops (in both the US and abroad) are caused by unexpected blowback, unanticipated consequences run amok. There is a fairly popular book out now called “Seemingly Like A State<\/a>” by James Scott, which provides a list of cautionary tales regarding well-intended and not-so-well-intended big government initiatives (often involving central planning and social engineering) that melted down and made things worse for the populace. The world often turns out to be much more complex, chaotic and unpredictable than the central planners believe \u2013 regardless of the good or bad intentions of those planners. And big data \/ big analysis will not turn out to be the cure for this, no more than it was the \u201ccure\u201d for the stodgy, conservative market for home mortgages that arose after WW2 (hey, remember Jimmy Stewart and It\u2019s A Wonderful Life<\/a> \u2013 back when there were still some human values in the home finance system, or so we had hoped!).<\/p>\n

Obviously, President Obama and his top staff know about all this and are mustering all the wisdom they can garner in order to prevent Obamacare from melting down and becoming the next chapter of some future edition of this book. Or so I hope!<\/p>\n","protected":false},"excerpt":{"rendered":"

I’m seeing a lot written these days about “Big Data”. I’m not fully versed with it yet, but the concept seems to be grounded in the fact that many people today (especially younger people) put a whole lot of info about themselves on the Internet via social network sites, and through buying stuff on-line, banking […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,29],"tags":[],"_links":{"self":[{"href":"https:\/\/jimgworld.com\/blog1\/index.php?rest_route=\/wp\/v2\/posts\/3808"}],"collection":[{"href":"https:\/\/jimgworld.com\/blog1\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jimgworld.com\/blog1\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jimgworld.com\/blog1\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jimgworld.com\/blog1\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3808"}],"version-history":[{"count":2,"href":"https:\/\/jimgworld.com\/blog1\/index.php?rest_route=\/wp\/v2\/posts\/3808\/revisions"}],"predecessor-version":[{"id":3810,"href":"https:\/\/jimgworld.com\/blog1\/index.php?rest_route=\/wp\/v2\/posts\/3808\/revisions\/3810"}],"wp:attachment":[{"href":"https:\/\/jimgworld.com\/blog1\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3808"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jimgworld.com\/blog1\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3808"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jimgworld.com\/blog1\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3808"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}