John Foreman, himself a data-scientist, writes a (somewhat rambling but) funny and self-aware essay on machine learning:

Data Privacy, Machine Learning, and the Destruction of Mysterious Humanity

I highly recommend you read it. Keep an eye for two coinages: “data-driven probabilistic determinism” and “data-laundered discrimination”. Machine learning is one side of the argument here. For the other side I also recommend the book Big Data: A Revolution That Will Transform How We Live, Work, and Think

Here are a few juicy quotes from Foreman’s essay:

Our past data betrays our future actions, and rather than put us in a police state, corporations have realized that if they say just the right thing, we’ll put the chains on ourselves.

The promise of better machine learning is not to bring machines up to the level of humans but to bring humans down to the level of machines.

“A human being is a deciding being,” but if our decisions can be hacked by corporations then we have to admit that perhaps we cease to be human as we’ve known it.

A little bit of Huxley there, and reminiscent of Tim Wu who called us humans “comfort-seeking missiles”:

… for most of us, our technological identities are determined by what companies decide to sell based on what they believe we, as consumers, will pay for. … Comfort-seeking missiles, we spend the most to minimize pain and maximize pleasure. When it comes to technologies, we mainly want to make things easy. Not to be bored. Oh, and maybe to look a bit younger.

The imagery of the WALL-E at the end of Foreman’s essay is an appropriate warning.