Night has fallen by the time you get off work. Machine learning helps keep you safe as you walk to your car, monitoring the video feed from the surveillance camera in the parking lot and alerting off-site security staff if it detects suspicious activity. On your way home, you stop at the supermarket, where you walk down aisles that were laid out with the help of learning algorithms: which goods to stock, which end-of-aisle displays to set up, whether to put the salsa in the sauce section or next to the tortilla chips. You pay with a credit card. A learning algorithm decided to send you the offer for that card and approved your application. Another one continually looks for suspicious transactions and alerts you if it thinks your card number was stolen. A third one tries to estimate how happy you are with this card. If youвЂ™re a good customer but seem dissatisfied, you get a sweetened offer before you switch to another one.. To date or not to date?. Machine learning is what mathematicians call an ill-posed problem: it doesnвЂ™t have a unique solution. HereвЂ™s a simple ill-posed problem: Which two numbers add up to 1,000? Assuming the numbers are positive, there are five hundred possible answers: 1 and 999, 2 and 998, and so on. The only way to solve an ill-posed problem is to introduce additional assumptions. If I tell you the second number is triple the first, bingo: the answer is 250 and 750.. Humans are not immune to overfitting, either. You could even say that itвЂ™s the root cause of a lot of our evils. Consider the little white girl who, upon seeing a Latina baby at the mall, blurted out вЂњLook, Mom, a baby maid!вЂќ (True event.) ItвЂ™s not that sheвЂ™s a natural-born bigot. Rather, she overgeneralized from the few Latina maids she has seen in her shortlife. The world is full of Latinas with other occupations, but she hasnвЂ™t met them yet. Our beliefs are based on our experience, which gives us a very incomplete picture of the world, and itвЂ™s easy to jump to false conclusions. Being smart and knowledgeable doesnвЂ™t immunize you against overfitting, either. Aristotle overfit when he said that it takes a force to keep an object moving. GalileoвЂ™s genius was to intuit that undisturbed objects keep moving without having visited outer space to witness it firsthand.. A decision tree is like playing a game of twenty questions with an instance. Starting at the root, each node asks about the value of one attribute, and depending on the answer, we follow one or another branch. When we arrive at a leaf, we read off the predicted concept. Each path from the root to a leaf corresponds to a rule. If this reminds you of those annoying phone menus you have to get through when you call customer service, itвЂ™s not an accident: a phone menu is a decision tree. The computer on the other end of the line is playing a game of twenty questions with you to figure out what you want, and each menu is a question.. [РљР°СЂС‚РёРЅРєР°: pic_7.jpg]. He who learns fastest wins. P(cause | effect) = P(cause)Г— P(effect | cause) / P(effect).. And so we have traveled through the territories of the five tribes, gathering their insights, negotiating the border crossings, wondering how the pieces might fit together. We know immensely more now than when we started out. But something is still missing. ThereвЂ™s a gaping hole in the center of the puzzle, making it hard to see the pattern. The problem is that all the learners weвЂ™ve seen so far need a teacher to tell them the right answer. They canвЂ™t learn to distinguish tumor cells from healthy ones unless someone labels them вЂњtumorвЂќ or вЂњhealthy.вЂќ But humans can learn without a teacher; they do it from the day theyвЂ™re born. Like Frodo at the gates of Mordor, our long journey will have been in vain if we donвЂ™t find a way around this barrier. But there is a path past the ramparts and the guards, and the prize is near. Follow meвЂ¦. Clearly, if Robby did know them, it would be smooth sailing: as in NaГЇve Bayes, each cluster would be defined by its probability (17 percent of the objects generated were toys), and by the probability distribution of each attribute among the clusterвЂ™s members (for example, 80 percent of the toys are brown). Robby could estimate these probabilities just by counting the number of toys in the data, the number of brown toys, and so on. But in order to do that, we would need to know which objects are toys. This seems like a tough nut to crack, but it turns out we already know how to do it as well. If Robby has a NaГЇve Bayes classifier and needs to figure out the class of a new object, all he needs to do is apply the classifier and compute the probability of each class given the objectвЂ™s attributes. (Small, fluffy, brown, bear-like, with big eyes, and a bow tie? Probably a toy but possibly an animal.). If we endow Robby the robot with all the learning abilities weвЂ™ve seen so far in this book, heвЂ™ll be pretty smart but still a bit autistic. HeвЂ™ll see the world as a bunch of separate objects, which he can identify, manipulate, and even make predictions about, but he wonвЂ™t understand that the world is a web of interconnections. Robby the doctor would be very good at diagnosing someone with the flu based on his symptoms but unable to suspect that the patient has swine flu because he has been in contact with someone infected with it. Before Google, search engines decided whether a web page was relevant to your query by looking at its content-what else? Brin and PageвЂ™s insight was that the strongest sign a page is relevant is that relevant pages link to it. Similarly, if you want to predict whether a teenager is at risk of starting to smoke, by far the best thing you can do is check whether her close friends smoke. An enzymeвЂ™s shape is as inseparable from the shapes of the molecules it brings together as a lock is from its key. Predator and prey have deeply entwined properties, each evolved to defeat the otherвЂ™s properties. In all of these cases, the best way to understand an entity-whether itвЂ™s a person, an animal, a web page, or a molecule-is to understand how it relates to other entities. This requires a new kind of learning that doesnвЂ™t treat the data as a random sample of unrelated objects but as a glimpse into a complex network. Nodes in the network interact; what you do to one affects the others and comes back to affect you. Relational learners, as theyвЂ™re called, may not quite have social intelligence, but theyвЂ™re the next best thing. In traditional statistical learning, every man is an island, entire of itself. In relational learning, every man is a piece of the continent, a part of the main. Humans arerelational learners, wired to connect, and if we want Robby to grow into a perceptive, socially adept robot, we need to wire him to connect, too.. Although it is less well known, many of the most important technologies in the world are the result of inventing a unifier, a single mechanism that does what previously required many. The Internet, as the name implies, is a network that interconnects networks. Without it, every type of network would need a different protocol to talk to every other, much like we need a different dictionary for every pair of languages in the world. The InternetвЂ™s protocols are an Esperanto that gives each computer the illusion of talking directly to any other and that allows e-mail and the web to ignore the details of the physical infrastructure they flow over. Relational databases do something similar for enterprise applications, allowing developers and users to think in terms of the abstract relational model and ignore the different ways computers go about answering queries. A microprocessor is an assembly of digital electronic components that can mimic any other assembly. Virtual machines allow the same computer to pose as a hundred different computers to a hundred different people at the same time, and help make the cloud possible. Graphical user interfaces let us edit documents, spreadsheets, slide decks, and much else using a common language of windows, menus, and mouse clicks. The computer itself is a unifier: a single device capableof solving any logical or mathematical problem, provided we know how to program it. Even plain old electricity is a kind of unifier: you can generate it from many different sources-coal, gas, nuclear, hydro, wind, solar-and consume it in an infinite variety of ways. A power station doesnвЂ™t know or care how the electricity it produces will be consumed, and your porch light, dishwasher, or brand-new Tesla are oblivious to where their electricity supply comes from. Electricity is the Esperanto of energy. The Master Algorithm is the unifier of machine learning: it lets any application use any learner, by abstracting the learners into a common form that is all the applications need to know.. You gaze intently at the map, trying to decipher its secret. The fifteen pieces all match quite precisely, but you need to figure out how they combine to form just three: the representation, evaluation, and optimization components of the Master Algorithm. Every learner has these three elements, but they vary from tribe to tribe.. Of course, robot armies also raise a whole different specter. According to Hollywood, the future of humanity is to be snuffed out by a gargantuan AI and its vast army of machine minions. (Unless, of course, a plucky hero saves the day in the last five minutes of the movie.) Google already has the gargantuan hardware such an AI would need, and itвЂ™s recently acquired an army of robotics startups to go with it. If we drop the Master Algorithm into its servers, is it game over for humanity? Why yes, of course. ItвЂ™s time to reveal my true agenda, with apologies to Tolkien:. IвЂ™m lucky to work in a very special place, the University of WashingtonвЂ™s Department of Computer Science and Engineering. IвЂ™m also grateful to Josh Tenenbaum, and to everyone in his group, for hosting the sabbatical at MIT during which I started this book. Thanks to Jim Levine, my indefatigable agent, for drinking the Kool-Aid (as he put it) and spreading the word; and to everyone at Levine Greenberg Rostan. Thanks to TJ Kelleher, my amazing editor, for helping make this a better book, chapter by chapter, line by line; and to everyone at Basic Books..