By Steve Hamm
IBM hosted the Cognitive Systems Colloquium at its famed IBM Research Center in Yorktown Heights, N.Y., on Oct. 2, 2013. The all-day event brought together leaders in science, technology and psychology to discuss the coming era of cognitive computing and to craft a shared agenda among industry, academia and government.
The following is a time-stamped stream of live updates and insights from the event from presenters including, Nobel Prize laureate Daniel Kahneman, A.I pioneer Danny Hillis, Irving Wladawsky-Berger, Visiting Professor, MIT and Imperial College, and others.
9:25 AM ET
Zach Lemnios, IBM Research
What’s happening is really profound and transformational in computing. The notion of building machines that can interact naturally with humans and augment human intelligence is profound. Machines will help people do what they do even better. We call it the era of cognitive computing.
It will be disruptive. It will change the relationship that we have with information, just like smartphones have. There are many unknowns. It will require new architectures. We’ll spot gaps where work needs to be done. It will require talent across IBM, academia and our clients. We have to cohere this work into a set of systems that we can turn into products and services for our clients.A lot of work has to be done together to deepen this understanding. No one person or company can do it alone. That’s why we’re gathered together today.
John E. Kelly III, Director, IBM Research:
Something big is happening in cognitive computing. It’s much bigger than Watson. We’re here today because this is bigger than us. It’s bigger than the IBM company. We need your help.
The move to this era is being driven by four synergistic factors: big data, mobile connectivity, social networking and the Internet of Things. Every element here has exponential growth.
There are no physical limits to the growth of data—which will challenge our ability to process and store it. If we try to use current-generation computing against this wave, we’re done. So we need a whole new set of systems.
We go around the world to get access to the biggest collections of data, including the Square Kilometer Array radio telescope project, the EU’s Human Brain Project, and the US BRAIN Initiative. We need cognitive systems close to those sources of data to process it in real time.
We found in Watson, that the very process of analyzing the data increases the amount of data we have to deal with and store by orders of magnitude. But the value of the information grows exponentially as well.
The first eras of computing were about automating human tasks. This era is fundamentally different. This era will about scaling and magnifying human capability. The separation and machine will blur. The synergy between the two will shine though. A key element: unlocking insights within a very narrow time window.
Healthcare is an example. We’re near a tipping point in healthcare. We can map individual’s genomes. We can map the genomes of specific cancer tumors. We can access all of the medical literature in the world instantaneously. So now, with cognitive systems, we can take full advantage of all of this knowledge, apply it to the problems of particular patients, and arrive at better diagnoses and treatments.
We think it would be valuable to have a system that actually does a deep analysis of a question or statement from a human. It could spot flawed logic and suggest alternative logic and show the evidence to support it.
We even thing cognitive systems will be able to help with creativity. One of our teams built a system that takes all of the world’s known recipes, and breaks them down to the components and the chemicals. They matched it with information about what people like in tastes. And they came up with a system that creates new dishes that are novel and tasty.
We know we can’t do this alone. We have teamed with universities, starting with MIT, Carnegie Mellon University, Rensselaer and NYU, to drive these technologies forward. We’ll also involve our clients. We’ll have colloquia at our labs around the world over the next few months to keep the dialogue going.
There are incredible opportunities ahead of us to create this new era of cognitive computing together.
Daniel Kahneman, Nobel laureate and Princeton professor emeritus
The people here today want to augment the human brain with computing assistance. So I’ll talk about the human brain—things it doesn’t do well or it could do better—which could be augmented by computers.
In my book, Thinking Fast and Slow, I write about two systems of thinking. System 1 is what happens automatically. System 2 is what happens in a controlled fashion. The systems correspond to two strands in artificial intelligence. One strand is solving problems in a logical fashion. That’s System 2. And there’s the strand that has to do with vision and the other senses. That’s System 1.
Only through System 2 can we think about two things at once and shift rapidly between them. We can quickly decide if we like something using System 1, but we choose between two things using System 2.
We think of ourselves as thinking individuals; System 2. But most of what we do is System 1, reacting automatically. These are thoughts that happen to us. We don’t have to work to make them happen.
System 1 enables us to act very quickly, but it also sometimes leads to mistakes.
Here’s an example: My wife and I were having dinner with another couple. When we headed back home, my wife said the man is “sexy.” Then I heard her say, “He doesn’t undress the maid himself.” I said, what do you mean? It turned out that what she really said was, “He doesn’t underestimate himself.” It takes one mistaken word to set a context and change the interpretation of things. It had been primed by the word “sexy” to hear an implausible sentence. That’s the kind of mistake that System 1 makes.
Because of System 1, we tend to be overconfident. We suppress complexity.
The main limitation of System 1 is that it creates the impression that we live in a world that is much more coherent and much simpler than the real world is.
But there’s also a marvel in System 1. We can develop expertise, and within the domain of our expertise the biases and shortcomings are minimized. The mistakes System 1 creates happen when we’re operating outside our areas of expertise.
What is intelligence, and what would be an intelligent computer? Traditionally, intelligence tests are tests of System 2. But I wonder what would be a test of System 1. It would be a test that shows whether we have a rich an accurate updating of a model of the world. It would assess the quality of the model that people have in their minds. We could set up targeted scenarios using System 2 that help create our model of the world, so that when we encounter situations we know better how to respond to them.
How can we use cognitive systems? We can think of an intelligent agent as an aide retrieving relevant facts. We can think of it as a critic—augmenting System 2. We can think of it as an alternative System 1. Probably the most difficult of them all would be modeling System 1.
Danny Hillis, Co-Chairman, Applied Minds
The thinking fast/think slow dichotomy can be applied to the way we think about A.I.
There has been a split between the symbolic planners and the connectionists in A.I. A lot of people saw it as an either/or situation. But intelligence requires both functions and artificial intelligence requires both functions.
We have long thought of the computer as System 2, slow thinking. It goes back to Alan Turing and his Turing Machine idea. All of computer science was about sequential deep thinking, a long chain of logic.
That was the dominant part of A.I., about building a rich and accurate model of the world. We came up with precise definition of concepts for computers. We trained them with the definitions of the concepts. The thought we’d just keep doing this and use it to build a rich and accurate model of the world.
But it turned out it worked well with things that were very hard, like chess. But things that were so easy we didn’t notice them, like recognizing a cat, computers couldn’t do.
But there was also some System 1 work, and at the core was the idea of a “perceptron.” If you broke down the elements of a thing into elements, the learning machines could put together all the elements, the perceptrons, and recognize a cat. People used neural networks to perform this kind of recognition—for things like handwriting recognition. The thought was if you could add more and more layers of perceptrons, you could develop more difficult concepts in the machine. But we didn’t have the computing power to do it.
Now we have the power. And we’re building five-layer networks that can look at objects and develop the ability to categorize—the System 1 capability.
Now it’s time to put the two systems together. We can build systems with some System 1 components and some System 2 components together. I’d argue that Watson is such a system. Its learning system, like a perceptron, weighs the possible answers and ranks them based on confidence.
Every Google search today is split into two searches. First there’s the traditional keyword search. But it also does a semantic search, based on a rich and accurate model of the world. It uses perceptrons, those weighting algorithms. This is more and more the way you get a search result. It guesses what you’re looking for and gives you better answers. It’s kind of like Watson.
We’re building knowledge bases—models of how the world works that machines can use to understand the world and help answer questions or make decisions.
Machines will have to be good at storytelling. They’ll have to explain how they reached their conclusions, so people will trust them.
Danny Hillis, Co-Chairman, Applied Minds: Audience Q&A
Question: Will machines know when there isn’t enough data to make the right decision?
Hillis: In humans, the systems compliment each other. System 1 comes up with an answer. System 2 is a critic. How much support is there for the answer? That kind of reasoning will go on in machines, and storytelling will help out with that. What I mean by storytelling is going back over the conclusion and identifying the cause-and-effect structure that led to it, and explaining that.
People should think of their relationship with computers like that of a violinist with a violin. The machine is your tool. You express yourself through the machine.
Thomas Malone, MIT
When I was an undergraduate in 1970s, I read an article by A.I. pioneer Marvin Minsky. I was inspired, but I worried that everything he laid out would be solved by the time I graduated.
But a friend assured me that those problems wouldn’t be solved for years. And many of them haven’t been resolved today—40 years later.
I was also inspired by J.C.R. Licklider, when I read his 1960 article, Man-Computer Symbiosis. A key quote: “The hope is that in not too many years human brains and computing machines will be coupled together very tightly, and the resulting partnership will think as no human brain has ever thought and process data in a way not approached by the information-handling machines we know today.”
From that idea, I’ve worked at the Center for Collective Intelligence on understanding the intelligence not just in individual brains or individual computers, but in groups. That’s collective intelligence–groups of individuals acting collectively in ways that seem intelligent.
We have seen new kinds of collective intelligence, enabled by the Internet. Google searches are an example. So is Wikipedia.
This is not the end of the story but just barely the beginning. We need to understand the possibilities much more deeply
The question we ask is: How can people and computers and be connected so that collectively they act more intelligently than any person or computer has ever done before?
In our research, we have been able to establish that you can measure collective intelligence. We also found that social intelligence predicts collective intelligence. Psychologists call those theory-of-mind skills.
At least two things matter in group effectiveness:
How well the individuals perform the basic tasks, and how well the individuals work together.
Computers can help by doing all or part of the basic tasks. They can be thinking partners. But they can also help by helping the people work together. The machines can be more useful to the degree that they have social intelligence, and the ease with which people can work with them. The machines need to have good models of the people they’re working with.
Collective intelligence also comes to bear on large organizations. The success of a company is determined not just by its products, but by the processes the organizations use to create their products. It’s possible to have innovations in organizational design. And cognitive computers will play a role in the way organizations are designed and how they operate.
You can envision different scenarios where people and cognitive computers will work together.
In so-called prediction markets, people buy and sell predictions about future events. The prices of the shares estimate the probabilities of given outcomes. It’s a kind of crowd sourcing. We combined people and computers to predict what would be the next play in a football game, a pass or a run. We had people and computer agents making predictions. Humans were better than agents, but computers and agents combined were more accurate than humans.
We need to develop a research agenda for applying cognitive systems to group problem solving.
We need new social programming protocols.
We need a social operating system that manages human resources.
We need new software engineering techniques where the engineers don’t just design software; they design organizations.
One last big thought about where this might be leading us:
As the world becomes more interconnected through the use of communications technology, it may become useful to view all the people and computers as part of a single global brain. It’s possible that the survival of our species will depend on combining human and machine intelligence to make choices that are not just smart but are also wise.
Kathleen McKeown, Columbia University
In our natural language work at Columbia, we’re working on generating presentations that connect events, opinions about them, personal accounts, and assessment of their impact on the world—and linking to supporting science.
This is text-to-text generation. It’s supervised machine learning. We’re working from news, scientific journal articles, online discussion forums and personal narratives. Each one poses different challenges.
We use a lot of different techniques, including summarization, generation, and translation. Our news system, NewsBlaster, summarizes news from the Web. It downloads articles and clusters them into events and generates summaries o the events.
Summarization is hard. It requires both interpretation and generation of text. When we deal with journal articles, we interpret the sentiment. Is the statement positive or negative?
In online discussions, we have to see if the poster is reliable. We check to see how influential they are in their networks.
We’re just beginning to look at personal narratives. We use the event of Hurricane Sandy to experiment will all of these modes of communication. In the end we want to be able to combine all these resources into one summary. We organize the information in terms of time and place. We present it in maps, so you can zoom in on the pieces of information you’re interested in.
We expect this kind of technology to be used in finance, medicine, politics and in social science and humanities.
Research is needed. Unfortunately, the US government is failing to compete. Businesses needs to step up. At our Institute of Data Sciences and Engineering, we look for collaborations with industry.
Panel Discussion: The Cognitive Economy – Decision Making in an Era of Uncertainty
Irving Wladawsky-Berger, moderator:
Thanks to big data and analysis tools, we can turn our analysis on social-technical systems so we can understand how they work. One such social-technical system is health care.
Douglas Johnston, Cleveland Clinic:
For healthcare generally, we’ve been applying data science poorly. We rely on administrative data and, even with EMR, we still have problems. We face the garbage-in-garbage-out dilemma. Much of the data is wildly inaccurate. How do we put together a collective and cognitive system that can help us make better decisions? I hope that systems like Watson will help us with this.
Help us come up with new ways of recording what’s wrong with our patients and how we’re treating them that’s vastly superior to dictation.
The inefficiencies are so rampant in what we do in healthcare. The system is not developed to support the level of care we provide at Cleveland Clinic. The model of how the data is gathered, distributed and accessed has to be totally rethought and remade. We surgeons need something like a fighter-jet dashboard to do our jobs better.
I’m working with Paul Horn at NYU in an effort to develop a new science of cities. Mike Carlen, how do you leverage all of these data in a crisis situation?
Mike Carlen, DTE Energy:
We’re working with IBM Research so we can forecast severe weather events as they approach and build a model to determine what the impact will be on our assets—then develop a plan for responding.
We look at wind speed. How hard will it blow and what direction will it come from? Do we have a lot of trees? Are a lot of them dead? Will they come down? Also, we look at all the resources we have, and how quickly we can deploy them.
The challenge is that we’ll have a plan, but the event will be different from our plan. How do we evolve our response?
Kevin, the financial services industry hasn’t done well with predicting crises. What’s it going to take, in terms of data analytics, to help the industry perform better?
Kevin Reardon, IBM:
At IBM, we have acquired and divested many companies in the past decade. It’s transformational both for the company and the way we approach it. We’re changing from a hardware centric company to a services centric company with software as an engine. We had to divest and then redeploy capital into new areas.
We put a lot of energy into getting fact-based information about acquisition targets. We had to take the emotion out of the equation.
We use a lot of data and statistical techniques to evaluate acquisitions. We learned we had to figure out if the cultures would mesh. We had to look at us from their point of view. Why would they become a part of a large organization. We could help then scale globally. We’d see if there was going to be a culture class.
The accumulated a lot of history and data on acquisitions. We have started to use unstructured data for evaluating cultural fit and even financial performance.
The Cognitive Economy – Panel Q&A:
Q: Won’t the cognitive economy make corporations flatter and more socially networked. Won’t decisions be made differently?
Irving Wladawsky-Berger: The macho leader probably isn’t the best leader in this work. It’s better to have a social leader. You can’t just tell people what to do. Social leadership seems to be more important than ever.
Kevin Reardon: There will be a much broader set of sources of information, but I don’t see the decision-making progress changing much.
Q: In a world where systems are logically superior and have access to more data, what it will do to people’s self image.
Mike Carlen: I welcome the information—the ability to interact and make better decisions.
Douglas Johnston: I’m thankful my specialty is a very physical one. I think it will make some of what we do irrelevant. I think those will be the less interesting things.
Q: Will cognitive computers have common sense? Companies make huge mistakes in M&A. Financial systems collapse. It seems like people should have seen these things coming. Will computers help?
Reardon: In the M&A community, there’s a recognition that fact-based decision making and using data analytics is essential.