There have been tremendous advances in AI and lots of buzz around the possibility of the singularity coming soon acclaimed by scientists like Kurzweil. Inspired by the success of Deep learning, the new kid in town that took the field by storm winning almost all machine learning competitions, this possibility seems realistic as never before.
Shouldn’t we all celebrate and rejoice the triumph of AI? It seems that all we have to do to achieve human level intelligence is use bigger networks, acquire more data and invoke more computational power. I’m still not convinced of this. Sometimes temporary success has the perverse effect of keeping us on the wrong path. Not saying that deep learning is wrong or isn’t powerful to solve many problems, but some fundamental elements are still missing.
What worries me the most in AI is the lack of common sense. Why is it so hard for machines to understand simple layman questions while at the same time they can solve the most complex equations? My guess is that intelligence is not a monolithic machinery but is made of many building blocks working in parallel at different scales. We are obsessed with symbolic manipulation and logic consistency, but maybe we need other less powerful and messier building blocks. Maybe abstractions have to be grounded in the concrete and evolve from the daily experience of our interaction with the world. After all math and logic isn’t the first thing we teach babies.
In a common sense language, maybe we are building the house by the roof. Logic and symbolic processing is incredible powerful, but to be useful these symbols has to have meaning. How can a machine be taught meaning? I am convinced that only after we engineer the process of grounded abstractions, by creating the right architecture and the right learning algorithms, can we build a new generation of AI – a meaningful one. Symbolic manipulation without meaning wouldn’t take us very far.
I think that more important than the architectures is the learning algorithms. Most, if not all, AI can be framed as a pattern recognition problem: inputs (data) -> map (algorithm) -> output (predictions). This approach is appropriated to classify objects, voice or detect subtle patterns in data, either supervised or unsupervised. These maps can take very forms, a neural network, a Hidden Markov Chain, etc. In probabilistic terms, the problem can be framed as an evaluation of the posterior probability distribution of a sequence of events in time or space based on previous evidences.
Lots of situations can be solved by this approach, like detecting fraud, finding objects in images, or weather predicting. But for other problems this approach is incomplete. Take language. Sure you can apply probabilistic methods to predict the next word after a given sequence but this will lead to a far too complex function. Is there any other way to do it?
I think the answer is yes, but we have to frame the problem differently. First we need to realize that words are not independent atomistic symbols. They have no meaning per se. Meaning requires an observer: its a two way process. Meaning isn’t absolute, it emerges from the interaction of an observer and an object. Likewise, there is no intrinsic meaning in data.
This may seem a triviality, but you will be surprised by the efforts the AI community puts in “understanding” data as if there was something to be understood without an implicit human perspective.
Language has a social component and its grounded in our actions and our life history. Words are just abstractions, or pointers as I call them, to actions, situations or memories. It’s a compact and abstract way to describe objects, situations or emotions. All words have a subjective meaning that is embedded in their own internal neural mapping.
So how can we create artificial devices to capture the meaning of language and capable of common sense understanding or sense of humour? My next blog entry will address this issue.