Distributed representations can provide a statistical advantage when an apparently complicated structure can be compactly represented with a small number of parameters. In contrast, some traditional nondistributed learning algorithms generalize only due to the smoothness assumption, which states that if $$u \approx v$$, then the target function $$f$$ to be learned has the property that $$f(u) \approx f(v)$$ in general. While this assumption is useful, it suffers from the curse of dimensionality.

Benefits of Distributed Representations

The curse of dimensionality refers to the challenge where many machine learning problems become exceedingly difficult when the number of dimensions (features) in the data is high. Specifically, the number of possible configurations of variables increases exponentially with the number of dimensions. For a dataset with $$d$$ dimensions and $$v$$ values to be distinguished for each dimension, the space is divided into $$O(v^d)$$ regions, requiring an exponential number of examples to adequately cover all configurations.

University of Connecticut

Google

Deep learning is taking off because as the amount of data and computational power increases, deep learning algorithms perform better than traditional machine learning methods. 

You can improve performance by increasing the scale of the NN or giving it more data. Increasing the scale of the NN often places greater strain on technological systems. The more accessible route is taking advantage of the increase in available data.

“Why is deep learning taking off?” 

The inability for traditional algorithms to generalize complex problems was what inspired machine learning in the first place. Today, a similar situation has arose that complex problems with high dimensional inputs (high number of features) are not generalizable with most simple machine learning algorithms, not to mention require high computational power to solve. This is what drives the development of deep learning, as it will solve some problems better than machine learning can.

Challenges Motivating Deep Learning

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

https://machinelearning-blog.com/2017/11/03/erster-blogbeitrag/#:~:text=Getting%20a%20better%20accuracy%20with,or%20huge%20amounts%20of%20data.&text=The%20recent%20breakthroughs%20in%20the,use%20more%20and%20more%20data.

If the basic technical ideas behind Deep Learning are around for decades, why are they taking off today? 

https://towardsdatascience.com/how-do-you-know-you-have-enough-training-data-ad9b1fd679ee

Why deep learning is taking off source

https://medium.com/@shaoliang.jia/why-is-deep-learning-taking-off-8244175929e7

Learn Before

Related

Learn After