PDF(Sci-hub)
Abstract：
There has been a growing interest in expressivity of deep neural networks. However, most of the existing work about this topic focuses only on the specific activation function such as ReLU or sigmoid. In this paper, we investigate the approximation ability of deep neural networks with a broad class of activation functions. This class of activation functions includes most of frequently used activation functions. We derive the required depth, width and sparsity of a deep neural network to approximate any Hölder smooth function upto a given approximation error for the large class of activation functions. Based on our approximation error analysis, we derive the minimax optimality of the deep neural network estimators with the general activation functions in both regression and classification problems.
摘要：
人们对深度神经网络的表达能力越来越感兴趣。然而,有关此主题的大多数现有工作仅关注特定的激活函数，例如ReLU或sigmoid。在本文中,我们研究了具有广泛激活函数的深度神经网络的逼近能力。这类激活函数包括大多数经常使用的激活函数。我们得出所需的深度，深度神经网络的宽度和稀疏性，以逼近任何Hölder平滑函数，直到大型激活函数的给定逼近误差。根据我们的近似误差分析，在回归和分类问题中，我们用一般激活函数推导了深度神经网络估计器的极小极大最优性。
