论文题目：β-DARTS: Beta-Decay Regularization for Differentiable Architecture Search
作者列表：Peng Ye (Fudan University), Baopu Li (BAIDU USA LLC), Yikang Li (Shanghai AI Laboratory), Tao Chen (Fudan University), Jiayuan Fan (Fudan University), Wanli Ouyang (The University of Sydney, SenseTime Computer Vision Group, Australia)
论文摘要：Neural Architecture Search (NAS) has attracted increasingly more attention in recent years because of its capability to design deep neural network automatically. Among them, differential NAS approaches such as DARTS, have gained popularity for the search efficiency. However, they suffer from two main issues, the weak robustness to the performance collapse and the poor generalization ability of the searched architectures. To solve these two problems, a simple-but-efficient regularization method, termed as Beta-Decay, is proposed to regularize the DARTS-based NAS searching process. Specifically, Beta-Decay regularization can impose constraints to keep the value and variance of activated architecture parameters from too large. Furthermore, we provide in-depth theoretical analysis on how it works and why it works. Experimental results on NAS-Bench-201 show that our proposed method can help to stabilize the searching process and makes the searched network more transferable across different datasets. In addition, our search scheme shows an outstanding property of being less dependent on training time and data. Comprehensive experiments on a variety of search spaces and datasets validate the effectiveness of the proposed method. The code is available at https://github.com /Sunshine-Ye/Beta-DARTS .
论文信息：
P. Ye, B. Li, Y. Li, T. Chen, J. Fan, W. Ouyang, “Beta-decay regularization for differentiable architecture search,”Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR oral), in press, 2022.
论文链接：https://arxiv.org/abs/2203.01665v1
代码链接：https://github.com/Sunshine-Ye/Beta-DARTS
视频讲者简介：叶鹏，复旦大学EDL lab博士研究生，主要研究方向是计算机视觉、模型轻量化和神经架构搜索。
