AI算法新论文诸位来点评一下
版主: verdelite, Tlexander
-
- 见习点评
- 帖子: 1603
- 注册时间: 2022年 7月 23日 09:43
#1 AI算法新论文诸位来点评一下
点评一下,是不是一个新突破
https://arxiv.org/html/2404.19756v1
Abstract
Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes (“neurons”), KANs have learnable activation functions on edges (“weights”). KANs have no linear weights at all – every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful “collaborators” helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today’s deep learning models which rely heavily on MLPs.
https://arxiv.org/html/2404.19756v1
Abstract
Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes (“neurons”), KANs have learnable activation functions on edges (“weights”). KANs have no linear weights at all – every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful “collaborators” helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today’s deep learning models which rely heavily on MLPs.
-
- 著名点评
- 帖子: 3381
- 注册时间: 2022年 7月 26日 16:46
- 昵称(选填): 令狐
#3 Re: AI算法新论文诸位来点评一下
忘了,现在美国还要用苏联人几十年前的老古董。米利坚应该立法禁止使用苏联的科学成果。
Kolmogorov–Arnold representation theorem (or superposition theorem) states that every multivariate continuous function can be represented as a superposition of the two-argument addition of continuous functions of one variable.
Kolmogorov–Arnold representation theorem (or superposition theorem) states that every multivariate continuous function can be represented as a superposition of the two-argument addition of continuous functions of one variable.
-
- 论坛支柱
TheMatrix 的博客 - 帖子: 9820
- 注册时间: 2022年 7月 26日 00:35
#4 Re: AI算法新论文诸位来点评一下
看了。我觉得很不错。论文陈述也非常清楚。StMichael 写了: ↑2024年 5月 1日 18:14 点评一下,是不是一个新突破
https://arxiv.org/html/2404.19756v1
Abstract
Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes (“neurons”), KANs have learnable activation functions on edges (“weights”). KANs have no linear weights at all – every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful “collaborators” helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today’s deep learning models which rely heavily on MLPs.
用spline来拟合曲线,感觉效率是很高的。我觉得可能可以减少back propagation的用量。我觉得如果说人的智能和deep learning的智能有什么区别的话,就在于back propagation。人没有那么多的back propagation。
我看了一下spline,里面很重要的在于control point。几个点定下来了,曲线形状就大致定下来了。而control point的数量,似乎对应论文里的grid。所以论文里面grid变细似乎比网络变深和变宽还管用。
-
- 论坛支柱
Caravel 的博客 - 帖子: 12560
- 注册时间: 2022年 7月 24日 17:21
#5 Re: AI算法新论文诸位来点评一下
做函数拟合这个更有效不奇怪,relu的非线性比较差,但是能不能像MLP那样通用存疑
-
- 论坛精英
- 帖子: 6913
- 注册时间: 2022年 7月 22日 15:29
- 昵称(选填): 海螺子
#6 Re: AI算法新论文诸位来点评一下
这些都是小打小闹,不是什么突破。基本的东西没有什么根本性的变化。
突破的例子,我来给你说一个:active learning,在工作中现学现用,这个深学做不到。其他不能做的事情太多了,所以扯什么AGI, 都是鬼扯,差得也太远了点。
突破的例子,我来给你说一个:active learning,在工作中现学现用,这个深学做不到。其他不能做的事情太多了,所以扯什么AGI, 都是鬼扯,差得也太远了点。
StMichael 写了: ↑2024年 5月 1日 18:14 点评一下,是不是一个新突破
https://arxiv.org/html/2404.19756v1
Abstract
Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes (“neurons”), KANs have learnable activation functions on edges (“weights”). KANs have no linear weights at all – every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful “collaborators” helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today’s deep learning models which rely heavily on MLPs.
-
- 论坛支柱
Caravel 的博客 - 帖子: 12560
- 注册时间: 2022年 7月 24日 17:21
-
- 论坛精英
- 帖子: 6913
- 注册时间: 2022年 7月 22日 15:29
- 昵称(选填): 海螺子
#8 Re: AI算法新论文诸位来点评一下
你去重新train一下试试?LOL.
context window很长,可没有任何learning,weights没有任何变化。
现在这个AI,只有nature,没有nurture。我老是世界上唯一这么说的。
context window很长,可没有任何learning,weights没有任何变化。
现在这个AI,只有nature,没有nurture。我老是世界上唯一这么说的。
上次由 hci 在 2024年 5月 2日 16:40,总共编辑 1 次。
-
- 论坛支柱
Caravel 的博客 - 帖子: 12560
- 注册时间: 2022年 7月 24日 17:21
-
- 论坛精英
- 帖子: 6913
- 注册时间: 2022年 7月 22日 15:29
- 昵称(选填): 海螺子
-
- 论坛精英
- 帖子: 6913
- 注册时间: 2022年 7月 22日 15:29
- 昵称(选填): 海螺子
-
- 论坛支柱
TheMatrix 的博客 - 帖子: 9820
- 注册时间: 2022年 7月 26日 00:35
-
- 论坛精英
- 帖子: 6913
- 注册时间: 2022年 7月 22日 15:29
- 昵称(选填): 海螺子
-
- 论坛支柱
TheMatrix 的博客 - 帖子: 9820
- 注册时间: 2022年 7月 26日 00:35
-
- 著名点评
- 帖子: 4251
- 注册时间: 2022年 9月 17日 19:18
-
- 著名写手
- 帖子: 324
- 注册时间: 2023年 4月 17日 08:26
#17 Re: AI算法新论文诸位来点评一下
Kolmogorov complexity也没啥唬人的,KC因为非常普适,几乎可以囊括一切,所以找不到更具体的理论时,就拿KC解释一下。KC事实上不可计算,而又可以涵盖一切,所以就好像可以解释一样,
上次由 forecasting 在 2024年 5月 6日 18:37,总共编辑 1 次。
-
- 著名点评
- 帖子: 4251
- 注册时间: 2022年 9月 17日 19:18
#18 Re: AI算法新论文诸位来点评一下
一个不可计算,涵盖一切的量、是不是有点玄学的感觉了?
搞研究的人,经常就这么虚晃一枪,然后说我这个是低阶的近似。
搞研究的人,经常就这么虚晃一枪,然后说我这个是低阶的近似。
forecasting 写了: ↑2024年 5月 6日 12:04 Kolmogorov complexity也没啥唬人的,KC因为非常普适,几乎可以囊括一切,所以找不到更具体地理论时,就拿KC解释一下。KC事实上不可计算,而有涵盖一切,所以就好像可以解释一样,
-
- 著名写手
- 帖子: 324
- 注册时间: 2023年 4月 17日 08:26