我补充一下
灾难性遗忘对于端到端的自动驾驶训练确实有影响
但是工业界知道这个问题，如果这个问题都不解决或者规避，那就无法商用
现在采用的方法有很多：
Experience Replay
Task Incremental Learning
Continual Learning
现在中国的智驾问题不是你所说的这些问题
中国的智驾还没到这一步呢，现在大多数的厂家还没有采用AI的端到端的大模型训练
现在国内大多数还是采用规则判断
所以你看到的国内的智驾都是一个城市一个城市的开通，因为采用规则判断需要了解城市的道路情况才能智驾
而一条路如果没有走过，确实智驾水平就下降厉害
但是走过了，智驾水平就会提高
所以对于国内采用规则判断的方法，这样的做法能够快速迭代
正是因为我们目前的智能驾驶采用的是落后与特斯拉的方法
才让我们能够迭代，但这种方法不够普适，所以还很落后
【 在 qtpr 的大作中提到: 】
: 不断增加训练数据并不必然能不断提高深度学习和大模型的整体性能。事实上这类基于梯度的数据驱动模型的可解释性很差，没有人能预见新的训练数据会对模型之前“学到”的信息造成何种潜在影响。有时候，这种影响是剧烈而负面的，即所谓的“灾难性遗忘”。虽然最近大模型在文本
: 斫夂蜕煞矫嫒〉昧嗣飨缘慕剑盎镁酢蔽侍庖廊皇瞧毡榇嬖诘模壳翱床坏接行У母伟旆āＲ欢斡葾I生成的文本是一本正经的胡说八道，也许问题不大。但是，一个由“智驾”控制的汽车操作序列，也是一本正经的胡说八道呢？呵呵。奉劝智驾吹们收敛一些。如果你们想自己当小
: 白鼠，没人阻拦。但不要误导他人。
: An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning arXiv:2312.10549
: Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, Yue Zhang
: Catastrophic forgetting (CF) is a phenomenon that occurs in machine learning when a model forgets previously learned information while acquiring new knowledge. As large language models (LLMs) have demonstrated remarkable performance, it is intriguing to i
: nvestigate whether CF exists during the continual instruction tuning of LLMs. This study empirically evaluates the forgetting phenomenon in LLMs' knowledge during continual instruction tuning from the perspectives of domain knowledge, reasoning, and readi
: ng comprehension. The experiments reveal that catastrophic forgetting is generally observed in LLMs ranging from 1b to 7b parameters. Moreover, as the model scale increases, the severity of forgetting intensifies. Comparing the decoder-only model BLOOMZ w
: ith the encoder-decoder model mT0, BLOOMZ exhibits less forgetting and retains more knowledge. Interestingly, we also observe that LLMs can mitigate language biases, such as gender bias, during continual fine-tuning. Furthermore, our findings indicate tha
: t ALPACA maintains more knowledge and capacity compared to LLAMA during continual fine-tuning, suggesting that general instruction tuning can help alleviate the forgetting phenomenon in LLMs during subsequent fine-tuning processes.
--
FROM 36.152.167.*