update
This commit is contained in:
MarioTX
parent
09a5c0fe3f
commit
58353d8ff9
128
README.md
128
README.md
|
|
@ -6,42 +6,55 @@
|
|||
> Embodied AI（具身智能）入门的路径以及高质量信息的总结，期望是按照路线走完后，新手可以快速建立关于这个领域的认知，希望能帮助到各位入门具身智能的朋友，欢迎点Star、分享与提PR🌟~<br>【 <a href="https://github.com/tianxingchen/Embodied-AI-Guide">Embodied-AI-Guide</a>, Latest Update: Dec 29, 2024 】<img src="https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2Ftianxingchen%2FEmbodied-AI-Guide&count_bg=%232B8DD9&title_bg=%237834C6&icon=github.svg&icon_color=%23E7E7E7&title=Page+Viewers&edge_flat=false"/> <img alt="GitHub repo stars" src="https://img.shields.io/github/stars/TianxingChen/Embodied-AI-Guide">
|
||||
|
||||
## Contents - 目录
|
||||
|
||||
<nav>
|
||||
<ul>
|
||||
<li><a href="#start">Start Up - 从这里开始</a></li>
|
||||
<li><a href="#info">Useful Info - 有利于搭建认知的资料</a></li>
|
||||
<li><a href="#foundation-models">Foundation Models - 基础模型</a></li>
|
||||
<li><a href="#common-tools">Common Tools - 常用工具</a></li>
|
||||
<li><a href="#robot-learning">Robot Learning - 机器人学习</a>
|
||||
<li><a href="#start">1. Start Up - 从这里开始</a></li>
|
||||
<li><a href="#info">2. Useful Info - 有利于搭建认知的资料</a></li>
|
||||
<li><a href="#algorithm">3. Algorithm - 算法</a>
|
||||
<ul>
|
||||
<li><a href="#rl">Reinforcement Learning - 强化学习</a></li>
|
||||
<li><a href="#il">Imitation Learning - 模仿学习</a></li>
|
||||
<li><a href="#common-tools">3.1 Common Tools - 常用工具</a></li>
|
||||
<li><a href="#foundation-models">3.2 Foundation Models - 基础模型</a></li>
|
||||
<li><a href="#robot-learning">3.3 Robot Learning - 机器人学习</a>
|
||||
<ul>
|
||||
<li><a href="#rl">3.3.1 Reinforcement Learning - 强化学习</a></li>
|
||||
<li><a href="#il">3.3.2 Imitation Learning - 模仿学习</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><a href="#llm_robot">3.4 LLM for Robotics - 大模型在机器人学中的应用</a></li>
|
||||
<li><a href="#cv">3.5 Computer Vision - 计算机视觉</a>
|
||||
<ul>
|
||||
<li><a href="#3dv">3.5.1 3D Vision - 三维视觉</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><a href="#embodied-ai-4-x">3.6 Embodied AI for X - 具身智能+X</a>
|
||||
<ul>
|
||||
<li>Embodied AI for Healthcare - 具身智能+医疗</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><a href="#llm_robot">LLM for Robotics - 大模型在机器人学中的应用</a></li>
|
||||
<li><a href="#medical">MLLM for Medical - 多模态大语言模型在医学中的应用</a></li>
|
||||
<li><a href="#cv">Computer Vision - 计算机视觉</a>
|
||||
<li><a href="#hardware">4. Hardware - 硬件</a>
|
||||
<ul>
|
||||
<li><a href="#3dv">3D Vision - 三维视觉</a></li>
|
||||
<li><a href="#control">4.1 Control - 控制学</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><a href="#control">Control - 控制学</a></li>
|
||||
<li><a href="#benchmarks">Benchmarks & Simulators - 基准 & 仿真器</a></li>
|
||||
<li><a href="#embodied-ai-4-x">Embodied AI for X - 具身智能+X</a>
|
||||
<li><a href="#software">5. Software - 软件</a>
|
||||
<ul>
|
||||
<li>Embodied AI for Healthcare - 具身智能+医疗</li>
|
||||
<li><a href="#benchmarks">5.1 Benchmarks & Simulators - 基准 & 仿真器</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><a href="#paper_list">Paper Lists - 论文列表</a></li>
|
||||
<li><a href="#communities">Communities - 社区</a></li>
|
||||
<li><a href="#companies">Companies - 公司</a></li>
|
||||
<li><a href="#acknowledgement">Acknowledgement - 致谢</a></li>
|
||||
<li><a href="#paper_list">6. Paper Lists - 论文列表</a></li>
|
||||
<li><a href="#communities">7. Communities - 社区</a></li>
|
||||
<li><a href="#companies">8. Companies - 公司</a></li>
|
||||
<li><a href="#acknowledgement">9. Acknowledgement - 致谢</a></li>
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
|
||||
<section id="start"></section>
|
||||
|
||||
## Start Up - 从这里开始
|
||||
## 1. Start Up - 从这里开始
|
||||
|
||||
> 具身智能是指一种基于物理身体进行感知和行动的智能系统，其通过智能体与环境的交互获取信息、理解问题、做出决策并实现行动，从而产生智能行为和适应性。
|
||||
|
||||
|
|
@ -59,7 +72,7 @@
|
|||
|
||||
<section id="info"></section>
|
||||
|
||||
## Useful Info - 有利于搭建认知的资料
|
||||
## 2. Useful Info - 有利于搭建认知的资料
|
||||
|
||||
* 具身智能基础技术路线-YunlongDong [2]: [PDF](./files/具身智能基础技术路线-YunlongDong.pdf), [bilibili](https://www.bilibili.com/video/BV1d5ukedEsi/?buvid=XXCD799C01878A6CFDECF3FB4427E2F070877&from_spmid=default-value&is_story_h5=false&mid=iWFclAyh36UYMh2G6ZcsDw%3D%3D&p=1&plat_id=114&share_from=ugc&share_medium=android&share_plat=android&share_session_id=9c0dccf5-ec0b-4369-8b89-ff1d848467ee&share_source=WEIXIN&share_tag=s_i&spmid=united.player-video-detail.0.0×tamp=1716466406&unique_k=Q0CaIUj&up_id=249218043)
|
||||
|
||||
|
|
@ -67,9 +80,22 @@
|
|||
|
||||
* Robotics实验室总结 [4]: [zhihu_1](https://zhuanlan.zhihu.com/p/682671294?utm_psn=1782122763157188608), [zhihu_2](https://zhuanlan.zhihu.com/p/682692024?utm_psn=1782122945184796672)
|
||||
|
||||
|
||||
<section id="algorithm"></section>
|
||||
|
||||
## 3. Algorithm - 算法
|
||||
|
||||
<section id="common-tools"></section>
|
||||
|
||||
### 3.1 Common Tools - 常用工具
|
||||
|
||||
> 这个部分是关于具身中常用技巧的分享
|
||||
|
||||
* 点云降采样: [zhihu](https://zhuanlan.zhihu.com/p/558683732?utm_campaign=shareopn&utm_medium=social&utm_psn=1772067996070236160&utm_source=wechat_session), 包括随机降采样、均匀降采样、最远点降采样、法线空间降采样等，需要了解清楚每一种降采样的优劣，这个技巧的选择对于3D应用来说是至关重要的。
|
||||
|
||||
<section id="foundation-models"></section>
|
||||
|
||||
## Foundation Models - 基础模型
|
||||
### 3.2 Foundation Models - 基础模型
|
||||
|
||||
> 以下是部分具身智能中常用的基础模型, 计算机视觉中发展的非常好的工具可以直接赋能具身智能的下游应用。
|
||||
|
||||
|
|
@ -92,25 +118,16 @@
|
|||
* Depth Anything (v1 & v2): [repo](https://github.com/LiheYoung/Depth-Anything), [repo](https://github.com/DepthAnything/Depth-Anything-V2), 港大和字节的研究工作，单目深度估计模型
|
||||
|
||||
* Point Transformer (v3): [repo](https://github.com/Pointcept/PointTransformerV3), 点云特征提取的工作
|
||||
|
||||
<section id="common-tools"></section>
|
||||
|
||||
## Common Tools - 常用工具
|
||||
|
||||
> 这个部分是关于具身中常用技巧的分享
|
||||
|
||||
* 点云降采样: [zhihu](https://zhuanlan.zhihu.com/p/558683732?utm_campaign=shareopn&utm_medium=social&utm_psn=1772067996070236160&utm_source=wechat_session), 包括随机降采样、均匀降采样、最远点降采样、法线空间降采样等，需要了解清楚每一种降采样的优劣，这个技巧的选择对于3D应用来说是至关重要的。
|
||||
|
||||
<section id="robot-learning"></section>
|
||||
|
||||
## Robot Learning - 机器人学习
|
||||
### 3.3 Robot Learning - 机器人学习
|
||||
|
||||
机器人学习 Robot Learning 的发展: [zhihu](https://zhuanlan.zhihu.com/p/26988866)
|
||||
|
||||
|
||||
<section id="rl"></section>
|
||||
|
||||
### Reinforcement Learning - 强化学习
|
||||
#### 3.3.1 Reinforcement Learning - 强化学习
|
||||
* 推荐直接跟着李宏毅老师一套走: bilibili上课+刷蘑菇书巩固+gymnasium动手实践, 重点了解一下PPO。
|
||||
* 台湾大学李宏毅公开课: [bilibili](https://www.bilibili.com/video/BV1XP4y1d7Bk/?spm_id_from=333.337.search-card.all.click&vd_source=ab9cf5374617c2867aaea34af29b53c9)<br>
|
||||
* EasyRL - 蘑菇书: [website](https://datawhalechina.github.io/easy-rl/#/), 基本是配套李宏毅老师的课程<br>
|
||||
|
|
@ -120,23 +137,22 @@
|
|||
|
||||
<section id="il"></section>
|
||||
|
||||
### Imitation Learning - 模仿学习
|
||||
#### 3.3.2 Imitation Learning - 模仿学习
|
||||
|
||||
* 模仿学习简洁教程 - 南京大学LAMDA: [PDF](https://www.lamda.nju.edu.cn/xut/Imitation_Learning.pdf)<br>
|
||||
* Supervised Policy Learning for Real Robots, RSS 2024 Workshop 教程：真实机器人的监督策略学习, [bilibili](https://www.bilibili.com/video/BV1Fx4y1s7if/?buvid=XY415384A771A6C681C9BEB3817566ED57724&is_story_h5=false&mid=ORgXkVzTHaOKTsml0RX5Gw%3D%3D&plat_id=240&share_from=ugc&share_medium=android&share_plat=android&share_source=WEIXIN&share_tag=s_i&spmid=dt.space-dt.0.0×tamp=1721464513&unique_k=Cqj5d9J&up_id=2185804&vd_source=ab9cf5374617c2867aaea34af29b53c9)
|
||||
|
||||
* 实践[RoboTwin]()
|
||||
<!-- * 实践[RoboTwin]() -->
|
||||
|
||||
<section id="llm_robot"></section>
|
||||
|
||||
## LLM for Robotics - 大模型在机器人学中的应用
|
||||
### 3.4 LLM for Robotics - 大模型在机器人学中的应用
|
||||
* Robotics+LLM系列通过大语言模型控制机器人 [2]: [zhihu](https://zhuanlan.zhihu.com/p/668053911)<br>
|
||||
* PDDL-wiki: [website](https://planning.wiki/)<br>
|
||||
* An Introduction to PDDL: [PDF](https://www.cs.toronto.edu/~sheila/2542/s14/A1/introtopddl2.pdf)<br>
|
||||
* Embodied Agent wiki: [website](https://en.wikipedia.org/wiki/Embodied_agent)<br>
|
||||
* Lilian Weng 个人博客 - AI Agent 系统综述 [5]: 中文: [website](https://mp.weixin.qq.com/s/Jb8HBbaKYXXxTSQOBsP5Wg) 英文: [website](https://lilianweng.github.io/posts/2023-06-23-agent/)<br>
|
||||
|
||||
|
||||
<section id="medical"></section>
|
||||
|
||||
## MLLM for Medical - 多模态大语言模型在医学中的应用
|
||||
|
|
@ -148,14 +164,14 @@
|
|||
|
||||
<section id="cv"></section>
|
||||
|
||||
## Computer Vision - 计算机视觉
|
||||
### 3.5 Computer Vision - 计算机视觉
|
||||
|
||||
计算机视觉课程: [website](https://cs231n.stanford.edu/schedule.html)<br>
|
||||
该课程对深度学习在计算机视觉的应用有较为全面的介绍。因为已经在具体实现某个论文的算法了，所以这个阶段可以不用做作业，只需要看课程视频和课程讲义即可。<br>
|
||||
|
||||
<section id="3dv"></section>
|
||||
|
||||
### 3D Vision - 三维视觉
|
||||
#### 3.5.1 3D Vision - 三维视觉
|
||||
第一阶段：学习最基础的3DV知识，追求广度，了解一些基础的概念和算法<br>
|
||||
* 三维视觉导论 - Andreas Geiger: [website](https://uni-tuebingen.de/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/autonomous-vision/lectures/computer-vision/) （重点是完成课程里面的作业） <br>
|
||||
* GAMES203 - 三维重建和理解: [bilibili](https://www.bilibili.com/video/BV1pw411d7aS/?share_source=copy_web&vd_source=0b7603f37af6d369a97df34525b149be)<br>
|
||||
|
|
@ -174,9 +190,21 @@
|
|||
* 3DGS在具身上的综述
|
||||
* 3D Gaussian Splatting in Robotics: A Survey. [PDF](https://arxiv.org/pdf/2410.12262v2)<br>
|
||||
|
||||
|
||||
<section id="embodied-ai-4-x"></section>
|
||||
|
||||
### 3.6 Embodied AI for X - 具身智能+X
|
||||
|
||||
#### 3.6.1 Embodied AI for Healthcare - 具身智能+医疗
|
||||
Coming Soon...
|
||||
|
||||
<section id="hardware"></section>
|
||||
|
||||
## 4. Hardware - 硬件
|
||||
|
||||
<section id="control"></section>
|
||||
|
||||
## Control - 控制学
|
||||
### 4.1 Control - 控制学
|
||||
|
||||
> 关于控制部分的学习，最好从实践出发！
|
||||
|
||||
|
|
@ -192,10 +220,13 @@
|
|||
* ROS多传感器时间戳同步：[website](https://blog.csdn.net/qq_43495930/article/details/125649446)
|
||||
|
||||
|
||||
<section id="software"></section>
|
||||
|
||||
## 5. Software - 软件
|
||||
|
||||
<section id="benchmarks"></section>
|
||||
|
||||
## Benchmarks & Simulators - 基准 & 仿真器
|
||||
### 5.1 Benchmarks & Simulators - 基准 & 仿真器
|
||||
具身智能常用benchmark总结 [1]: [zhihu](https://zhuanlan.zhihu.com/p/695342864)<br>
|
||||
常见仿真器wiki: [wiki](https://simulately.wiki/)
|
||||
| 仿真器 | 基准 |
|
||||
|
|
@ -207,16 +238,9 @@
|
|||
| [PyBullet](https://pybullet.org/wordpress/) | [Calvin](https://github.com/mees/calvin?tab=readme-ov-file)<br>[Ravens](https://github.com/google-research/ravens)<br>[VimaBench](https://github.com/vimalabs/VimaBench) |
|
||||
| [Genesis](https://genesis-embodied-ai.github.io/) ||
|
||||
|
||||
<section id="embodied-ai-4-x"></section>
|
||||
|
||||
## Embodied AI for X - 具身智能+X
|
||||
|
||||
### Embodied AI for Healthcare - 具身智能+医疗
|
||||
Coming Soon...
|
||||
|
||||
<section id="paper_list"></section>
|
||||
|
||||
## Paper Lists - 论文列表
|
||||
## 6. Paper Lists - 论文列表
|
||||
|
||||
* Awesome Humanoid Robot Learning - Yanjie Ze: [repo](https://github.com/YanjieZe/awesome-humanoid-robot-learning)
|
||||
* Paper Reading List - DeepTimber Community: [repo](https://github.com/DeepTimber-Robot-Lab/Paper-Reading-List)
|
||||
|
|
@ -227,7 +251,7 @@ Coming Soon...
|
|||
|
||||
<section id="communities"></section>
|
||||
|
||||
## Communities - 社区
|
||||
## 7. Communities - 社区
|
||||
> 以下部分资料引用自[7]
|
||||
|
||||
* DeepTimber Robotics Innovations Community, 深木科研交流社区: [website](https://gamma.app/public/DeepTimber-Robotics-Innovations-Community-A-Community-for-Multi-m-og0uv8mswl1a3q7?mode=doc)
|
||||
|
|
@ -245,7 +269,7 @@ Coming Soon...
|
|||
|
||||
<section id="companies"></section>
|
||||
|
||||
## Companies - 公司
|
||||
## 8. Companies - 公司
|
||||
|
||||
| 公司 | 主营产品 | Others |
|
||||
|-------|------|------|
|
||||
|
|
@ -265,7 +289,7 @@ Coming Soon...
|
|||
|
||||
<a name="acknowledgement"></a>
|
||||
|
||||
## 🙏 Acknowledgement - 致谢
|
||||
## 9. Acknowledgement - 致谢
|
||||
本文转载/引用了一些博主的文章，我们对他们的知识分享表示感谢，引用列表如下：
|
||||
[1] 知乎 [穆尧](https://www.zhihu.com/people/mu-yao-12-34), [2] 知乎 [东林钟声](https://www.zhihu.com/people/dong-lin-zhong-sheng-76), Github [Yunlong Dong](https://github.com/yunlongdong), [3] 知乎 [强化学徒](https://www.zhihu.com/people/heda-he-28), [4] 知乎 [Biang哥](https://www.zhihu.com/people/qi-da-guang), [5] OpenAI [Lilian Weng](https://lilianweng.github.io/), [6] B站 [木木具身](https://space.bilibili.com/350563565), [7] Github [Zhuoheng Li](https://github.com/StarCycle/EmbodiedAI-Reading-List-For-Lists?tab=readme-ov-file), [8] 知乎 [Flood Sung](https://www.zhihu.com/people/flood-sung), [9] Github [Sida Peng](https://github.com/pengsida/learning_research)
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue