-
- {% for video in site.data.video_failure %}
-
-
-
-
-
- {% endfor %}
+
Failure Examples
+
OKAMI's policies may fail to grasp objects due to inaccuracies in the robot controllers, the human reconstruction model or the vision models, or fail to complete tasks because of unwanted collisions, undesired upper body rotations, or inaccuracy in solving inverse kinematics. Here we provide typical failure examples.
+
+
+
+
+
+
Failed to grasp the snack because the robot hand didn't move to a proper position for grasping.
+
+
+
+
+
+
+
Failed to grasp the bottle because the robot hand didn't move to a proper position for grasping.
+
-
OKAMI's policies may fail to grasp objects due to inaccuracies in controllers and the human reconstruction model, or fail to complete tasks because of unwanted collisions, undesired upper body rotations, or inaccuracy in solving inverse kinematics.
+
+
+
+
+
+
Failed to complete the task because of inaccurate inverse kinematics results.
+
+
+
+
+
+
+
Failed to complete the task due to unwanted collisions and unwanted body rotation.
+
+
+
+
+
+
+
+
+
+
+
+
Team
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Citation
+
@inproceedings{okami2024,
+ title={OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitation},
+ author={Jinhan Li and Yifeng Zhu and Xuqi Xie and Zhenyu Jiang and Mingyo Seo and Georgios Pavlakos and Yuke Zhu},
+ booktitle={Conference on Robot Learning (CoRL)},
+ year={2024}
+}
@@ -512,7 +684,7 @@
Real Robot Rollouts
-
Acknowledgement
+
Acknowledgement
We would like to thank William Yue for providing the initial implementation of the behavior cloning policies, Peter Stone for his valuable support with task design and demo shooting, and Yuzhe Qin for sharing the dex-retargeting codebase.
This work has taken place in the Robot Perception and Learning Group (RPL) at UT Austin. RPL research has been partially supported by the National Science Foundation (FRR2145283, EFRI-2318065) and the Office of Naval Research (N00014-22-1-2204).
diff --git a/docs/static/css/index.css b/docs/static/css/index.css
index 4c62080..88fb959 100644
--- a/docs/static/css/index.css
+++ b/docs/static/css/index.css
@@ -334,4 +334,8 @@ padding: 0 40px;
text-align: center;
left: 0;
right: 0;
+}
+
+.has-text-red {
+ color: red;
}
\ No newline at end of file
diff --git a/docs/static/imgs/authors/georgios_pavlakos.jpg b/docs/static/imgs/authors/georgios_pavlakos.jpg
new file mode 100644
index 0000000..6ae58d3
Binary files /dev/null and b/docs/static/imgs/authors/georgios_pavlakos.jpg differ
diff --git a/docs/static/imgs/authors/jinhan.jpg b/docs/static/imgs/authors/jinhan.jpg
new file mode 100644
index 0000000..529bcb3
Binary files /dev/null and b/docs/static/imgs/authors/jinhan.jpg differ
diff --git a/docs/static/imgs/authors/mingyo_seo.png b/docs/static/imgs/authors/mingyo_seo.png
new file mode 100644
index 0000000..018add1
Binary files /dev/null and b/docs/static/imgs/authors/mingyo_seo.png differ
diff --git a/docs/static/imgs/authors/yifeng.jpg b/docs/static/imgs/authors/yifeng.jpg
new file mode 100644
index 0000000..c6fae96
Binary files /dev/null and b/docs/static/imgs/authors/yifeng.jpg differ
diff --git a/docs/static/imgs/authors/yuke_zhu.jpg b/docs/static/imgs/authors/yuke_zhu.jpg
new file mode 100644
index 0000000..8c10bc7
Binary files /dev/null and b/docs/static/imgs/authors/yuke_zhu.jpg differ
diff --git a/docs/static/imgs/authors/yuqi_xie.png b/docs/static/imgs/authors/yuqi_xie.png
new file mode 100644
index 0000000..a78091b
Binary files /dev/null and b/docs/static/imgs/authors/yuqi_xie.png differ
diff --git a/docs/static/imgs/authors/zhenyu_jiang.jpg b/docs/static/imgs/authors/zhenyu_jiang.jpg
new file mode 100644
index 0000000..45e1c2b
Binary files /dev/null and b/docs/static/imgs/authors/zhenyu_jiang.jpg differ
diff --git a/docs/static/videos/bagging/bagging_deploy4.mp4 b/docs/static/videos/bagging/bagging_deploy4.mp4
deleted file mode 100644
index bafaffb..0000000
Binary files a/docs/static/videos/bagging/bagging_deploy4.mp4 and /dev/null differ
diff --git a/docs/static/videos/bagging/deploy1.mp4 b/docs/static/videos/bagging/deploy1.mp4
deleted file mode 100644
index 8142a5e..0000000
Binary files a/docs/static/videos/bagging/deploy1.mp4 and /dev/null differ
diff --git a/docs/static/videos/bagging/deploy2.mp4 b/docs/static/videos/bagging/deploy2.mp4
deleted file mode 100644
index d6eb3bb..0000000
Binary files a/docs/static/videos/bagging/deploy2.mp4 and /dev/null differ
diff --git a/docs/static/videos/bagging/deploy3.mp4 b/docs/static/videos/bagging/deploy3.mp4
deleted file mode 100644
index c6284ad..0000000
Binary files a/docs/static/videos/bagging/deploy3.mp4 and /dev/null differ
diff --git a/docs/static/videos/drawer/close_drawer_black_table.mp4 b/docs/static/videos/drawer/close_drawer_black_table.mp4
deleted file mode 100644
index e3fad98..0000000
Binary files a/docs/static/videos/drawer/close_drawer_black_table.mp4 and /dev/null differ
diff --git a/docs/static/videos/drawer/close_drawer_cabinet.mp4 b/docs/static/videos/drawer/close_drawer_cabinet.mp4
deleted file mode 100644
index a35e879..0000000
Binary files a/docs/static/videos/drawer/close_drawer_cabinet.mp4 and /dev/null differ
diff --git a/docs/static/videos/drawer/close_drawer_green_tablecloth.mp4 b/docs/static/videos/drawer/close_drawer_green_tablecloth.mp4
deleted file mode 100644
index 2f38849..0000000
Binary files a/docs/static/videos/drawer/close_drawer_green_tablecloth.mp4 and /dev/null differ
diff --git a/docs/static/videos/drawer/close_drawer_shield.mp4 b/docs/static/videos/drawer/close_drawer_shield.mp4
deleted file mode 100644
index 54c253b..0000000
Binary files a/docs/static/videos/drawer/close_drawer_shield.mp4 and /dev/null differ
diff --git a/docs/static/videos/failure_videos/fail_to_complete1.mp4 b/docs/static/videos/failure_videos/fail_to_complete1.mp4
index 9d6e49e..aa78251 100644
Binary files a/docs/static/videos/failure_videos/fail_to_complete1.mp4 and b/docs/static/videos/failure_videos/fail_to_complete1.mp4 differ
diff --git a/docs/static/videos/failure_videos/fail_to_grasp_bagging.mp4 b/docs/static/videos/failure_videos/fail_to_grasp_bagging.mp4
index 0b67466..59b797a 100644
Binary files a/docs/static/videos/failure_videos/fail_to_grasp_bagging.mp4 and b/docs/static/videos/failure_videos/fail_to_grasp_bagging.mp4 differ
diff --git a/docs/static/videos/grid_videos/bagging_grid.mp4 b/docs/static/videos/grid_videos/bagging_grid.mp4
new file mode 100644
index 0000000..cffbacd
Binary files /dev/null and b/docs/static/videos/grid_videos/bagging_grid.mp4 differ
diff --git a/docs/static/videos/grid_videos/drawer_grid.mp4 b/docs/static/videos/grid_videos/drawer_grid.mp4
new file mode 100644
index 0000000..952a900
Binary files /dev/null and b/docs/static/videos/grid_videos/drawer_grid.mp4 differ
diff --git a/docs/static/videos/grid_videos/laptop_grid.mp4 b/docs/static/videos/grid_videos/laptop_grid.mp4
new file mode 100644
index 0000000..57a0a13
Binary files /dev/null and b/docs/static/videos/grid_videos/laptop_grid.mp4 differ
diff --git a/docs/static/videos/grid_videos/plush_toy_grid.mp4 b/docs/static/videos/grid_videos/plush_toy_grid.mp4
new file mode 100644
index 0000000..33f2322
Binary files /dev/null and b/docs/static/videos/grid_videos/plush_toy_grid.mp4 differ
diff --git a/docs/static/videos/grid_videos/salt_grid.mp4 b/docs/static/videos/grid_videos/salt_grid.mp4
new file mode 100644
index 0000000..62f426e
Binary files /dev/null and b/docs/static/videos/grid_videos/salt_grid.mp4 differ
diff --git a/docs/static/videos/grid_videos/snack_grid.mp4 b/docs/static/videos/grid_videos/snack_grid.mp4
new file mode 100644
index 0000000..7daf47a
Binary files /dev/null and b/docs/static/videos/grid_videos/snack_grid.mp4 differ
diff --git a/docs/static/videos/human_demo/bagging_human_demo.mp4 b/docs/static/videos/human_demo/bagging_human_demo.mp4
index 726b440..78a1229 100644
Binary files a/docs/static/videos/human_demo/bagging_human_demo.mp4 and b/docs/static/videos/human_demo/bagging_human_demo.mp4 differ
diff --git a/docs/static/videos/laptop/close_laptop_high_table.mp4 b/docs/static/videos/laptop/close_laptop_high_table.mp4
deleted file mode 100644
index b1000d1..0000000
Binary files a/docs/static/videos/laptop/close_laptop_high_table.mp4 and /dev/null differ
diff --git a/docs/static/videos/laptop/close_laptop_low_table.mp4 b/docs/static/videos/laptop/close_laptop_low_table.mp4
deleted file mode 100644
index 1d3bb75..0000000
Binary files a/docs/static/videos/laptop/close_laptop_low_table.mp4 and /dev/null differ
diff --git a/docs/static/videos/laptop/close_laptop_office.mp4 b/docs/static/videos/laptop/close_laptop_office.mp4
deleted file mode 100644
index ad2c187..0000000
Binary files a/docs/static/videos/laptop/close_laptop_office.mp4 and /dev/null differ
diff --git a/docs/static/videos/laptop/close_laptop_outdoor.mp4 b/docs/static/videos/laptop/close_laptop_outdoor.mp4
deleted file mode 100644
index acc7d0a..0000000
Binary files a/docs/static/videos/laptop/close_laptop_outdoor.mp4 and /dev/null differ
diff --git a/docs/static/videos/laptop/close_laptop_table_center.mp4 b/docs/static/videos/laptop/close_laptop_table_center.mp4
deleted file mode 100644
index 306ecfa..0000000
Binary files a/docs/static/videos/laptop/close_laptop_table_center.mp4 and /dev/null differ
diff --git a/docs/static/videos/main.mp4 b/docs/static/videos/main.mp4
index a931dc1..158a39f 100644
Binary files a/docs/static/videos/main.mp4 and b/docs/static/videos/main.mp4 differ
diff --git a/docs/static/videos/method1.mp4 b/docs/static/videos/method1.mp4
new file mode 100644
index 0000000..150469e
Binary files /dev/null and b/docs/static/videos/method1.mp4 differ
diff --git a/docs/static/videos/method2.mp4 b/docs/static/videos/method2.mp4
new file mode 100644
index 0000000..81dfd6e
Binary files /dev/null and b/docs/static/videos/method2.mp4 differ
diff --git a/docs/static/videos/plush_toy/plush_toy_different_toy_1.mp4 b/docs/static/videos/plush_toy/plush_toy_different_toy_1.mp4
deleted file mode 100644
index 8bfa38e..0000000
Binary files a/docs/static/videos/plush_toy/plush_toy_different_toy_1.mp4 and /dev/null differ
diff --git a/docs/static/videos/plush_toy/plush_toy_different_toy_2.mp4 b/docs/static/videos/plush_toy/plush_toy_different_toy_2.mp4
deleted file mode 100644
index 1da6259..0000000
Binary files a/docs/static/videos/plush_toy/plush_toy_different_toy_2.mp4 and /dev/null differ
diff --git a/docs/static/videos/plush_toy/plush_toy_different_toy_3.mp4 b/docs/static/videos/plush_toy/plush_toy_different_toy_3.mp4
deleted file mode 100644
index 3ba6bf0..0000000
Binary files a/docs/static/videos/plush_toy/plush_toy_different_toy_3.mp4 and /dev/null differ
diff --git a/docs/static/videos/plush_toy/plush_toy_outdoor_1.mp4 b/docs/static/videos/plush_toy/plush_toy_outdoor_1.mp4
deleted file mode 100644
index 2ed32ce..0000000
Binary files a/docs/static/videos/plush_toy/plush_toy_outdoor_1.mp4 and /dev/null differ
diff --git a/docs/static/videos/salt/salt_frontview_kitchen.mp4 b/docs/static/videos/salt/salt_frontview_kitchen.mp4
deleted file mode 100644
index 6940c46..0000000
Binary files a/docs/static/videos/salt/salt_frontview_kitchen.mp4 and /dev/null differ
diff --git a/docs/static/videos/salt/salt_kitchen_sink.mp4 b/docs/static/videos/salt/salt_kitchen_sink.mp4
deleted file mode 100644
index 6faf223..0000000
Binary files a/docs/static/videos/salt/salt_kitchen_sink.mp4 and /dev/null differ
diff --git a/docs/static/videos/salt/salt_lab.mp4 b/docs/static/videos/salt/salt_lab.mp4
deleted file mode 100644
index 055eea8..0000000
Binary files a/docs/static/videos/salt/salt_lab.mp4 and /dev/null differ
diff --git a/docs/static/videos/snack/snack_different_snack_and_plate_1.mp4 b/docs/static/videos/snack/snack_different_snack_and_plate_1.mp4
deleted file mode 100644
index c302baf..0000000
Binary files a/docs/static/videos/snack/snack_different_snack_and_plate_1.mp4 and /dev/null differ
diff --git a/docs/static/videos/snack/snack_different_snack_and_plate_2.mp4 b/docs/static/videos/snack/snack_different_snack_and_plate_2.mp4
deleted file mode 100644
index 65a55f5..0000000
Binary files a/docs/static/videos/snack/snack_different_snack_and_plate_2.mp4 and /dev/null differ
diff --git a/docs/static/videos/snack/snack_different_snack_and_plate_3.mp4 b/docs/static/videos/snack/snack_different_snack_and_plate_3.mp4
deleted file mode 100644
index 888c988..0000000
Binary files a/docs/static/videos/snack/snack_different_snack_and_plate_3.mp4 and /dev/null differ
diff --git a/docs/static/videos/snack/snack_different_spatial_relation.mp4 b/docs/static/videos/snack/snack_different_spatial_relation.mp4
deleted file mode 100644
index 0df7888..0000000
Binary files a/docs/static/videos/snack/snack_different_spatial_relation.mp4 and /dev/null differ
diff --git a/docs/static/videos/snack/snack_from_high_table.mp4 b/docs/static/videos/snack/snack_from_high_table.mp4
deleted file mode 100644
index 460d8c9..0000000
Binary files a/docs/static/videos/snack/snack_from_high_table.mp4 and /dev/null differ
diff --git a/docs/static/videos/snack/snack_init_location_from_away.mp4 b/docs/static/videos/snack/snack_init_location_from_away.mp4
deleted file mode 100644
index cbe27c1..0000000
Binary files a/docs/static/videos/snack/snack_init_location_from_away.mp4 and /dev/null differ
diff --git a/docs/static/videos/snack/snack_kitchen.mp4 b/docs/static/videos/snack/snack_kitchen.mp4
deleted file mode 100644
index db0f715..0000000
Binary files a/docs/static/videos/snack/snack_kitchen.mp4 and /dev/null differ
diff --git a/docs/static/videos/visuomotor/bc_bagging.mp4 b/docs/static/videos/visuomotor/bc_bagging.mp4
index 4dc686c..1457824 100644
Binary files a/docs/static/videos/visuomotor/bc_bagging.mp4 and b/docs/static/videos/visuomotor/bc_bagging.mp4 differ