From 68c5beb98203a1e5abfe934c9b7a3986f022c17e Mon Sep 17 00:00:00 2001 From: Kevin Lin Date: Sun, 3 Nov 2024 22:50:03 -0600 Subject: [PATCH 1/3] Update demo docs w/ teleop usage info --- docs/demos.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/demos.md b/docs/demos.md index 23d8b5eb0b..a5b2818cfa 100644 --- a/docs/demos.md +++ b/docs/demos.md @@ -123,6 +123,12 @@ The `demo_device_control.py` scripts shows how to teleoperate robot with [contro This current implementation only supports macOS (Linux support can be added). Download and install the [driver](https://www.3dconnexion.com/service/drivers.html) before running the script. +* **Mujoco GUI** + The Mujoco GUI provides a graphical user interface for viewing and interacting with a mujoco simulation. We use the GUI and a mouse to drag and drop mocap bodies, whose + poses are tracked by a controller. More specifically, once the mujoco GUI is loaded from running `python demo_device_control.py`, you first need to hit the key to reach the interactive mujoco viewer state. Then, you should double click on + a mocap body. Finally, to drag the mocap body, you can hit to or key to translate or rotate the mocap body. For Mac users, you need to use `mjpython demo_device_control.py`. See the note from [mujoco](https://mujoco.readthedocs.io/en/stable/python.html#passive-viewer) for more details. + + Additionally, `--pos_sensitivity` and `--rot_sensitivity` provide relative gains for increasing / decreasing the user input device sensitivity. The `--controller` argument determines the choice of using either inverse kinematics controller (`ik`) or operational space controller (`osc`). The main difference is that user inputs with `ik`'s rotations are always taken relative to eef coordinate frame, whereas user inputs with `osc`'s rotations are taken relative to global frame (i.e., static / camera frame of reference). `osc` also tends to be more computationally efficient since `ik` relies on the backend [mink](https://github.com/kevinzakka/mink) IK solver. From 7658cd1e967b9edce5513695848fa9985eb13a93 Mon Sep 17 00:00:00 2001 From: kevin-thankyou-lin <33344633+kevin-thankyou-lin@users.noreply.github.com> Date: Mon, 4 Nov 2024 11:32:46 -0600 Subject: [PATCH 2/3] Update demos.md --- docs/demos.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/docs/demos.md b/docs/demos.md index a5b2818cfa..0d3359e192 100644 --- a/docs/demos.md +++ b/docs/demos.md @@ -130,7 +130,7 @@ The `demo_device_control.py` scripts shows how to teleoperate robot with [contro Additionally, `--pos_sensitivity` and `--rot_sensitivity` provide relative gains for increasing / decreasing the user input -device sensitivity. The `--controller` argument determines the choice of using either inverse kinematics controller (`ik`) or operational space controller (`osc`). The main difference is that user inputs with `ik`'s rotations are always taken relative to eef coordinate frame, whereas user inputs with `osc`'s rotations are taken relative to global frame (i.e., static / camera frame of reference). `osc` also tends to be more computationally efficient since `ik` relies on the backend [mink](https://github.com/kevinzakka/mink) IK solver. +device sensitivity. Furthermore, please choose environment specifics with the following arguments: @@ -166,18 +166,16 @@ Furthermore, please choose environment specifics with the following arguments: Examples: * For normal single-arm environment: ``` -$ python demo_device_control.py --environment PickPlaceCan --robots Sawyer --controller osc +$ python demo_device_control.py --environment PickPlaceCan --robots Sawyer ``` * For two-arm bimanual environment: ``` -$ python demo_device_control.py --environment TwoArmLift --robots Baxter --config bimanual --arm left --controller osc +$ python demo_device_control.py --environment TwoArmLift --robots Baxter --config bimanual --arm left ``` * For two-arm multi single-arm robot environment: ``` -$ python demo_device_control.py --environment TwoArmLift --robots Sawyer Sawyer --config parallel --controller osc +$ python demo_device_control.py --environment TwoArmLift --robots Sawyer Sawyer --config parallel ``` -In **robosuite**, we use this teleoperation script extensively for debugging environment designs, tuning reward functions, and collecting human demonstration data. - ### Video Recording The `demo_video_recording.py` script shows how to record a video of robot roll-out with the `imageio` library. This script uses offscreen rendering. This is useful for generating qualitative videos of robot policy behaviors. The generated video is in the mp4 format. Example: From 85de4a1428a7426be4e88bcb5fc95ddf2464464b Mon Sep 17 00:00:00 2001 From: kevin-thankyou-lin <33344633+kevin-thankyou-lin@users.noreply.github.com> Date: Mon, 4 Nov 2024 15:39:04 -0600 Subject: [PATCH 3/3] Update demos.md --- docs/demos.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/demos.md b/docs/demos.md index 0d3359e192..636cefe938 100644 --- a/docs/demos.md +++ b/docs/demos.md @@ -99,9 +99,7 @@ The `demo_device_control.py` scripts shows how to teleoperate robot with [contro * **Keyboard** We use the keyboard to control the end-effector of the robot. - The keyboard provides 6-DoF control commands through various keys. - The commands are mapped to joint velocities through an inverse kinematics - solver from Bullet physics. + The keyboard provides 6-DoF control commands through various keyboard keys. **Note:** To run this script with macOS, you must run it with root access.