Real-Time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction

Shubham Agrawal; Nikhil Chavan-Dafle; Isaac Kasahara; Selim Engin; Jinwook Huh; Volkan Isler

doi:10.1109/IROS55552.2023.10342307

Real-Time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction

Shubham Agrawal, Nikhil Chavan-Dafle, Isaac Kasahara, Selim Engin, Jinwook Huh, Volkan Isler

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper, we present a realtime method for simultaneous object-level scene understanding and grasp prediction. Specifically, given a single RGBD image of a scene, our method localizes all the objects in the scene and for each object, it generates the following: full 3D shape, scale, pose with respect to the camera frame, and a dense set of feasible grasps. The main advantage of our method is its computation speed as it avoids sequential perception and grasp planning. With detailed quantitative analysis of reconstruction quality and grasp accuracy, we show that our method delivers competitive performance compared to the state-of-the-art methods, while providing fast inference at 30 frames per second speed.

Original language	English (US)
Title of host publication	2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	3184-3191
Number of pages	8
ISBN (Electronic)	9781665491907
DOIs	https://doi.org/10.1109/IROS55552.2023.10342307
State	Published - 2023
Externally published	Yes
Event	2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023 - Detroit, United States Duration: Oct 1 2023 → Oct 5 2023

Publication series

Name	IEEE International Conference on Intelligent Robots and Systems
ISSN (Print)	2153-0858
ISSN (Electronic)	2153-0866

Conference

Conference	2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023
Country/Territory	United States
City	Detroit
Period	10/1/23 → 10/5/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Access

10.1109/IROS55552.2023.10342307

OpenUrl availability

Full text

Cite this

Agrawal, S., Chavan-Dafle, N., Kasahara, I., Engin, S., Huh, J., & Isler, V. (2023). Real-Time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023 (pp. 3184-3191). (IEEE International Conference on Intelligent Robots and Systems). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IROS55552.2023.10342307

Real-Time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction. / Agrawal, Shubham; Chavan-Dafle, Nikhil; Kasahara, Isaac et al.
2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 3184-3191 (IEEE International Conference on Intelligent Robots and Systems).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Agrawal, S, Chavan-Dafle, N, Kasahara, I, Engin, S, Huh, J & Isler, V 2023, Real-Time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction. in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023. IEEE International Conference on Intelligent Robots and Systems, Institute of Electrical and Electronics Engineers Inc., pp. 3184-3191, 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023, Detroit, United States, 10/1/23. https://doi.org/10.1109/IROS55552.2023.10342307

Agrawal S, Chavan-Dafle N, Kasahara I, Engin S, Huh J, Isler V. Real-Time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 3184-3191. (IEEE International Conference on Intelligent Robots and Systems). doi: 10.1109/IROS55552.2023.10342307

Agrawal, Shubham ; Chavan-Dafle, Nikhil ; Kasahara, Isaac et al. / Real-Time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction. 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 3184-3191 (IEEE International Conference on Intelligent Robots and Systems).

@inproceedings{c96350f3c9a24cf092061c3c99ad6705,

title = "Real-Time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction",

abstract = "In this paper, we present a realtime method for simultaneous object-level scene understanding and grasp prediction. Specifically, given a single RGBD image of a scene, our method localizes all the objects in the scene and for each object, it generates the following: full 3D shape, scale, pose with respect to the camera frame, and a dense set of feasible grasps. The main advantage of our method is its computation speed as it avoids sequential perception and grasp planning. With detailed quantitative analysis of reconstruction quality and grasp accuracy, we show that our method delivers competitive performance compared to the state-of-the-art methods, while providing fast inference at 30 frames per second speed.",

author = "Shubham Agrawal and Nikhil Chavan-Dafle and Isaac Kasahara and Selim Engin and Jinwook Huh and Volkan Isler",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023 ; Conference date: 01-10-2023 Through 05-10-2023",

year = "2023",

doi = "10.1109/IROS55552.2023.10342307",

language = "English (US)",

series = "IEEE International Conference on Intelligent Robots and Systems",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "3184--3191",

booktitle = "2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023",

}

TY - GEN

T1 - Real-Time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction

AU - Agrawal, Shubham

AU - Chavan-Dafle, Nikhil

AU - Kasahara, Isaac

AU - Engin, Selim

AU - Huh, Jinwook

AU - Isler, Volkan

PY - 2023

Y1 - 2023

N2 - In this paper, we present a realtime method for simultaneous object-level scene understanding and grasp prediction. Specifically, given a single RGBD image of a scene, our method localizes all the objects in the scene and for each object, it generates the following: full 3D shape, scale, pose with respect to the camera frame, and a dense set of feasible grasps. The main advantage of our method is its computation speed as it avoids sequential perception and grasp planning. With detailed quantitative analysis of reconstruction quality and grasp accuracy, we show that our method delivers competitive performance compared to the state-of-the-art methods, while providing fast inference at 30 frames per second speed.

AB - In this paper, we present a realtime method for simultaneous object-level scene understanding and grasp prediction. Specifically, given a single RGBD image of a scene, our method localizes all the objects in the scene and for each object, it generates the following: full 3D shape, scale, pose with respect to the camera frame, and a dense set of feasible grasps. The main advantage of our method is its computation speed as it avoids sequential perception and grasp planning. With detailed quantitative analysis of reconstruction quality and grasp accuracy, we show that our method delivers competitive performance compared to the state-of-the-art methods, while providing fast inference at 30 frames per second speed.

UR - http://www.scopus.com/inward/record.url?scp=85182525849&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85182525849&partnerID=8YFLogxK

U2 - 10.1109/IROS55552.2023.10342307

DO - 10.1109/IROS55552.2023.10342307

M3 - Conference contribution

AN - SCOPUS:85182525849

T3 - IEEE International Conference on Intelligent Robots and Systems

SP - 3184

EP - 3191

BT - 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023

Y2 - 1 October 2023 through 5 October 2023

ER -

Real-Time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction

Abstract

Publication series

Conference

Bibliographical note

Access

OpenUrl availability

Other files and links

Fingerprint

Cite this