DragAPart: learning a part-level motion prior for articulated objects
We introduce DragAPart, a method that, given an image and a set of drags as input, generates a new image of the same object that responds to the action of the drags. Differently from prior works that focused on repositioning objects, DragAPart predicts part-level interactions, such as opening and cl...
| Main Authors: | , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Springer
2024
|
| _version_ | 1817932410255310848 |
|---|---|
| author | Li, R Zheng, C Rupprecht, C Vedaldi, A |
| author_facet | Li, R Zheng, C Rupprecht, C Vedaldi, A |
| author_sort | Li, R |
| collection | OXFORD |
| description | We introduce DragAPart, a method that, given an image
and a set of drags as input, generates a new image of the same object
that responds to the action of the drags. Differently from prior works
that focused on repositioning objects, DragAPart predicts part-level interactions, such as opening and closing a drawer. We study this problem
as a proxy for learning a generalist motion model, not restricted to a specific kinematic structure or object category. We start from a pre-trained
image generator and fine-tune it on a new synthetic dataset, Drag-aMove, which we introduce. Combined with a new encoding for the drags
and dataset randomization, the model generalizes well to real images and
different categories. Compared to prior motion-controlled generators, we
demonstrate much better part-level motion understanding. |
| first_indexed | 2024-09-25T04:17:22Z |
| format | Conference item |
| id | oxford-uuid:82953d93-d9f2-495d-89fe-8daf137a2590 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-12-09T03:37:28Z |
| publishDate | 2024 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:82953d93-d9f2-495d-89fe-8daf137a25902024-12-04T10:18:13ZDragAPart: learning a part-level motion prior for articulated objectsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:82953d93-d9f2-495d-89fe-8daf137a2590EnglishSymplectic ElementsSpringer2024Li, RZheng, CRupprecht, CVedaldi, AWe introduce DragAPart, a method that, given an image and a set of drags as input, generates a new image of the same object that responds to the action of the drags. Differently from prior works that focused on repositioning objects, DragAPart predicts part-level interactions, such as opening and closing a drawer. We study this problem as a proxy for learning a generalist motion model, not restricted to a specific kinematic structure or object category. We start from a pre-trained image generator and fine-tune it on a new synthetic dataset, Drag-aMove, which we introduce. Combined with a new encoding for the drags and dataset randomization, the model generalizes well to real images and different categories. Compared to prior motion-controlled generators, we demonstrate much better part-level motion understanding. |
| spellingShingle | Li, R Zheng, C Rupprecht, C Vedaldi, A DragAPart: learning a part-level motion prior for articulated objects |
| title | DragAPart: learning a part-level motion prior for articulated objects |
| title_full | DragAPart: learning a part-level motion prior for articulated objects |
| title_fullStr | DragAPart: learning a part-level motion prior for articulated objects |
| title_full_unstemmed | DragAPart: learning a part-level motion prior for articulated objects |
| title_short | DragAPart: learning a part-level motion prior for articulated objects |
| title_sort | dragapart learning a part level motion prior for articulated objects |
| work_keys_str_mv | AT lir dragapartlearningapartlevelmotionpriorforarticulatedobjects AT zhengc dragapartlearningapartlevelmotionpriorforarticulatedobjects AT rupprechtc dragapartlearningapartlevelmotionpriorforarticulatedobjects AT vedaldia dragapartlearningapartlevelmotionpriorforarticulatedobjects |