Add cable (1D rod) assets via Newton#5641
Draft
mmichelis wants to merge 163 commits into
Draft
Conversation
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
…ractive scene) Co-authored-by: Copilot <copilot@github.com>
…pawners, schemas)
…article sync Create isaaclab_experimental/deformable/ module containing: - DeformableObject and DeformableObjectData (Newton backend) - Warp kernels for particle gather/scatter operations - CoupledSolver for rigid-body + VBD cloth interaction - VBDSolverCfg, CoupledSolverCfg, NewtonModelCfg configs - Solver factory functions registered with NewtonManager - Particle sync for USD/Fabric viewport rendering - Cloner hooks for deformable body replication - Model cfg hook for post-finalize parameter application Bug fixes included: - init_pos/init_rot zeroed after Xform bake to prevent double-application - vis_mesh_prim fallback for empty vis_candidates (surface cloth) - No weakref in data class (direct references for particle_q/qd) - model_cfg always applied (not gated behind contact attributes)
Add test_deformable_object.py and test_rigid_deformable_coupling.py under isaaclab_experimental/test/deformable/ with imports adapted for the experimental module layout. Introduce register_hooks() in the deformable __init__ so hooks survive NewtonManager.clear() across test fixtures. Include pre-commit formatting fixes.
Add two deformable object manipulation tasks to isaaclab_tasks_experimental: - Isaac-Pick-Cloth-Direct-v0: Franka robot + cloth (shirt) with coupled MJWarp + VBD solver - Isaac-Pick-VBD-Cube-Direct-v0: Franka robot + deformable cube with coupled MJWarp + VBD solver Both tasks use DeformableNewtonCfg (NewtonCfg subclass with model_cfg field) and import isaaclab_experimental.deformable to trigger hook registration. Also add isaaclab_tasks_experimental import to zero_agent.py for task discovery.
Update the deformable tutorial to support both PhysX and Newton backends via --backend argument. Use core isaaclab.assets imports for backend-agnostic DeformableObject, and import VBDSolverCfg from isaaclab_experimental for the Newton backend path.
Add _get_deformable_ignore_paths() to skip deformable sim/visual mesh prims in builder.add_usd() calls, preventing Newton from creating redundant static mesh colliders for deformable bodies. Also add rebuild_bvh() call in _simulate_physics_only() for solvers that require BVH rebuilds (e.g. VBD cloth).
Gracefully handle particle-only scenes (no rigid bodies) by logging a warning instead of raising RuntimeError when body_label/body_key is empty. Add Fabric particle sync setup for deformable bodies: resize visual mesh topology to match sim mesh, and create per-instance particle offset attributes so the sync kernel can write points.
Defer heavy imports in isaaclab_experimental.deformable behind __getattr__ so pxr is not loaded before Kit starts. Ensure register_hooks() always re-registers after NewtonManager.clear() by resetting the _hooks_registered guard on each explicit call. Fix SurfaceDeformableBodyMaterialCfg class-identity mismatch in from_files.py that caused isinstance() to always fail, making PhysX create volume hierarchies for surface cloth. In deformable_object.py, prefer the original authored Mesh over the PhysX sim_mesh proxy (which has no points at construction time) when a surface deformable material is configured. In newton_replicate.py, pass deformable prim paths as ignore_paths to add_usd so Newton's USD importer skips PhysX proxy meshes.
Replace bare `import isaaclab_experimental.deformable` with explicit `register_hooks()` calls since deferred imports no longer trigger hook registration on import.
The Newton deformable implementation (solvers, kernels, particle sync, cloner hooks) and its tests now live under isaaclab_contrib. All import references in tutorials, task envs, and the isaaclab_newton re-export stub are updated accordingly.
Moves the two deformable task environments from isaaclab_tasks_experimental to isaaclab_tasks so they are discovered by zero_agent.py without needing an extra import of the experimental package.
… code Restore dt=0.01 in SimulationCfg, update env prim comment to explain Newton's env_\d+ naming requirement, restore original reset() ordering, and remove stale commented-out loop.
Remove TetMesh class, spawn_tet_mesh_cuboid, spawn_tet_mesh_from_file, spawn_mesh_from_file, _spawn_tet_mesh_geom_from_tet_mesh and their config counterparts. None are used by any task, test, or tutorial. Also restore Newton docstring in modify_deformable_body_properties.
The isaaclab_newton/assets/deformable_object/__init__.py stub re-exported DeformableObject from isaaclab_contrib, creating a reverse dependency. This is unnecessary since register_hooks() already registers the Newton backend with the DeformableObject factory directly.
Extract the 50-line inline block that sets up Fabric attributes for deformable particle sync from NewtonManager.start_simulation() into setup_fabric_particle_sync() in isaaclab_contrib/deformable/particle_sync. Registered via a new _post_start_simulation_fn hook, keeping isaaclab_newton free of deformable-specific logic.
Cherry-pick Mike's fixes from xiangdonglai#4: - Move visual mesh topology overwrite to define_deformable_body_properties in schemas.py so it happens at USD authoring time, not simulation start. - Add per-instance newton:particleCount Fabric attribute so the sync kernel supports heterogeneous deformable bodies instead of assuming a single particles_per_body for all instances. - Simplify setup_fabric_particle_sync() to only create Fabric attributes.
…e logging - Replace CPU-bound Python for-loop in write_nodal_kinematic_target_to_sim_index with vectorized torch advanced indexing (eliminates per-env GPU-CPU sync). - Log sync_particles_to_usd failures at warning level on first occurrence, then throttle to debug for subsequent failures. - Cache usdrt module import to avoid re-importing every render frame. - Add clarifying comment on particle_q_prev aliasing in two-way coupling friction (known limitation, acceptable for one-way primary use case). - Document _MAX_REACTION_CONTACTS fixed upper bound in coupled_solver.
Signed-off-by: Mike Yan Michelis <46975745+mmichelis@users.noreply.github.com>
…BasisCurve in USD. Currently only supported in Newton backend as add_rod_graph
![isaaclab-review-bot[bot]](https://avatars.githubusercontent.com/u/157846462?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review Summary
This is a massive PR (147 files) that adds experimental 1D cable/rod asset support via Newton physics, along with significant deformable body infrastructure improvements. The PR introduces a new asset type and refactors the deformable system to support multiple physics backends (PhysX and Newton).
High-Level Architecture Assessment
Strengths:
- Clean separation of concerns:
CableObjectinherits fromArticulationsince Newton implements rods as articulations - Registry pattern (
CableRegistryEntry) mirrors the existing deformable registry approach - Backend-neutral
DeformableObjectCfginisaaclab.assetswith backend-specific implementations - Comprehensive documentation with VBD solver guide, cable usage guide, and Newton manager abstraction docs
Key Design Decisions:
- Cables are Newton-only — USD authored on any backend but physics only on Newton
- Cables have no actuators — stiffness is material-like, not controllable
- Self-collision prevention via negative
collision_groupper cable
High-Impact Findings
1. Pre-commit Check Failing (CI)
The pre-commit CI check is failing. This should be resolved before merge.
2. Cable connections Attribute Documentation (Medium)
File: source/isaaclab/isaaclab/sim/spawners/shapes/shapes.py (lines 290-295)
The custom int2[] connections attribute is critical for cable topology but its requirement for user-imported USD files needs clearer documentation. The error message in _register_cable is helpful, but the spawner docstring could emphasize this requirement earlier.
# TODO: Remove in the future once UsdGeomBasisCurves natively supports curve topology.
# For now, the Newton replicate hook expects this attribute to build the rod graph.Suggestion: Add to the CableCfg docstring: "User-imported cable USDs must include a connections attribute (int2[]) listing edge pairs."
3. Hardcoded Demo Paths (Low - Cleanup Needed)
File: scripts/demos/cables.py
The demo references hardcoded absolute paths like /home/mmichelis/Documents/.... These should be removed or parameterized before merge:
# Example of loading a cable from a USD file (needs path cleanup)4. eval_fk Workaround is Well-Documented (Informational)
File: source/isaaclab_contrib/isaaclab_contrib/deformable/vbd_manager.py
The _non_cable_articulation_mask workaround for Newton's missing JointType.CABLE case in eval_fk is properly documented with clear comments indicating it can be removed once Newton patches cable joints. Good defensive coding with RuntimeError if mask building fails.
5. Test Coverage is Good (Positive)
test_spawn_cable.pycovers basic spawning, validation, and material authoringtest_cable_object.pycovers hook idempotency and registry behaviortest_forward_preserves_cable_body_qvalidates the FK workaround
API Design Notes
Breaking Changes (Documented)
MeshSquareCfg→MeshRectangleCfgwithsizechanging fromfloattotuple[float, float]- Deformable cfgs moved from
isaaclab_physxto backend-neutral locations - PhysX deformable cfgs renamed to
Physx*prefix
New Public API Surface
CableCfg,spawn_cableinisaaclab.sim.spawners.shapesCableObject,CableObjectCfginisaaclab_contrib.cableNewtonCableMaterialCfginisaaclab_newton.sim.spawners.materialsVBDSolverCfg,CoupledMJWarpVBDSolverCfg,CoupledFeatherstoneVBDSolverCfginisaaclab_contrib.deformable
Documentation Quality
The documentation is excellent:
using-cables.rst(241 lines) — comprehensive cable guide with parameter tablesusing-vbd-solver.rst(344 lines) — detailed VBD tuning guide with symptom troubleshootingnewton-manager-abstraction.rst(237 lines) — developer guide for adding new solvers- Updated migration guide for deformable backend split
Overall Assessment
This PR represents a well-designed extension of Isaac Lab's physics capabilities. The cable implementation follows established patterns, documentation is thorough, and test coverage is solid. The main blocker is the failing pre-commit check.
Recommendation: Fix the pre-commit failure and remove/parameterize hardcoded demo paths, then this PR is ready for detailed maintainer review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds end-to-end support for cable / rod assets driven by Newton, layered on top of the existing deformable contrib pattern.
isaaclab.sim.spawners.shapes: newCableCfg/spawn_cablethat author cables asUsdGeomBasisCurvesprims. Physics is materialized by the Newton replicate hook, not at spawn time.isaaclab_newton: newNewtonCableMaterialCfg(stretch / bend stiffness, damping, density) and a per-cable ignore-paths block innewton_physics_replicatesoadd_usdskips cableBasisCurves.isaaclab_contrib.cable: newCableObject(subclass of the NewtonArticulation) +CableObjectCfg, plus the replicate-hook plumbing (CableRegistryEntry,add_cable_entry_to_builder,add_registered_cables_to_builder,install_cable_builder_hooks) mirroring the deformable contrib layout.particle_enable_self_contactviagetattr(..., False)so cable-only (zero-particle) scenes no longer hitAttributeError— Newton'sSolverVBDskips_init_particle_systemin that case.forward()to mask cable articulations out ofeval_fk, which has noJointType.CABLEcase and was collapsing rod segments onto their parent anchors on every Kit pre-render FK pass (cable explosion fix). Open Bug in Newton 2887.UsdGeomBasisCurvespoints are refreshed from Newton cable body transforms at render cadence (CPU only for now).scripts/demos/cables.py,test_spawn_cable.py,test_cable_object.py.Builds on top of
mym/deformable_experimental(#5443 ) because cables build on the in-progress Newton deformable contrib stack.Test plan
./isaaclab.sh -p -m pytest source/isaaclab/test/sim/test_spawn_cable.py./isaaclab.sh -p -m pytest source/isaaclab_contrib/test/cable/test_cable_object.py./isaaclab.sh -p scripts/demos/cables.py— visual sanity check in Kit./isaaclab.sh -p scripts/demos/deformables.py— confirm no regression in mixed scenes./isaaclab.sh -fType of change
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there