feat: mixed-embodiment simulator, PX4 drones and legged robots in one MuJoCo world - #3819
feat: mixed-embodiment simulator, PX4 drones and legged robots in one MuJoCo world#3819Ez4ezka wants to merge 4 commits into
Conversation
One MuJoCo world drives N PX4 SITL instances over the HIL interface (TCP 4560+i, sim owns PX4's clock via lockstep) alongside M simulated quadrupeds. Includes rotor aerodynamics (spool lag, H-force drag, ground effect, optional OU-gust wind), real Holybro X500 assets fetched on demand (gitignored), a one-command bring-up script with honest readiness gating, and sim-time-scaled PX4 parameters (battery, geofence, datalink-loss failsafe). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per-vehicle Px4DroneModule (namespaced), SwarmCoordinator with layered guardrails (operating radius, separation admission, airborne gate, altitude cap), fleet maneuvers (grid sweep, line formation, investigate, staged air-then-ground sweep), and a fleet-level GCS presence that satisfies PX4's arming checks once NAV_DLL_ACT is set. Safety broadcasts repeat 3x and are verified against reported flight modes with per-robot re-send: a lost rtl_all once left an armed drone hovering while its twin landed, and that incident is now a regression test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LeggedSimModule mirrors the drone module on the shared fleet bus (same state shape, robot_class field) so coordinator queries span embodiments. Real Menagerie Go1 driven by the repo's trained ONNX policy when assets are present, primitive fallback otherwise; heading-held relative move runs inside the simulator at physics rate; ground fence and inter-robot proximity halt. mixed-fleet-mcp and mixed-fleet-agentic compose N drones + M dogs from SIM_DRONES/SIM_DOGS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…gent loop Skills from namespaced module instances qualify as instance/module/skill only when ambiguous; the LLM client shows sanitized aliases (OpenAI rejects '/' in function names, and one bad name 400s the whole request) while calling the server by the real name; the agent message loop survives a failed turn instead of dying silently behind a 'thinking...' UI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
github-actions
Bot
added
the
first-time-contributor
Greptile SummaryThis change introduces mixed PX4 and legged-robot fleet simulation, fleet coordination, namespaced MCP skills, telemetry handling, geospatial guardrails, and fleet safety-command delivery checks. A drone movement request can be accepted within the configured separation distance when another connected, armed drone has telemetry that is only slightly stale. The affected movement commands are Fleet safety-command verification also stops retrying an aircraft when the telemetry field needed to verify delivery is absent, allowing a lost safety broadcast to go without an individually addressed resend. T-Rex validation blockedThe focused fleet safety retry reproduction could not reach the coordinator because the Confidence Score: 3/5Not safe to merge until stale armed peers are handled conservatively during separation checks and incomplete verification telemetry remains eligible for safety-command retries. A focused runtime reproduction confirmed that the real coordinator publishes unsafe movement commands when an armed peer is just beyond the freshness window. The incomplete-telemetry retry behavior remains a clear control-flow concern, but its focused runtime reproduction was blocked by unavailable runtime dependencies. Files Needing Attention:
 T-Rex LogsWhat T-Rex did
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if (needs_mode and st.get("mode") is None) or ( | ||
| not needs_mode and st.get("armed") is None | ||
| ): | ||
| logger.warning( | ||
| f"[verify:{action}] {key}: telemetry lacks the field needed to " | ||
| "verify delivery -- cannot confirm" | ||
| ) | ||
| continue |
There was a problem hiding this comment.
When a fleet safety broadcast is lost while an aircraft lacks the telemetry field needed for verification, this branch logs and continues without retaining the aircraft in still. It is therefore removed from pending and receives no individually addressed resend. A lost RTL, land, hold, emergency-land, or kill command can leave that aircraft armed and flying.
| globals_ = self._globals(max_age_s=GUARDRAIL_MAX_AGE_SEC) | ||
| issuing_global = globals_.get(issuing_key) | ||
| issuing_local = self._local_ned(issuing_key) | ||
| # Fail closed. Without a *fresh* position for the issuing vehicle this | ||
| # check cannot be performed, and "cannot verify" is not "safe" -- the | ||
| # previous behaviour let the command through and hoped PX4 would nack | ||
| # it. Telemetry arrives within a second, so refusing costs a retry. | ||
| if issuing_global is None or issuing_local is None: | ||
| return ( | ||
| f"no position for {issuing_key} fresher than " | ||
| f"{GUARDRAIL_MAX_AGE_SEC:.2f}s — cannot verify separation" | ||
| ) | ||
| dn = target_n - issuing_local[0] | ||
| de = target_e - issuing_local[1] | ||
| pred_lat, pred_lon = offset_latlon(issuing_global[0], issuing_global[1], dn, de) | ||
| pred_world = (pred_lat, pred_lon, -target_d) # NED down -> altitude up | ||
| for key, point in globals_.items(): | ||
| if key == issuing_key: | ||
| continue | ||
| dist = distance_3d_m(pred_world, point) | ||
| if dist < self.config.min_separation_m: | ||
| return f"{key} @ {dist:.2f}m" | ||
| return None |
There was a problem hiding this comment.
_violates_separation obtains the comparison set through _globals(max_age_s=GUARDRAIL_MAX_AGE_SEC), which omits a connected, armed peer when its position is slightly older than 0.75 seconds. The issuing drone can still have fresh telemetry and pass the fail-closed check, so goto_drone, goto_drone_global, and the per-drone assignment in investigate can accept a destination inside the configured separation floor. Reject the request while any other connected armed drone lacks a fresh global position instead of treating that peer as absent.
Artifacts
Focused stale-peer admission reproduction source
- Authored Python harness calls the real coordinator command methods with an armed peer whose telemetry is 0.76 seconds old and whose position is inside the destination separation floor; it exercises the claimed admission path.
Stale-peer command-admission runtime output
- Captured output of the focused harness run from `/home/user/repo` with exit code 0; it shows all three command paths admitted an unsafe drone1 waypoint, confirming the claim.
aclauer
left a comment
aclauer
left a comment
There was a problem hiding this comment.
Plz no AI spam pr descriptions
There was a problem hiding this comment.
we now currently have robot asset manager, can checck https://docs.dimensional.org/capabilities/manipulation/adding_a_custom_arm/#4a-add-your-urdf
3 participants
What this is
One MuJoCo world that flies N quadrotors running real, unmodified PX4 firmware next to M simulated Unitree Go1 quadrupeds, all commanded through namespaced DimOS modules, a fleet coordinator, and MCP tools (plus an optional natural-language agent). It replaces the previous Gazebo drone path, which could not fly three drones in realtime on a laptop.
Someone with a laptop and this branch can be flying a mixed fleet in about 15 minutes: see the Quickstart on a fresh machine section at the top of
dimos/simulation/px4_hil/README.md. Every dependency tier degrades gracefully (no meshes -> primitive airframes, no trained policy -> hand-tuned gait), so nothing blocks a first run.How it works
HIL_SENSOR/HIL_GPSand owns PX4's clock, so a slow machine runs the world slower instead of starving the EKF. Runs 5 to 18x realtime depending on fleet size and machine.SIM_WIND_N/E,SIM_GUST_STD). Verified: position hold within ~0.1 m in 3 m/s wind with gusts.tools/fetch_x500_meshes.py(BSD-3, from PX4-gazebo-models) and gitignored.Timeouts and clocks
Every PX4 timeout parameter runs on the sim clock (10 to 18x wall speed here), while DimOS-side loops tick in wall time.
sim_params.pyscalesCOM_DL_LOSS_T,COM_OF_LOSS_T,SIM_BAT_DRAINaccordingly and documents why; robot-side control loops (the quadruped's move controller) run inside the bridge at physics rate for the same reason.Verification
sweep_then_ground), daemon-kill autonomous RTL, wind hold, collision-guard halt between converging dogs.ruff checkclean on every touched file.Known limits (documented in the README)
🤖 Generated with Claude Code