I'm not sure if this has changed in a more recent release of PhysX but previously there was not yet a cylinder prim object for the GPU pipeline, which defaulted to an n-gon (I believe it was 18 but could be configured up to 64). As a result directly simulating wheels for RL was not physically realistic. Instead, the supported workflow was to simulate the robot as the end effector of a virtual arm parallel to the ground, and then use the action space for that virtual arm (pose deviation, velocity, acceleration, whatever you prefer) to control the physical robot with some low level pose/velocity/etc. controller. The idea was that for most indoor environment "locomotion" is not the difficult task that must be learned with RL, but rather (potentially perceptive) navigation, as well as navigation with medium-to-long horizon manipulation. I'm not sure if an official example of this workflow exists, and of course if the supported workflow has changed please someone correct me.

The PhysX team is working on proper wheel simulation. Once that's ready, we'll add some concrete examples on wheel-based mobile robots. Stay tuned :)