Right, as promised here are the files for the USB HID joystick. The source code only makes sense for my schematic so adjust as you see fit. As always, if you use this code, please do at least give credit where credit is due. It's been a long battle trying to understand both USB on the PIC32MZ and understanding how USB works in general.

Schematic

Here's a low quality render to give some approximation of what the board looks like in real life

OK, enough of that, Here's the source code

And here are the KiCad files if you need them.