You can undo a push, yes. It undoes the local state of the remote branches after the push operation. ie: branch@origin will point at the commit it did before the push action. It doesn't actually update the branch on the remote. That would require another push operation.
jj itself doesn't have a native ‘push’. `jj git push` and `jj piper upload`¹ operate on external systems that don't necessarily support undoing arbitrary operations. Pure speculation, but if I were ESRC I'd make my backend handle it (subject to enterprise audit policies).