Interestingly, in Crunchbang (which uses the same plugin version, but from its own repository) if you do the same thing, the expected result happens: an archive of the folder and contents is created in the same directory. I suppose a patch must have been applied in Crunchbang.
There is an alternative method, which I found on XFCE Docs: add a custom action.
Name: Create tar.gz from folder
Command: tar czvf %n.tar.gz %n
File pattern: *
Appears if selection contains: Directories