I have the same rig and have had it working great (until tonight when i ran a dist-upgrade ;) )
There's an FT-891 menu setup that you need to configure that i think will fix your transmit audio problem.
Menu 08-09: Data In Select: REAR
That should use the rear data port for audio, yours is probably set to "mic" which is the default i think.
Since mine isn't working at the moment because i'm rebuilding it i'm using my notes - but the audio devices i was using were:
Input: plughw:CARD=udrc,DEV=0 (mono)
Output: plughw:CARD=udrc,DEV=0 (mono)
I'm using the same port you are (next to the GPS port).
I don't know the peak to peak voltage but i'd been running mine with the program defaults and it was working fine, i hadn't been able to spend much time yet fine tuning it.