Remote Access connects for about 15 seconds then disconnects

Huh, well it seems that it may be a modem thing, this article seems to have solved it for my Motorola SBG6580

Set external IP to 0.0.0.0 in the port forward