I crossed the river using the tunnel between Greenwich and Island Gardens, which brought me out on the Isle of Dogs. Unfortunately the north of the river doesn’t give you as good access to the river as the south. I ended up cycling down some busy roads, and when I tried to get off them I ended up down blind alleys and had to return again.
Instead of crossing the river using the tunnel at Woolwich I decided to get the Woolwich Free Ferry instead. Much nicer. The ferry does a sort of hand brake turn style manouver to get around, dodging the ferry coming in the opposite direction.