I Suspect that the listener you use runs only on one of the nodes, OR the client side tnsnames.ora points ONLY to one of your listeners. Hence when the node with the listener is down, there really is no listener.
Put the tnsnames.ora as well