Error connecting to Always on Listener


I'm looping over the CMS and evaluating each server with no problems except when it hits a SQL A.O. Listener. When it hits SOME Listeners I get the following error:
Microsoft.SqlServer.Management.Common.ConnectionFailureException, Failed to connect to server .

Is there a way to increase the amount of time that is used to connect? I have noticed similar behavior using SSMS to connect to listeners and found that if I change the connection timeout from the default of 15 seconds to 30 seconds it solves the problem.

It would be much easier to not evaluate the listener and evaluate the servers used in the cluster but we use the CMS tree for different scripts and one of them requires the listener instead of the server so that is how we have it set up. I'd prefer not to have to run multiple instances of CMS if possible.

If you can assist it would be a great help! I am running version 4.12