Merge branch 'ssh-args' of https://github.com/amenonsen/ansible into amenonsen-ssh-args

CHANGELOG.md

@@ -22,7 +22,12 @@ Major Changes:
 They will retain the value of `None`. To go back to the old behaviour, you can override
 the `null_representation` setting to an empty string in your config file or by setting the
 `ANSIBLE_NULL_REPRESENTATION` environment variable.
-* Use "pattern1,pattern2" to combine host matching patterns. The use of
+* The `ansible_ssh_common_args` inventory variable now provides a
+  convenient way to configure a per-group or per-host ssh ProxyCommand
+  or set any other ssh options. Also, `ansible_ssh_extra_args` can be
+  used to set options that are accepted only by ssh (not sftp or scp,
+  which have their own analogous settings).
+* Use `pattern1,pattern2` to combine host matching patterns. The use of
   ':' as a separator is deprecated (accepted with a warning) because it
   conflicts with IPv6 addresses. The undocumented use of ';' as a
   separator is no longer supported.

docs/man/man1/ansible-playbook.1.asciidoc.in

@@ -151,10 +151,23 @@ run operations with su as this user (default=root)
 
 Run operations with sudo (nopasswd) (deprecated, use become)
 
-*--ssh-extra-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
+*--ssh-common-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
 
-Add the specified arguments to any ssh command-line. Useful to set a
-ProxyCommand to use a jump host, but any arguments may be specified.
+Add the specified arguments to any sftp/scp/ssh command-line. Useful to
+set a ProxyCommand to use a jump host, but any arguments that are
+accepted by all three programs may be specified.
+
+*--sftp-extra-args=*''-f ...''::
+
+Add the specified arguments to any sftp command-line.
+
+*--scp-extra-args=*''-l ...''::
+
+Add the specified arguments to any scp command-line.
+
+*--ssh-extra-args=*''-R ...''::
+
+Add the specified arguments to any ssh command-line.
 
 *-U*, 'SUDO_USER', *--sudo-user=*'SUDO_USER'::
 

docs/man/man1/ansible-pull.1.asciidoc.in

@@ -105,10 +105,23 @@ Purge the checkout after the playbook is run.
 
 Sleep for random interval (between 0 and SLEEP number of seconds) before starting.  This is a useful way ot disperse git requests.
 
-*--ssh-extra-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
+*--ssh-common-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
 
-Add the specified arguments to any ssh command-line. Useful to set a
-ProxyCommand to use a jump host, but any arguments may be specified.
+Add the specified arguments to any sftp/scp/ssh command-line. Useful to
+set a ProxyCommand to use a jump host, but any arguments that are
+accepted by all three programs may be specified.
+
+*--sftp-extra-args=*''-f ...''::
+
+Add the specified arguments to any sftp command-line.
+
+*--scp-extra-args=*''-l ...''::
+
+Add the specified arguments to any scp command-line.
+
+*--ssh-extra-args=*''-R ...''::
+
+Add the specified arguments to any ssh command-line.
 
 *-t* 'TAGS', *--tags=*'TAGS'::
 

docs/man/man1/ansible.1.asciidoc.in

@@ -143,10 +143,23 @@ Run operations with su as this user (default=root)
 
 Run the command as the user given by -u and sudo to root.
 
-*--ssh-extra-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
+*--ssh-common-args=*''-o ProxyCommand="ssh -W %h:%p ..." ...''::
 
-Add the specified arguments to any ssh command-line. Useful to set a
-ProxyCommand to use a jump host, but any arguments may be specified.
+Add the specified arguments to any sftp/scp/ssh command-line. Useful to
+set a ProxyCommand to use a jump host, but any arguments that are
+accepted by all three programs may be specified.
+
+*--sftp-extra-args=*''-f ...''::
+
+Add the specified arguments to any sftp command-line.
+
+*--scp-extra-args=*''-l ...''::
+
+Add the specified arguments to any scp command-line.
+
+*--ssh-extra-args=*''-R ...''::
+
+Add the specified arguments to any ssh command-line.
 
 *-U* 'SUDO_USERNAME', *--sudo-user=*'SUDO_USERNAME'::
 

docsite/rst/faq.rst

@@ -60,23 +60,23 @@ for new users.
 How do I configure a jump host to access servers that I have no direct access to?
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
-With Ansible version 2, it's possible to set `ansible_ssh_extra_args` as
-an inventory variable. Any arguments specified this way are added to the
-ssh command line when connecting to the relevant host(s), so it's a good
-way to set a `ProxyCommand`. Consider the following inventory group:
+With Ansible 2, you can set a `ProxyCommand` in the
+`ansible_ssh_common_args` inventory variable. Any arguments specified in
+this variable are added to the sftp/scp/ssh command line when connecting
+to the relevant host(s). Consider the following inventory group::
 
     [gatewayed]
     foo ansible_host=192.0.2.1
     bar ansible_host=192.0.2.2
 
-You can create `group_vars/gatewayed.yml` with the following contents:
+You can create `group_vars/gatewayed.yml` with the following contents::
 
-    ansible_ssh_extra_args: '-o ProxyCommand="ssh -W %h:%p -q user@gateway.example.com"'
+    ansible_ssh_common_args: '-o ProxyCommand="ssh -W %h:%p -q user@gateway.example.com"'
 
-Ansible will then add these arguments when trying to connect to any host
-in the group `gatewayed`. (These arguments are added to any `ssh_args`
-that may be configured, so it isn't necessary to repeat the default
-`ControlPath` settings in `ansible_ssh_extra_args`.)
+Ansible will append these arguments to the command line when trying to
+connect to any hosts in the group `gatewayed`. (These arguments are used
+in addition to any `ssh_args` from `ansible.cfg`, so you do not need to
+repeat global `ControlPersist` settings in `ansible_ssh_common_args`.)
 
 Note that `ssh -W` is available only with OpenSSH 5.4 or later. With
 older versions, it's necessary to execute `nc %h:%p` or some equivalent

docsite/rst/intro_inventory.rst

@@ -212,11 +212,16 @@ SSH connection::
       The ssh password to use (this is insecure, we strongly recommend using --ask-pass or SSH keys)
     ansible_ssh_private_key_file
       Private key file used by ssh.  Useful if using multiple keys and you don't want to use SSH agent.
-    ansible_ssh_args
-      This setting overrides any ``ssh_args`` configured in ``ansible.cfg``.
+    ansible_ssh_common_args
+      This setting is always appended to the default command line for
+      sftp, scp, and ssh. Useful to configure a ``ProxyCommand`` for a
+      certain host (or group).
+    ansible_sftp_extra_args
+      This setting is always appended to the default sftp command line.
+    ansible_scp_extra_args
+      This setting is always appended to the default scp command line.
     ansible_ssh_extra_args
-      Additional arguments for ssh. Useful to configure a ``ProxyCommand`` for a certain host (or group).
-      This is used in addition to any ``ssh_args`` configured in ``ansible.cfg`` or the inventory.
+      This setting is always appended to the default ssh command line.
     ansible_ssh_pipelining
       Determines whether or not to use SSH pipelining. This can override the
       ``pipelining`` setting in ``ansible.cfg``.

lib/ansible/cli/__init__.py

@@ -314,8 +314,14 @@ class CLI(object):
                 help="connection type to use (default=%s)" % C.DEFAULT_TRANSPORT)
             parser.add_option('-T', '--timeout', default=C.DEFAULT_TIMEOUT, type='int', dest='timeout',
                 help="override the connection timeout in seconds (default=%s)" % C.DEFAULT_TIMEOUT)
+            parser.add_option('--ssh-common-args', default='', dest='ssh_common_args',
+                help="specify common arguments to pass to sftp/scp/ssh (e.g. ProxyCommand)")
+            parser.add_option('--sftp-extra-args', default='', dest='sftp_extra_args',
+                help="specify extra arguments to pass to sftp only (e.g. -f, -l)")
+            parser.add_option('--scp-extra-args', default='', dest='scp_extra_args',
+                help="specify extra arguments to pass to scp only (e.g. -l)")
             parser.add_option('--ssh-extra-args', default='', dest='ssh_extra_args',
-                help="specify extra arguments to pass to ssh (e.g. ProxyCommand)")
+                help="specify extra arguments to pass to ssh only (e.g. -R)")
 
         if async_opts:
             parser.add_option('-P', '--poll', default=C.DEFAULT_POLL_INTERVAL, type='int', dest='poll_interval',

lib/ansible/constants.py

@@ -224,7 +224,7 @@ RETRY_FILES_SAVE_PATH          = get_config(p, DEFAULTS, 'retry_files_save_path'
 DEFAULT_NULL_REPRESENTATION    = get_config(p, DEFAULTS, 'null_representation', 'ANSIBLE_NULL_REPRESENTATION', None, isnone=True)
 
 # CONNECTION RELATED
-ANSIBLE_SSH_ARGS               = get_config(p, 'ssh_connection', 'ssh_args', 'ANSIBLE_SSH_ARGS', None)
+ANSIBLE_SSH_ARGS               = get_config(p, 'ssh_connection', 'ssh_args', 'ANSIBLE_SSH_ARGS', '-o ControlMaster=auto -o ControlPersist=60s')
 ANSIBLE_SSH_CONTROL_PATH       = get_config(p, 'ssh_connection', 'control_path', 'ANSIBLE_SSH_CONTROL_PATH', "%(directory)s/ansible-ssh-%%h-%%p-%%r")
 ANSIBLE_SSH_PIPELINING         = get_config(p, 'ssh_connection', 'pipelining', 'ANSIBLE_SSH_PIPELINING', False, boolean=True)
 ANSIBLE_SSH_RETRIES            = get_config(p, 'ssh_connection', 'retries', 'ANSIBLE_SSH_RETRIES', 0, integer=True)

lib/ansible/playbook/play_context.py

@@ -67,6 +67,10 @@ MAGIC_VARIABLE_MAPPING = dict(
    become_pass      = ('ansible_become_password','ansible_become_pass'),
    become_exe       = ('ansible_become_exe',),
    become_flags     = ('ansible_become_flags',),
+   ssh_common_args  = ('ansible_ssh_common_args',),
+   sftp_extra_args  = ('ansible_sftp_extra_args',),
+   scp_extra_args   = ('ansible_scp_extra_args',),
+   ssh_extra_args   = ('ansible_ssh_extra_args',),
    sudo             = ('ansible_sudo',),
    sudo_user        = ('ansible_sudo_user',),
    sudo_pass        = ('ansible_sudo_password', 'ansible_sudo_pass'),
@@ -140,6 +144,10 @@ class PlayContext(Base):
     _private_key_file = FieldAttribute(isa='string', default=C.DEFAULT_PRIVATE_KEY_FILE)
     _timeout          = FieldAttribute(isa='int', default=C.DEFAULT_TIMEOUT)
     _shell            = FieldAttribute(isa='string')
+    _ssh_args         = FieldAttribute(isa='string', default=C.ANSIBLE_SSH_ARGS)
+    _ssh_common_args  = FieldAttribute(isa='string')
+    _sftp_extra_args  = FieldAttribute(isa='string')
+    _scp_extra_args   = FieldAttribute(isa='string')
     _ssh_extra_args   = FieldAttribute(isa='string')
     _connection_lockfd= FieldAttribute(isa='int')
     _pipelining       = FieldAttribute(isa='bool', default=C.ANSIBLE_SSH_PIPELINING)
@@ -240,6 +248,9 @@ class PlayContext(Base):
 
         self.remote_user = options.remote_user
         self.private_key_file = options.private_key_file
+        self.ssh_common_args = options.ssh_common_args
+        self.sftp_extra_args = options.sftp_extra_args
+        self.scp_extra_args = options.scp_extra_args
         self.ssh_extra_args = options.ssh_extra_args
 
         # privilege escalation

lib/ansible/plugins/connection/ssh.py

@@ -47,15 +47,6 @@ class Connection(ConnectionBase):
         super(Connection, self).__init__(*args, **kwargs)
 
         self.host = self._play_context.remote_addr
-        self.ssh_extra_args = ''
-        self.ssh_args = ''
-
-    def set_host_overrides(self, host):
-        v = host.get_vars()
-        if 'ansible_ssh_extra_args' in v:
-            self.ssh_extra_args = v['ansible_ssh_extra_args']
-        if 'ansible_ssh_args' in v:
-            self.ssh_args = v['ansible_ssh_args']
 
     # The connection is created by running ssh/scp/sftp from the exec_command,
     # put_file, and fetch_file methods, so we don't need to do any connection
@@ -151,8 +142,7 @@ class Connection(ConnectionBase):
         if binary == 'sftp' and C.DEFAULT_SFTP_BATCH_MODE:
             self._command += ['-b', '-']
 
-        elif binary == 'ssh':
-            self._command += ['-C']
+        self._command += ['-C']
 
         if self._play_context.verbosity > 3:
             self._command += ['-vvv']
@@ -160,22 +150,11 @@ class Connection(ConnectionBase):
             # Older versions of ssh (e.g. in RHEL 6) don't accept sftp -q.
             self._command += ['-q']
 
-        # Next, we add ansible_ssh_args from the inventory if it's set, or
-        # [ssh_connection]ssh_args from ansible.cfg, or the default Control*
-        # settings.
+        # Next, we add [ssh_connection]ssh_args from ansible.cfg.
 
-        if self.ssh_args:
-            args = self._split_args(self.ssh_args)
-            self._add_args("inventory set ansible_ssh_args", args)
-        elif C.ANSIBLE_SSH_ARGS:
-            args = self._split_args(C.ANSIBLE_SSH_ARGS)
+        if self._play_context.ssh_args:
+            args = self._split_args(self._play_context.ssh_args)
             self._add_args("ansible.cfg set ssh_args", args)
-        else:
-            args = (
-                "-o", "ControlMaster=auto",
-                "-o", "ControlPersist=60s"
-            )
-            self._add_args("default arguments", args)
 
         # Now we add various arguments controlled by configuration file settings
         # (e.g. host_key_checking) or inventory variables (ansible_ssh_port) or
@@ -189,7 +168,7 @@ class Connection(ConnectionBase):
 
         if self._play_context.port is not None:
             self._add_args(
-                "ANSIBLE_REMOTE_PORT/remote_port/ansible_ssh_port set",
+                "ANSIBLE_REMOTE_PORT/remote_port/ansible_port set",
                 ("-o", "Port={0}".format(self._play_context.port))
             )
 
@@ -212,7 +191,7 @@ class Connection(ConnectionBase):
         user = self._play_context.remote_user
         if user and user != pwd.getpwuid(os.geteuid())[0]:
             self._add_args(
-                "ANSIBLE_REMOTE_USER/remote_user/ansible_ssh_user/user/-u set",
+                "ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set",
                 ("-o", "User={0}".format(self._play_context.remote_user))
             )
 
@@ -221,19 +200,16 @@ class Connection(ConnectionBase):
             ("-o", "ConnectTimeout={0}".format(self._play_context.timeout))
         )
 
-        # If any extra SSH arguments are specified in the inventory for
-        # this host, or specified as an override on the command line,
-        # add them in.
+        # Add in any common or binary-specific arguments from the PlayContext
+        # (i.e. inventory or task settings or overrides on the command line).
 
-        if self._play_context.ssh_extra_args:
-            args = self._split_args(self._play_context.ssh_extra_args)
-            self._add_args("command-line added --ssh-extra-args", args)
-        elif self.ssh_extra_args:
-            args = self._split_args(self.ssh_extra_args)
-            self._add_args("inventory added ansible_ssh_extra_args", args)
+        for opt in ['ssh_common_args', binary + '_extra_args']:
+            attr = getattr(self._play_context, opt, None)
+            if attr is not None:
+                args = self._split_args(attr)
+                self._add_args("PlayContext set %s" % opt, args)
 
-        # Check if ControlPersist is enabled (either by default, or using
-        # ssh_args or ssh_extra_args) and add a ControlPath if one hasn't
+        # Check if ControlPersist is enabled and add a ControlPath if one hasn't
         # already been set.
 
         controlpersist, controlpath = self._persistence_controls(self._command)