On Python 2, shlex.split() raises if you pass it a unicode object with
non-ASCII characters in it.  The Ansible codebase copes by explicitly
converting the string using to_bytes() before passing it to
shlex.split().

On Python 3, shlex.split() raises ('bytes' object has no attribute 'read')
if you pass a bytes object.  Oops.

This commit introduces a new wrapper function, shlex_split, that
transparently performs the to_bytes/to_unicode conversions only on
Python 2.

Currently I've only converted one call site (the one that was causing a
unit test to fail on Python 3).  If this approach is deemed suitable,
I'll convert them all.

Fixes #12488

fixes #12495

Fix typo in checking select results

We used to display input chunks earlier anyway, so this isn't making
things more verbose.

It's possible for more than one fd to be set, so 'elif' is obviously not
the right thing to use.

Fix one more failing test on Python 3

Without this, we could execute «ssh -q ...» and call select(), which
would timeout after the default 10s, and only then send initial data.
(This is a relic of the earlier change where we always ran ssh with
-vvv, so the situation where it would sit quietly never happened in
practice; but this would have been the right thing to do even then.)

Fixes #12478

Fixes #12473

Fixes one failing test.

The long series of module_utils/basic.py fixes were all because
module_utils/basic is imported in ansible/inventory/script.py.

Because the builtin map() acts like an iterator already.

Even Python 2.4 automatically promotes int to long.

NB: we can't use 'from __future__ import print_function', but luckily
print(one_thing) works fine on both Python 2 and Python 3 without that.

Make the code compatible with Pythons 2.4 through 3.5 by using
sys.exc_info()[1] instead.

This is necessary but not sufficient for Python 3 compatibility.

The event loop (even after it was brought into one place in _run in the
previous commit) was hard to follow. The states and transitions weren't
clear or documented, and the privilege escalation code was non-blocking
while the rest was blocking.

Now we have a state machine with four states: awaiting_prompt,
awaiting_escalation, ready_to_send (initial data), and awaiting_exit.
The actions in each state and the transitions between then are clearly
documented.

The check_incorrect_password() method no longer checks for empty strings
(since they will always match), and check_become_success() uses equality
rather than a substring match to avoid thinking an echoed command is an
indication of successful escalation. Also adds a check_missing_password
connection method to detect the error from sudo -n/doas -n.

The main exec_command/put_file/fetch_file methods now _build_command and
call _run to handle input from/output to the ssh process. The purpose is
to bring connection handling together in one place so that the locking
doesn't have to be split across functions.

Note that this doesn't change the privilege escalation and connection IO
code at all—just puts it all into one function.

Most of the changes are just moving code from one place to another (e.g.
from _connect to _build_command, from _exec_command and _communicate to
_run), but there are some other notable changes:

1. We test for the existence of sshpass the first time we need to use
   password authentication, and remember the result.
2. We set _persistent in _build_command if we're using ControlPersist,
   for later use in close(). (The detection could be smarter.)
3. Some apparently inadvertent inconsistencies between put_file and
   fetch_file (e.g. argument quoting, sftp -b use) have been removed.

Also reorders functions into a logical sequence, removes unused imports
and functions, etc.

Aside: the high-level EXEC/PUT/FETCH description should really be logged
from ConnectionBase, while individual subclasses log transport-specific
details.

Fixes #12484

* Make LookupBase an abc with required methods (run()) marked as an
  abstractmethod
* Mark methods that don't use self as @staticmethod
* Document how to implement the run method of a lookup plugin.

Add mention of roles_path parameter to `ansible-galaxy install` command

Fixes #12464

Fix for user defined modules not overriding modules from core.

This fix takes into account that powershell modules are somewhat
different than regular modules and have to be kept separate.