Commit 079fa398 by willmcgugan

Lots of docs, some cosmetic changes

parent a9aecf14
fs.contrib fs.contrib
========= ==========
The ``fs.contrib`` module contains a number of filesystem implementations provided by third parties. The ``fs.contrib`` module contains a number of filesystem implementations provided by third parties.
......
...@@ -17,11 +17,13 @@ Guide ...@@ -17,11 +17,13 @@ Guide
introduction.rst introduction.rst
getting_started.rst getting_started.rst
concepts.rst concepts.rst
opening.rst
interface.rst interface.rst
filesystems.rst filesystems.rst
contrib.rst contrib.rst
expose.rst expose.rst
utilities.rst utilities.rst
commands.rst
Code Documentation Code Documentation
------------------ ------------------
......
...@@ -3,11 +3,11 @@ Filesystem Interface ...@@ -3,11 +3,11 @@ Filesystem Interface
It requires a relatively small number of methods to implement a working FS object. It requires a relatively small number of methods to implement a working FS object.
If you are looking to implement a working FS object, derive a class from fs.base.FS and implement the essential methods (below). Be sure to convert all exceptions to instances of :class:`fs.errors.FSError`. If you are looking to implement a working FS object, derive a class from :py:class:`fs.base.FS` and implement the essential methods (below). Be sure to convert all exceptions to instances of :class:`fs.errors.FSError`.
It may also be worthwhile implementing some of the non-essential methods, as the default implementations may not be optimal. For example, the method :meth:`fs.base.FS.move` is implemeneted as a file copy followed by a delete, but many filesystems can move a file without copying data. It may also be worthwhile implementing some of the non-essential methods, as the default implementations may not be optimal. For example, the method :meth:`fs.base.FS.move` is implemented as a file copy followed by a delete, but many filesystems can move a file without copying data.
If the filesystem you are implementing maps paths to the native filesystem, be sure to implement `getsyspath`. Doing so will improve performance, especialy when copying / moving files between FS objects. If the filesystem you are implementing maps paths to the native filesystem, be sure to implement :py:meth:`~fs.base.FS.getsyspath`. Doing so will improve performance, especially when copying / moving files between FS objects.
Essential Methods Essential Methods
----------------- -----------------
...@@ -28,7 +28,7 @@ The following methods are required for a minimal Filesystem interface: ...@@ -28,7 +28,7 @@ The following methods are required for a minimal Filesystem interface:
Non - Essential Methods Non - Essential Methods
----------------------- -----------------------
The following methods have default implementations in fs.base.FS and aren't required for a functional FS interface. They may be overriden if an alternative implementation can be supplied: The following methods have default implementations in :py:class:`fs.base.FS` and aren't required for a functional FS interface. They may be overriden if an alternative implementation can be supplied:
* :meth:`~fs.base.FS.copy` Copy a file to a new location * :meth:`~fs.base.FS.copy` Copy a file to a new location
* :meth:`~fs.base.FS.copydir` Recursively copy a directory to a new location * :meth:`~fs.base.FS.copydir` Recursively copy a directory to a new location
...@@ -53,7 +53,7 @@ The following methods have default implementations in fs.base.FS and aren't requ ...@@ -53,7 +53,7 @@ The following methods have default implementations in fs.base.FS and aren't requ
Utility Methods Utility Methods
--------------- ---------------
The following members have implementations in fs.base.FS and will probably never require a non-default implementation, although there is nothing to prevent a derived class from implementing these: The following members have implementations in :py:class:`fs.base.FS` and will probably never require a non-default implementation, although there is nothing to prevent a derived class from implementing these:
* :meth:`~fs.base.FS.createfile` Create a file with data * :meth:`~fs.base.FS.createfile` Create a file with data
* :meth:`~fs.base.FS.getcontents` Returns the contents of a file as a string * :meth:`~fs.base.FS.getcontents` Returns the contents of a file as a string
......
.. automodule:: fs.opener .. automodule:: fs.opener
:members: :members:
\ No newline at end of file
Opening Filesystems
===================
Generally, when you want to work with the files and directories of any of the supported filesytems,
you create an instance of the appropriate class. For example, the following opens the directory /foo/bar::
from fs.osfs import OSFS
my_fs = OSFS('/foo/bar')
This is fine if you know beforehand where the directory you want to work with is located, and on what medium.
However, there are occasions where the location of the files may change at runtime or should be specified in a config file or from the command line.
In these situations you can use an _opener_ which is a generic way of specifying a filesystem. For example, the following is equivalent to the code above::
from fs.opener import fsopen
my_fs = fsopen('/foo/bar')
The `fsopen` method takes a string that identifies the filesystem, but if called with a regular path, it will return an OSFS instance.
To open a different kind of filesystem, you specify it with a URI like syntax. The following code opens an ftp filesystem rather than a directory on your harddrive::
from fs.opener import fsopen
my_fs = fsopen('ftp://example.org/foo/bar')
For further information regarding filesystem openers see :doc:`opener`.
\ No newline at end of file
...@@ -18,13 +18,11 @@ __all__ = ['DummyLock', ...@@ -18,13 +18,11 @@ __all__ = ['DummyLock',
'flags_to_mode', 'flags_to_mode',
'NoDefaultMeta'] 'NoDefaultMeta']
import os, os.path import os.path
import sys
import shutil import shutil
import fnmatch import fnmatch
import datetime import datetime
import time import time
import re
try: try:
import threading import threading
except ImportError: except ImportError:
...@@ -41,6 +39,7 @@ class DummyLock(object): ...@@ -41,6 +39,7 @@ class DummyLock(object):
directly use the Lock class from the dummy_threading module, since directly use the Lock class from the dummy_threading module, since
it attempts to sanity-check the sequence of acquire/release calls it attempts to sanity-check the sequence of acquire/release calls
in a way that breaks when real threading is available. in a way that breaks when real threading is available.
""" """
def acquire(self,blocking=1): def acquire(self,blocking=1):
...@@ -136,9 +135,8 @@ def synchronize(func): ...@@ -136,9 +135,8 @@ def synchronize(func):
class FS(object): class FS(object):
"""The base class for Filesystem abstraction objects. """The base class for Filesystem abstraction objects.
An instance of a class derived from FS is an abstraction on some kind of filesytem, such as the OS filesystem or a zip file.
An instance of a class derived from FS is an abstraction on some kind
of filesytem, such as the OS filesystem or a zip file.
""" """
_meta = {} _meta = {}
...@@ -150,6 +148,7 @@ class FS(object): ...@@ -150,6 +148,7 @@ class FS(object):
:type thread_synchronize: bool :type thread_synchronize: bool
""" """
super(FS, self).__init__() super(FS, self).__init__()
self.closed = False self.closed = False
self.thread_synchronize = thread_synchronize self.thread_synchronize = thread_synchronize
...@@ -166,8 +165,10 @@ class FS(object): ...@@ -166,8 +165,10 @@ class FS(object):
"""Recommends the use of caching. Implementations are free to use or """Recommends the use of caching. Implementations are free to use or
ignore this value. ignore this value.
:param enabled: If True the implementation is permitted to cache directory :param enabled: If True the implementation is permitted to aggressively cache directory
structure / file info. structure / file information. Caching such information speeds up most operations,
particularly for network based filesystems. The downside of caching is that
changes made to directories or files outside of this interface may not be picked up.
""" """
pass pass
...@@ -175,8 +176,11 @@ class FS(object): ...@@ -175,8 +176,11 @@ class FS(object):
def close(self): def close(self):
"""Close the filesystem. This will perform any shutdown related """Close the filesystem. This will perform any shutdown related
operations required. This method will be called automatically when operations required. This method will be called automatically when
an the filesystem object is cleaned up, but it is a good idea to call the filesystem object is garbage collected, but it is good practice
it explicitly.""" to call it explicitly so that any attached resourced are freed when they
are no longer required.
"""
self.closed = True self.closed = True
def __getstate__(self): def __getstate__(self):
...@@ -255,9 +259,9 @@ class FS(object): ...@@ -255,9 +259,9 @@ class FS(object):
return True return True
def getsyspath(self, path, allow_none=False): def getsyspath(self, path, allow_none=False):
"""Returns the system path (a path recognised by the OS) if present. """Returns the system path (a path recognised by the OS) if one is present.
If the path does not map to a system path (and allow_none is False) If the path does not map to a system path (and `allow_none` is False)
then a NoSysPathError exception is thrown. Otherwise, the system then a NoSysPathError exception is thrown. Otherwise, the system
path will be returned as a unicode string. path will be returned as a unicode string.
...@@ -292,7 +296,7 @@ class FS(object): ...@@ -292,7 +296,7 @@ class FS(object):
:param path: a path within the filesystem :param path: a path within the filesystem
:param allow_none: if true, this method can return None if there is no :param allow_none: if true, this method can return None if there is no
URL form of the given path URL form of the given path
:type allow_none: bool :type allow_none: bool
:raises NoPathURLError: If no URL form exists, and allow_none is False (the default) :raises NoPathURLError: If no URL form exists, and allow_none is False (the default)
:rtype: unicode :rtype: unicode
...@@ -316,25 +320,31 @@ class FS(object): ...@@ -316,25 +320,31 @@ class FS(object):
"""Open a the given path as a file-like object. """Open a the given path as a file-like object.
:param path: a path to file that should be opened :param path: a path to file that should be opened
:param mode: ,ode of file to open, identical to the mode string used :param mode: mode of file to open, identical to the mode string used
in 'file' and 'open' builtins in 'file' and 'open' builtins
:param kwargs: additional (optional) keyword parameters that may :param kwargs: additional (optional) keyword parameters that may
be required to open the file be required to open the file
:rtype: a file-like object :rtype: a file-like object
""" """
raise UnsupportedError("open file") raise UnsupportedError("open file")
def safeopen(self, *args, **kwargs): def safeopen(self, path, mode="r", **kwargs):
"""Like 'open', but returns a NullFile if the file could not be opened. """Like :py:meth:`~fs.base.FS.open`, but returns a :py:class:`~fs.base.NullFile` if the file could not be opened.
A NullFile is a dummy file which has all the methods of a file-like object, A NullFile is a dummy file which has all the methods of a file-like object,
but contains no data. but contains no data.
:rtype: file-like object :param path: a path to file that should be opened
:param mode: mode of file to open, identical to the mode string used
in 'file' and 'open' builtins
:param kwargs: additional (optional) keyword parameters that may
be required to open the file
:rtype: a file-like object
""" """
try: try:
f = self.open(*args, **kwargs) f = self.open(path, mode, **kwargs)
except ResourceNotFoundError: except ResourceNotFoundError:
return NullFile() return NullFile()
return f return f
...@@ -367,7 +377,7 @@ class FS(object): ...@@ -367,7 +377,7 @@ class FS(object):
raise UnsupportedError("check for file") raise UnsupportedError("check for file")
def __iter__(self): def __iter__(self):
""" Iterates over paths returned by listdir method with default params. """ """ Iterates over paths returned by :py:meth:`~fs.base.listdir` method with default params. """
for f in self.listdir(): for f in self.listdir():
yield f yield f
...@@ -395,8 +405,8 @@ class FS(object): ...@@ -395,8 +405,8 @@ class FS(object):
:type files_only: bool :type files_only: bool
:rtype: iterable of paths :rtype: iterable of paths
:raises ResourceNotFoundError: if the path is not found :raises :py:class:`fs.errors.ResourceNotFoundError`: if the path is not found
:raises ResourceInvalidError: if the path exists, but is not a directory :raises :py:class:`fs.errror.ResourceInvalidError`: if the path exists, but is not a directory
""" """
raise UnsupportedError("list directory") raise UnsupportedError("list directory")
...@@ -413,8 +423,9 @@ class FS(object): ...@@ -413,8 +423,9 @@ class FS(object):
the name of each item in the directory, it returns a tuple of the the name of each item in the directory, it returns a tuple of the
name and the info dict as returned by getinfo. name and the info dict as returned by getinfo.
Depending on the filesystem, this may be more efficient than calling This method may be more efficient than calling
getinfo() on each individual item returned by listdir(). :py:meth:`~fs.base.FS.getinfo` on each individual item returned by :py:meth:`~fs.base.FS.listdir`, particularily
for network based filesystems.
:param path: root of the path to list :param path: root of the path to list
:param wildcard: filter paths that match this wildcard :param wildcard: filter paths that match this wildcard
...@@ -423,8 +434,8 @@ class FS(object): ...@@ -423,8 +434,8 @@ class FS(object):
:param files_only: only retrieve files :param files_only: only retrieve files
:type files_only: bool :type files_only: bool
:raises ResourceNotFoundError: If the path is not found :raises :py:class:`fs.errors.ResourceNotFoundError`: If the path is not found
:raises ResourceInvalidError: If the path exists, but is not a directory :raises :py:class:`ResourceInvalidError`: If the path exists, but is not a directory
""" """
path = normpath(path) path = normpath(path)
...@@ -445,7 +456,8 @@ class FS(object): ...@@ -445,7 +456,8 @@ class FS(object):
dirs_only=dirs_only, dirs_only=dirs_only,
files_only=files_only)] files_only=files_only)]
def _listdir_helper(self, path, entries, def _listdir_helper(self, path,
entries,
wildcard=None, wildcard=None,
full=False, full=False,
absolute=False, absolute=False,
...@@ -455,8 +467,9 @@ class FS(object): ...@@ -455,8 +467,9 @@ class FS(object):
Given the path to a directory and a list of the names of entries within Given the path to a directory and a list of the names of entries within
that directory, this method applies the semantics of the listdir() that directory, this method applies the semantics of the listdir()
keyword arguments. An appropriately modified and filtered list of keyword arguments. An appropriately modified and filtered list of
directory entries is returned. directory entries is returned.
""" """
if dirs_only and files_only: if dirs_only and files_only:
raise ValueError("dirs_only and files_only can not both be True") raise ValueError("dirs_only and files_only can not both be True")
...@@ -485,11 +498,12 @@ class FS(object): ...@@ -485,11 +498,12 @@ class FS(object):
absolute=False, absolute=False,
dirs_only=False, dirs_only=False,
files_only=False): files_only=False):
"""Generator yielding the files and directories under a given path. """Iterator yielding the files and directories under a given path.
This method behaves identically to listdir() but returns a generator This method behaves identically to :py:meth:`fs.base.FS.listdir` but returns an iterator
instead of a list. Depending on the filesystem this may be more instead of a list. Depending on the filesystem this may be more
efficient than calling listdir() and iterating over the resulting list. efficient than calling :py:meth:`fs.base.FS.listdir` and iterating over the resulting list.
""" """
return iter(self.listdir(path, return iter(self.listdir(path,
wildcard=wildcard, wildcard=wildcard,
...@@ -504,14 +518,20 @@ class FS(object): ...@@ -504,14 +518,20 @@ class FS(object):
absolute=False, absolute=False,
dirs_only=False, dirs_only=False,
files_only=False): files_only=False):
"""Generator yielding paths and path info under a given path. """Iterator yielding paths and path info under a given path.
This method behaves identically to listdirinfo() but returns a generator This method behaves identically to :py:meth:`~fs.base.listdirinfo` but returns an iterator
instead of a list. Depending on the filesystem this may be more instead of a list. Depending on the filesystem this may be more
efficient than calling listdirinfo() and iterating over the resulting efficient than calling :py:meth:`~fs.base.listdirinfo` and iterating over the resulting
list. list.
""" """
return iter(self.listdirinfo(path,wildcard,full,absolute,dirs_only,files_only)) return iter(self.listdirinfo(path,
wildcard,
full,
absolute,
dirs_only,
files_only))
def makedir(self, path, recursive=False, allow_recreate=False): def makedir(self, path, recursive=False, allow_recreate=False):
"""Make a directory on the filesystem. """Make a directory on the filesystem.
...@@ -521,10 +541,10 @@ class FS(object): ...@@ -521,10 +541,10 @@ class FS(object):
:type recursive: bool :type recursive: bool
:param allow_recreate: if True, re-creating a directory wont be an error :param allow_recreate: if True, re-creating a directory wont be an error
:type allow_create: bool :type allow_create: bool
:raises DestinationExistsError: if the path is already a directory, and allow_recreate is False :raises :py:class:`fs.errors.DestinationExistsError`: if the path is already a directory, and allow_recreate is False
:raises ParentDirectoryMissingError: if a containing directory is missing and recursive is False :raises :py:class:`fs.errors.ParentDirectoryMissingError`: if a containing directory is missing and recursive is False
:raises ResourceInvalidError: if a path is an existing file :raises :py:class:`fs.errors.ResourceInvalidError`: if a path is an existing file
""" """
raise UnsupportedError("make directory") raise UnsupportedError("make directory")
...@@ -534,8 +554,8 @@ class FS(object): ...@@ -534,8 +554,8 @@ class FS(object):
:param path: Path of the resource to remove :param path: Path of the resource to remove
:raises ResourceNotFoundError: if the path does not exist :raises :py:class:`fs.errors.ResourceNotFoundError`: if the path does not exist
:raises ResourceInvalidError: if the path is a directory :raises :py:class:`fs.errors.ResourceInvalidError`: if the path is a directory
""" """
raise UnsupportedError("remove resource") raise UnsupportedError("remove resource")
...@@ -544,14 +564,14 @@ class FS(object): ...@@ -544,14 +564,14 @@ class FS(object):
"""Remove a directory from the filesystem """Remove a directory from the filesystem
:param path: path of the directory to remove :param path: path of the directory to remove
:param recursive: pf True, then empty parent directories will be removed :param recursive: if True, empty parent directories will be removed
:type recursive: bool :type recursive: bool
:param force: if True, any directory contents will be removed :param force: if True, any directory contents will be removed
:type force: bool :type force: bool
:raises ResourceNotFoundError: If the path does not exist :raises :py:class:`fs.errors.ResourceNotFoundError`: if the path does not exist
:raises ResourceInvalidError: If the path is not a directory :raises :py:class:`fs.errors.ResourceInvalidError`: if the path is not a directory
:raises DirectoryNotEmptyError: If the directory is not empty and force is False :raises :py:class:`fs.errors.DirectoryNotEmptyError:` if the directory is not empty and force is False
""" """
raise UnsupportedError("remove directory") raise UnsupportedError("remove directory")
...@@ -561,6 +581,7 @@ class FS(object): ...@@ -561,6 +581,7 @@ class FS(object):
:param src: path to rename :param src: path to rename
:param dst: new name :param dst: new name
""" """
raise UnsupportedError("rename resource") raise UnsupportedError("rename resource")
...@@ -595,15 +616,13 @@ class FS(object): ...@@ -595,15 +616,13 @@ class FS(object):
in info dictionaries for most implementations: in info dictionaries for most implementations:
* "size" - Number of bytes used to store the file or directory * "size" - Number of bytes used to store the file or directory
* "created_time" - A datetime object containing the time the resource * "created_time" - A datetime object containing the time the resource was created
was created * "accessed_time" - A datetime object containing the time the resource was last accessed
* "accessed_time" - A datetime object containing the time the resource * "modified_time" - A datetime object containing the time the resource was modified
was last accessed
* "modified_time" - A datetime object containing the time the resource
was modified
:param path: a path to retrieve information for :param path: a path to retrieve information for
:rtype: dict :rtype: dict
""" """
raise UnsupportedError("get resource info") raise UnsupportedError("get resource info")
...@@ -679,7 +698,7 @@ class FS(object): ...@@ -679,7 +698,7 @@ class FS(object):
error_callback=None): error_callback=None):
"""Create a new file from a string or file-like object asynchronously """Create a new file from a string or file-like object asynchronously
This method returns a threading.Event object. Call the `wait` on the event This method returns a `threading.Event` object. Call the `wait` method on the event object
to block until all data has been written, or simply ignore it. to block until all data has been written, or simply ignore it.
:param path: a path of the file to create :param path: a path of the file to create
...@@ -783,16 +802,17 @@ class FS(object): ...@@ -783,16 +802,17 @@ class FS(object):
:param path: root path to start walking :param path: root path to start walking
:param wildcard: if given, only return files that match this wildcard :param wildcard: if given, only return files that match this wildcard
:type wildcard: A string containing a wildcard (e.g. *.txt) or a callable that takes the file path and returns a boolean :type wildcard: a string containing a wildcard (e.g. `*.txt`) or a callable that takes the file path and returns a boolean
:param dir_wildcard: if given, only walk directories that match the wildcard :param dir_wildcard: if given, only walk directories that match the wildcard
:type dir_wildcard: A string containing a wildcard (e.g. *.txt) or a callable that takes the directory name and returns a boolean :type dir_wildcard: a string containing a wildcard (e.g. `*.txt`) or a callable that takes the directory name and returns a boolean
:param search: -- a string dentifying the method used to walk the directories. There are two such methods: :param search: a string dentifying the method used to walk the directories. There are two such methods:
* 'breadth' Yields paths in the top directories first
* 'depth' Yields the deepest paths first * "breadth" yields paths in the top directories first
* "depth" yields the deepest paths first
:param ignore_errors: ignore any errors reading the directory :param ignore_errors: ignore any errors reading the directory
""" """
if wildcard is None: if wildcard is None:
wildcard = lambda f:True wildcard = lambda f:True
...@@ -864,11 +884,12 @@ class FS(object): ...@@ -864,11 +884,12 @@ class FS(object):
:param path: root path to start walking :param path: root path to start walking
:param wildcard: if given, only return files that match this wildcard :param wildcard: if given, only return files that match this wildcard
:type wildcard: A string containing a wildcard (e.g. *.txt) or a callable that takes the file path and returns a boolean :type wildcard: A string containing a wildcard (e.g. `*.txt`) or a callable that takes the file path and returns a boolean
:param dir_wildcard: if given, only walk directories that match the wildcard :param dir_wildcard: if given, only walk directories that match the wildcard
:type dir_wildcard: A string containing a wildcard (e.g. *.txt) or a callable that takes the directory name and returns a boolean :type dir_wildcard: A string containing a wildcard (e.g. `*.txt`) or a callable that takes the directory name and returns a boolean
:param search: same as walk method :param search: same as walk method
:param ignore_errors: ignore any errors reading the directory :param ignore_errors: ignore any errors reading the directory
""" """
for path, files in self.walk(path, wildcard=wildcard, dir_wildcard=dir_wildcard, search=search, ignore_errors=ignore_errors): for path, files in self.walk(path, wildcard=wildcard, dir_wildcard=dir_wildcard, search=search, ignore_errors=ignore_errors):
for f in files: for f in files:
...@@ -883,11 +904,12 @@ class FS(object): ...@@ -883,11 +904,12 @@ class FS(object):
:param path: root path to start walking :param path: root path to start walking
:param wildcard: if given, only return dictories that match this wildcard :param wildcard: if given, only return dictories that match this wildcard
:type wildcard: A string containing a wildcard (e.g. *.txt) or a callable that takes the directory name and returns a boolean :type wildcard: A string containing a wildcard (e.g. `*.txt`) or a callable that takes the directory name and returns a boolean
:param search: same as the walk method :param search: same as the walk method
:param ignore_errors: ignore any errors reading the directory :param ignore_errors: ignore any errors reading the directory
""" """
for p, files in self.walk(path, dir_wildcard=wildcard, search=search, ignore_errors=ignore_errors): for p, _files in self.walk(path, dir_wildcard=wildcard, search=search, ignore_errors=ignore_errors):
yield p yield p
...@@ -897,6 +919,7 @@ class FS(object): ...@@ -897,6 +919,7 @@ class FS(object):
:param path: a path to the resource :param path: a path to the resource
:rtype: integer :rtype: integer
:returns: the size of the file :returns: the size of the file
""" """
info = self.getinfo(path) info = self.getinfo(path)
size = info.get('size', None) size = info.get('size', None)
...@@ -913,7 +936,8 @@ class FS(object): ...@@ -913,7 +936,8 @@ class FS(object):
be overwritten; If False then DestinationExistsError be overwritten; If False then DestinationExistsError
will be raised. will be raised.
:param chunk_size: size of chunks to use if a simple copy is required :param chunk_size: size of chunks to use if a simple copy is required
(defaults to 16K). (defaults to 64K).
""" """
if not self.isfile(src): if not self.isfile(src):
...@@ -969,6 +993,7 @@ class FS(object): ...@@ -969,6 +993,7 @@ class FS(object):
:param chunk_size: Size of chunks to use when copying, if a simple copy :param chunk_size: Size of chunks to use when copying, if a simple copy
is required is required
:type chunk_size: integer :type chunk_size: integer
""" """
src_syspath = self.getsyspath(src, allow_none=True) src_syspath = self.getsyspath(src, allow_none=True)
...@@ -1002,6 +1027,7 @@ class FS(object): ...@@ -1002,6 +1027,7 @@ class FS(object):
exceptions when moving files exceptions when moving files
:param chunk_size: size of chunks to use when copying, if a simple copy :param chunk_size: size of chunks to use when copying, if a simple copy
is required is required
""" """
if not self.isdir(src): if not self.isdir(src):
if self.isfile(src): if self.isfile(src):
...@@ -1062,6 +1088,7 @@ class FS(object): ...@@ -1062,6 +1088,7 @@ class FS(object):
:type ignore_errors: bool :type ignore_errors: bool
:param chunk_size: size of chunks to use when copying, if a simple copy :param chunk_size: size of chunks to use when copying, if a simple copy
is required (defaults to 16K) is required (defaults to 16K)
""" """
if not self.isdir(src): if not self.isdir(src):
raise ResourceInvalidError(src, msg="Source is not a directory: %(path)s") raise ResourceInvalidError(src, msg="Source is not a directory: %(path)s")
...@@ -1101,6 +1128,7 @@ class FS(object): ...@@ -1101,6 +1128,7 @@ class FS(object):
:param path: a directory path :param path: a directory path
:rtype: bool :rtype: bool
""" """
path = normpath(path) path = normpath(path)
iter_dir = iter(self.listdir(path)) iter_dir = iter(self.listdir(path))
......
...@@ -5,14 +5,14 @@ fs.opener ...@@ -5,14 +5,14 @@ fs.opener
Open filesystems via a URI. Open filesystems via a URI.
There are occasions when you want to specify a filesytem from the command line or in a config file. There are occasions when you want to specify a filesytem from the command line or in a config file.
This module enables that functionality, and can return a FS object given a URI like syntax (http://commons.apache.org/vfs/filesystems.html). This module enables that functionality, and can return an FS object given a URI like syntax (http://commons.apache.org/vfs/filesystems.html).
The `OpenerRegistry` class maps the protocol (file, ftp etc.) on to an Opener object, which returns an appropriate filesystem object and path. The `OpenerRegistry` class maps the protocol (file, ftp etc.) on to an Opener object, which returns an appropriate filesystem object and path.
You can create a custom opener registry that opens just the filesystems you require, or use the opener registry defined here (also called `opener`) that can open any supported filesystem. You can create a custom opener registry that opens just the filesystems you require, or use the opener registry defined here (also called `opener`) that can open any supported filesystem.
The `parse` method of an `OpenerRegsitry` object returns a tuple of an FS object a path. Here's an example of how to use the default opener registry:: The `parse` method of an `OpenerRegsitry` object returns a tuple of an FS object a path. Here's an example of how to use the default opener registry::
>>> from fs.opener import opener >>> from fs.opener import opener
>>> opener.parse('ftp://ftp.mozilla.org') >>> opener.parse('ftp://ftp.mozilla.org/pub')
(<fs.ftpfs.FTPFS object at 0x96e66ec>, u'pub') (<fs.ftpfs.FTPFS object at 0x96e66ec>, u'pub')
You can use use the `opendir` method, which just returns an FS object. In the example above, `opendir` will return a FS object for the directory `pub`:: You can use use the `opendir` method, which just returns an FS object. In the example above, `opendir` will return a FS object for the directory `pub`::
...@@ -26,10 +26,9 @@ If you are just interested in a single file, use the `open` method of a registry ...@@ -26,10 +26,9 @@ If you are just interested in a single file, use the `open` method of a registry
<fs.ftpfs._FTPFile object at 0x973764c> <fs.ftpfs._FTPFile object at 0x973764c>
The `opendir` and `open` methods can also be imported from the top-level of this module for sake of convenience. The `opendir` and `open` methods can also be imported from the top-level of this module for sake of convenience.
To avoid shadowing the builtin `open` methd, they are named `fsopendir` and `fsopen`. Here's how you might import them:: To avoid shadowing the builtin `open` method, they are named `fsopendir` and `fsopen`. Here's how you might import them::
from fs.opener import fsopendir, fsopen from fs.opener import fsopendir, fsopen
""" """
...@@ -55,10 +54,7 @@ __all__ = ['OpenerError', ...@@ -55,10 +54,7 @@ __all__ = ['OpenerError',
'DavOpener', 'DavOpener',
'HTTPOpener'] 'HTTPOpener']
import sys from fs.path import pathsplit, join, iswildcard, normpath
from fs.osfs import OSFS
from fs.path import pathsplit, basename, join, iswildcard, normpath
import os
import os.path import os.path
import re import re
from urlparse import urlparse from urlparse import urlparse
...@@ -104,7 +100,7 @@ def _parse_name(fs_name): ...@@ -104,7 +100,7 @@ def _parse_name(fs_name):
def _split_url_path(url): def _split_url_path(url):
if '://' not in url: if '://' not in url:
url = 'http://' + url url = 'http://' + url
scheme, netloc, path, params, query, fragment = urlparse(url) scheme, netloc, path, _params, _query, _fragment = urlparse(url)
url = '%s://%s' % (scheme, netloc) url = '%s://%s' % (scheme, netloc)
return url, path return url, path
...@@ -172,7 +168,7 @@ class OpenerRegistry(object): ...@@ -172,7 +168,7 @@ class OpenerRegistry(object):
:param fs_url: an FS url :param fs_url: an FS url
:param default_fs_name: the default FS to use if none is indicated (defaults is OSFS) :param default_fs_name: the default FS to use if none is indicated (defaults is OSFS)
:param writeable: if True, a writeable FS will be returned :param writeable: if True, a writeable FS will be returned
:oaram create_dir: if True, then the directory in the FS will be created :param create_dir: if True, then the directory in the FS will be created
""" """
...@@ -341,9 +337,7 @@ class ZipOpener(Opener): ...@@ -341,9 +337,7 @@ class ZipOpener(Opener):
* zip:ftp://ftp.example.org/myzip.zip (open a zip file stored on a ftp server)""" * zip:ftp://ftp.example.org/myzip.zip (open a zip file stored on a ftp server)"""
@classmethod @classmethod
def get_fs(cls, registry, fs_name, fs_name_params, fs_path, writeable, create_dir): def get_fs(cls, registry, fs_name, fs_name_params, fs_path, writeable, create_dir):
append_zip = fs_name_params == 'add'
zip_fs, zip_path = registry.parse(fs_path) zip_fs, zip_path = registry.parse(fs_path)
if zip_path is None: if zip_path is None:
...@@ -357,7 +351,7 @@ class ZipOpener(Opener): ...@@ -357,7 +351,7 @@ class ZipOpener(Opener):
open_mode = 'w+' open_mode = 'w+'
zip_file = zip_fs.open(zip_path, mode=open_mode) zip_file = zip_fs.open(zip_path, mode=open_mode)
username, password, fs_path = _parse_credentials(fs_path) _username, _password, fs_path = _parse_credentials(fs_path)
from fs.zipfs import ZipFS from fs.zipfs import ZipFS
if zip_file is None: if zip_file is None:
...@@ -384,11 +378,11 @@ rpc://www.example.org (opens an RPC server on www.example.org, default port 80)" ...@@ -384,11 +378,11 @@ rpc://www.example.org (opens an RPC server on www.example.org, default port 80)"
@classmethod @classmethod
def get_fs(cls, registry, fs_name, fs_name_params, fs_path, writeable, create_dir): def get_fs(cls, registry, fs_name, fs_name_params, fs_path, writeable, create_dir):
from fs.rpcfs import RPCFS from fs.rpcfs import RPCFS
username, password, fs_path = _parse_credentials(fs_path) _username, _password, fs_path = _parse_credentials(fs_path)
if '://' not in fs_path: if '://' not in fs_path:
fs_path = 'http://' + fs_path fs_path = 'http://' + fs_path
scheme, netloc, path, params, query, fragment = urlparse(fs_path) scheme, netloc, path, _params, _query, _fragment = urlparse(fs_path)
rpcfs = RPCFS('%s://%s' % (scheme, netloc)) rpcfs = RPCFS('%s://%s' % (scheme, netloc))
...@@ -411,10 +405,10 @@ examples: ...@@ -411,10 +405,10 @@ examples:
from fs.ftpfs import FTPFS from fs.ftpfs import FTPFS
username, password, fs_path = _parse_credentials(fs_path) username, password, fs_path = _parse_credentials(fs_path)
scheme, netloc, path, params, query, fragment = urlparse(fs_path) scheme, _netloc, _path, _params, _query, _fragment = urlparse(fs_path)
if not scheme: if not scheme:
fs_path = 'ftp://' + fs_path fs_path = 'ftp://' + fs_path
scheme, netloc, path, params, query, fragment = urlparse(fs_path) scheme, netloc, path, _params, _query, _fragment = urlparse(fs_path)
dirpath, resourcepath = pathsplit(path) dirpath, resourcepath = pathsplit(path)
url = netloc url = netloc
...@@ -519,7 +513,7 @@ example: ...@@ -519,7 +513,7 @@ example:
def get_fs(cls, registry, fs_name, fs_name_params, fs_path, writeable, create_dir): def get_fs(cls, registry, fs_name, fs_name_params, fs_path, writeable, create_dir):
from fs.wrapfs.debugfs import DebugFS from fs.wrapfs.debugfs import DebugFS
if fs_path: if fs_path:
fs, path = registry.parse(fs_path, writeable=writeable, create_dir=create_dir) fs, _path = registry.parse(fs_path, writeable=writeable, create_dir=create_dir)
return DebugFS(fs, verbose=False), None return DebugFS(fs, verbose=False), None
if fs_name_params == 'ram': if fs_name_params == 'ram':
from fs.memoryfs import MemoryFS from fs.memoryfs import MemoryFS
...@@ -590,8 +584,8 @@ class TahoeOpener(Opener): ...@@ -590,8 +584,8 @@ class TahoeOpener(Opener):
fs = TahoeFS(dircap, webapi=url) fs = TahoeFS(dircap, webapi=url)
if '/' in path: if '/' in path:
dirname, resourcename = pathsplit(path) dirname, _resourcename = pathsplit(path)
if createdir: if create_dir:
fs = fs.makeopendir(dirname) fs = fs.makeopendir(dirname)
else: else:
fs = fs.opendir(dirname) fs = fs.opendir(dirname)
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment