Documentation, fixes, A ReadOnlyFS wrapper and a plain old FTP FS class

ChangeLog

@@ -23,4 +23,7 @@
                   with a remote filesystem
     * fs.errors:  updated exception hierarchy, with support for converting
                   to/from standard OSError instances
+    * New FS implementation:
+        * FTPFS:        access a plain old FTP server
+        * ReadOnlyFS:   a WrapFS that makes an fs read-only
 

docs/base.rst

+fs.base
+=======
+
+This module contains the basic FS interface and a number of other essential interfaces.
+
+fs.base.FS
+----------
+
+All Filesystem objects inherit from this class
+
 .. autoclass:: fs.base.FS
     :members:
-    
\ No newline at end of file
+    
+fs.base.SubFS
+-------------
+
+A SubFS is an FS implementation that represents a directory on another Filesystem. When you use the `opendir` method it will return a SubFS instance. You should not need to instantiate a SubFS directly.
+
+For example::
+
+    from fs.osfs import OSFS
+    home_fs = OSFS('foo')
+    bar_gs = home_fs.opendir('bar')
+
+
+fs.base.NullFile
+----------------
+
+A NullFile is a file-like object with no functionality. It is used in situations where a file-like object is required but the caller doesn't have any data to read or write.
+
+The `safeopen` method returns an NullFile instance, which can reduce error-handling code.
+
+For example, the following code may be written to append some text to a log file::
+
+    logfile = None
+    try:
+        logfile = myfs.open('log.txt', 'a')
+        logfile.writeline('operation successful!')
+    finally:
+        if logfile is not None:
+            logfile.close()
+
+This could be re-written using the `safeopen` method::
+
+    myfs.safeopen('log.txt', 'a').writeline('operation successful!')
+
+If the file doesn't exist then the call to writeline will be a null-operation (i.e. not do anything)
+

docs/browsewin.rst

+.. automodule:: fs.browsewin
+    :members:
\ No newline at end of file

docs/concepts.rst

+Concepts
+========
+
+It is generally quite easy to get in to the mind-set of using PyFilesystem interface over lower level interfaces -- since the code tends to be simpler -- but there are a few concepts which you will need to keep in mind.
+
+Sandboxing
+----------
+
+FS objects are not permitted to work with any files / directories outside of the Filesystem they represent. If you attempt to open a file / directory outside the root of the FS (by using "../" in the path, you will get a ValueError).
+
+It is advisable to write functions that takes FS objects as parameters, possibly with an additional path relative to the root of the FS. This allows you to write code that works with files / directories, but is independant of where they are located.
+
+
+Paths
+-----
+
+Paths used within a Filesystem object use the same common format, regardless of the underlaying Filesystem it represents.
+
+ * Path components are separated by a forward path (/)
+ * Paths beginning with a forward slash are absolute (start at the root of the FS)
+ * Paths not beginning with a forward slash are relative
+ * A single dot means 'current directory'
+ * A double dot means 'previous directory'
+ 
+Note that paths used by the FS interface will use this format, but the constructor or additional methods may not. Notably the osfs.OSFS constructor which requires an OS path.
+

docs/errors.rst

+fs.errors
+=========
+
+.. automodule:: fs.errors
+    :members:
\ No newline at end of file

docs/filesystems.rst

+Filesystems
+===========
+
+This page lists the builtin filesystems.
+
+
+fs.osfs.OSFS
+------------
+An interface to the OS Filesystem
+
+fs.memoryfs.MemoryFS
+--------------------
+A filesystem that exists entirely in memory
+
+fs.mountfs.MountFS
+------------------
+A filesystem that can map directories in to other filesystems (like a symlink)
+
+fs.multifs.MultiFS
+------------------
+A filesystem that overlays other filesystems
+
+fs.sftpfs.SFTPFS
+----------------
+A secure FTP filesystem
+
+fs.s3fs.S3FS
+------------
+A filesystem to access an Amazon S3 service
+
+fs.tempfs.TempFS
+----------------
+Creates a temporary filesystem in an OS provided location
+
+fs.zipfs.ZipFS
+--------------
+A filesystem for zip files
+
+

docs/getting_started.rst

@@ -16,7 +16,7 @@ This will install the latest stable release. If you would prefer to install the
     cd pyfilesystem-read-only
     python setup.py install
 
-You should now have the _fs_ module on your path:
+You should now have the `fs` module on your path:
 
     >>> import fs
     >>> fs.__version__

docs/index.rst

@@ -6,13 +6,39 @@
 Welcome to PyFilesystem's documentation!
 ========================================
 
-Contents:
+PyFilesystem provides a simplified common interface to a variety of different filesystems, such as the local filesystem, zip files, ftp servers etc.
+
+Guide
+-----
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 3
 
+   concepts.rst
    getting_started.rst
+   interface.rst
+   filesystems.rst
+   
+Code Documentation
+------------------
+   
+.. toctree::
+   :maxdepth: 3
+   
    base.rst
+   browsewin.rst
+   errors.rst
+   memoryfs.rst
+   mountfs.rst
+   osfs.rst
+   path.rst
+   s3fs.rst
+   sftpfs.rst
+   tempfs.rst
+   utils.rst
+   wrapfs.rst
+   zipfs.rst
+   
 
 Indices and tables
 ==================

docs/interface.rst

+Filesystem Interface
+====================
+
+It requires a relatively small number of methods to implement a working Filesystem object.
+
+
+Essential Methods
+-----------------
+
+The following methods are required for a minimal Filesystem interface:
+
+    * `open` Opens a file for read/writing
+    * `isfile` Check wether the path exists and is a file
+    * `isdir` Check wether a path exists and is a directory
+    * `listdir` List the contents of a directory
+    * `makedir` Create a new directory
+    * `remove` Remove an existing file
+    * `removedir` Remove an existing directory
+    * `rename` Automically rename a file or directory
+    * `getinfo` Return information about the path e.h. size, mtime
+    
+
+Non - Essential Methods
+-----------------------
+
+The following methods have default implementations in fs.base.FS and aren't required for a functional FS interface. They may be overriden if an alternative implementation can be supplied:
+
+    * `copy` Copy a file to a new location
+    * `copydir` Recursively copy a directory to a new location
+    * `desc` Return a short destriptive text regarding a path
+    * `exists` Check whether a path exists as file or directory    
+    * `getsyspath` Get a file's name in the local filesystem, if possible
+    * `hassyspath` Check if a path maps to a system path (recognised by the OS)
+    * `move` Move a file to a new location        
+    * `movedir` Recursively move a directory to a new location
+    * `opendir` Opens a directory and returns an FS object that represents it
+    * `safeopen` Like `open` but returns a NullFile if the file could not be opened
+
+
+Utility Methods
+---------------
+
+The following members have implementations in fs.base.FS but will probably never need a non-default implementation, although there is nothing to prevent a derived class from implementing these:
+
+    * `createfile` Create a file with data
+    * `getcontents` Returns the contents of a file as a string
+    * `getsize` Returns the number of bytes used for a given file or directory
+    * `isdirempty` Checks if a directory contains no files
+    * `makeopendir` Creates a directroy (if it exists) and returns an FS object for that directory
+    * `walk` Like `listdir` but descends in to sub-directories
+    * `walkfiles` Returns an iterable of file paths in a directory, and its sub-directories
+    * `walkdirs` Returns an iterable of paths to sub-directories
\ No newline at end of file

docs/memoryfs.rst

+.. automodule:: fs.memoryfs
+    :members:
\ No newline at end of file

docs/mountfs.rst

+.. automodule:: fs.mountfs
+    :members:
\ No newline at end of file

docs/osfs.rst

+.. automodule:: fs.osfs
+    :members:
\ No newline at end of file

docs/path.rst

+fs.path
+=========
+
+.. automodule:: fs.path
+    :members:
\ No newline at end of file

docs/s3fs.rst

+.. automodule:: fs.s3fs
+    :members:
\ No newline at end of file

docs/sftpfs.rst

+.. automodule:: fs.sftpfs
+    :members:
\ No newline at end of file

docs/tempfs.rst

+.. automodule:: fs.tempfs
+    :members:
\ No newline at end of file

docs/utils.rst

+fs.utils
+========
+
+.. automodule:: fs.utils
+    :members:
\ No newline at end of file

docs/wrapfs.rst

+.. automodule:: fs.wrapfs
+    :members:
\ No newline at end of file

docs/zipfs.rst

+.. automodule:: fs.zipfs
+    :members:
\ No newline at end of file

fs/__init__.py

@@ -26,4 +26,22 @@ from base import *
 import errors
 import path
 
-
+_thread_syncronize_default = True
+def set_thread_syncronize_default(sync):
+    """Sets the default thread synctonisation flag.
+    
+    FS objects are made thread-safe through the use of a per-FS threading Lock
+    object. Since this can introduce an small overhead it can be disabled with
+    this function if the code is single-threaded.
+    
+    :param sync: Set wether to use thread syncronization for new FS objects
+    
+    """
+    global _thread_syncronization_default
+    _thread_syncronization_default = sync
+
+# Store some identifiers in the fs namespace
+import os
+SEEK_CUR = os.SEEK_CUR
+SEEK_END = os.SEEK_END
+SEEK_SET = os.SEEK_SET
\ No newline at end of file

fs/base.py

@@ -45,9 +45,9 @@ class DummyLock:
 def silence_fserrors(f, *args, **kwargs):
     """Perform a function call and return None if FSError is thrown
 
-    f -- Function to call
-    args -- Parameters to f
-    kwargs -- Keyword parameters to f
+    :param f: Function to call
+    :param args: Parameters to f
+    :param kwargs: Keyword parameters to f
 
     """
     try:
@@ -103,16 +103,20 @@ class NullFile(object):
     def writelines(self, *args, **kwargs):
         pass
 
-
+try:
+    from functools import wraps
+except ImportError:
+    wraps = lambda f:f
+    
 def synchronize(func):
     """Decorator to synchronize a method on self._lock."""
+    @wraps(func)
     def acquire_lock(self, *args, **kwargs):
         self._lock.acquire()
         try:
             return func(self, *args, **kwargs)
         finally:
             self._lock.release()
-    acquire_lock.__doc__ = func.__doc__
     return acquire_lock
 
 
@@ -122,30 +126,6 @@ class FS(object):
     An instance of a class derived from FS is an abstraction on some kind
     of filesytem, such as the OS filesystem or a zip file.
 
-    The following is the minimal set of methods that must be provided by
-    a new FS subclass:
-
-        * open -- open a file for reading/writing (like python's open() func)
-        * isfile -- check whether a path exists and is a file
-        * isdir -- check whether a path exists and is a directory
-        * listdir -- list the contents of a directory
-        * makedir -- create a new directory
-        * remove -- remove an existing file
-        * removedir -- remove an existing directory
-        * rename -- atomically rename a file or directory
-        * getinfo -- return information about the path e.g. size, mtime
-
-    The following methods have a sensible default implementation, but FS
-    subclasses are welcome to override them if a more efficient implementation
-    can be provided:
-
-        * getsyspath -- get a file's name in the local filesystem, if possible
-        * exists -- check whether a path exists as file or directory
-        * copy -- copy a file to a new location
-        * move -- move a file to a new location
-        * copydir -- recursively copy a directory to a new location
-        * movedir -- recursively move a directory to a new location
-
     """
 
     def __init__(self, thread_synchronize=False):
@@ -161,7 +141,7 @@ class FS(object):
             self._lock = DummyLock()
 
     def __del__(self):
-        if not self.closed:
+        if not getattr(self, 'closed', True):
             self.close()
 
     def close(self):        
@@ -308,6 +288,7 @@ class FS(object):
 
         """
         raise UnsupportedError("list directory")
+        
 
     def _listdir_helper(self, path, entries,
                               wildcard=None,
@@ -411,7 +392,7 @@ class FS(object):
         This is mainly for use as a debugging aid.
         """
         if not self.exists(path):
-            return "No description available"
+            return ''
         try:
             sys_path = self.getsyspath(path)
         except NoSysPathError:
@@ -616,6 +597,8 @@ class FS(object):
         :param overwrite: If True, then an existing file at the destination path
             will be silently overwritten; if False then an exception
             will be raised in this case.
+        :param chunk_size: Size of chunks to use when copying, if a simple copy
+            is required
         """
 
         src_syspath = self.getsyspath(src, allow_none=True)
@@ -724,6 +707,9 @@ class FS(object):
         src = abspath(src)
         dst = abspath(dst)
 
+        if not overwrite and self.exists(dst):
+            raise DestinationExistsError(dst)
+
         if dst:
             self.makedir(dst, allow_recreate=overwrite)
 
@@ -786,13 +772,13 @@ class SubFS(FS):
 
     def __str__(self):
         return "<SubFS: %s in %s>" % (self.sub_dir, self.parent)
+        
+    def __unicode__(self):
+        return u"<SubFS: %s in %s>" % (self.sub_dir, self.parent)
 
     def __repr__(self):
         return str(self)
 
-    def __unicode__(self):
-        return unicode(self.__str__())
-
     def desc(self, path):
         if self.isdir(path):
             return "Sub dir of %s"%str(self.parent)

fs/browsewin.py

 #!/usr/bin/env python
 """
+fs.browsewin
+============
+
 Creates a window which can be used to browse the contents of a filesystem.
 To use, call the 'browse' method with a filesystem object. Double click a file
 or directory to display its properties.
@@ -32,7 +35,7 @@ class InfoFrame(wx.Frame):
         self.list_ctrl.SetColumnWidth(1, 300)
 
         for key in keys:
-            self.list_ctrl.Append((key, repr(info[key])))
+            self.list_ctrl.Append((key, repr(info.get(key))))
 
 
 
@@ -76,7 +79,6 @@ class BrowseFrame(wx.Frame):
         self.Bind(wx.EVT_TREE_ITEM_EXPANDING, self.OnItemExpanding)
         self.Bind(wx.EVT_TREE_ITEM_ACTIVATED, self.OnItemActivated)
 
-
         wx.CallAfter(self.OnInit)
 
     def OnInit(self):
@@ -97,7 +99,7 @@ class BrowseFrame(wx.Frame):
             return
 
         paths = [(self.fs.isdir(p), p) for p in self.fs.listdir(path, absolute=True)]
-
+        
         if not paths:
             #self.tree.SetItemHasChildren(item_id, False)
             #self.tree.Collapse(item_id)
@@ -163,7 +165,7 @@ def browse(fs):
     """Displays a window containing a tree control that displays an FS
     object. Double-click a file/folder to display extra info.
 
-    fs -- A filesystem object
+    :param fs: A filesystem object
 
     """
 

fs/errors.py

 """
+Defines the Exception classes thrown by PyFilesystem objects. Exceptions relating to the underling filesystem are translated in to one of the following Exceptions. Exceptions that relate to a path store that path in `self.path`.
 
-  fs.errors:  error class definitions for FS
+All Exception classes are derived from `FSError` which can be used as a catch-all exception.
 
 """
 
@@ -39,15 +40,15 @@ class FSError(Exception):
         return str(self.msg % keys)
 
     def __unicode__(self):
-        keys = dict((k,v) for k,v in self.__dict__.iteritems())
-        return unicode(self.msg) % keys
+        return unicode(self.msg) % self.__dict__
 
     def __getstate__(self):
        return self.__dict__.copy()
 
 
 class PathError(FSError):
-    """Exception for errors to do with a path string."""
+    """Exception for errors to do with a path string.
+    """
     default_message = "Path is invalid: %(path)s"
 
     def __init__(self,path="",**kwds):
@@ -73,7 +74,7 @@ class UnsupportedError(OperationFailedError):
 
 class RemoteConnectionError(OperationFailedError):
     """Exception raised when operations encounter remote connection trouble."""
-    default_message = "Unable to %(opname)s: remote connection errror"
+    default_message = "%(opname)s: remote connection errror"
 
 
 class StorageSpaceError(OperationFailedError):
@@ -93,7 +94,6 @@ class OperationTimeoutError(OperationFailedError):
     default_message = "Unable to %(opname)s: operation timed out"
 
 
-
 class ResourceError(FSError):
     """Base exception class for error associated with a specific resource."""
     default_message = "Unspecified resource error: %(path)s"

fs/expose/django_storage.py

 """
+fs.expose.django
+================
 
-  fs.expose.django:  use an FS object for Django File Storage
+Use an FS object for Django File Storage
 
 """
 

fs/expose/fuse/__init__.py

 """
+fs.expose.fuse
+==============
 
-  fs.expose.fuse:  expose an FS object to the native filesystem via FUSE
+Expose an FS object to the native filesystem via FUSE
 
 This module provides the necessary interfaces to mount an FS object into
-the local filesystem via FUSE:
+the local filesystem via FUSE::
 
     http://fuse.sourceforge.net/
 
 For simple usage, the function 'mount' takes an FS object and a local path,
-and exposes the given FS at that path:
+and exposes the given FS at that path::
 
     >>> from fs.memoryfs import MemoryFS
     >>> from fs.expose import fuse
@@ -20,7 +22,7 @@ and exposes the given FS at that path:
 
 The above spawns a new background process to manage the FUSE event loop, which
 can be controlled through the returned subprocess.Popen object.  To avoid
-spawning a new process, set the 'foreground' option:
+spawning a new process, set the 'foreground' option::
 
     >>> #  This will block until the filesystem is unmounted
     >>> fuse.mount(fs,"/mnt/my-memory-fs",foreground=True)
@@ -30,7 +32,7 @@ to the 'mount' function.
 
 If you require finer control over the creation of the FUSE process, you can
 instantiate the MountProcess class directly.  It accepts all options available
-to subprocess.Popen:
+to subprocess.Popen::
 
     >>> from subprocess import PIPE
     >>> mp = fuse.MountProcess(fs,"/mnt/my-memory-fs",stderr=PIPE)
@@ -419,8 +421,8 @@ def mount(fs,path,foreground=False,ready_callback=None,unmount_callback=None,**k
     keyword arguments will be passed through as options to the underlying
     FUSE class.  Some interesting options include:
 
-        * nothreads:  switch off threading in the FUSE event loop
-        * fsname:     name to display in the mount info table
+        * nothreads Switch off threading in the FUSE event loop
+        * fsname Name to display in the mount info table
 
     """
     if foreground:

fs/expose/sftp.py

 """
+fs.expose.sftp
+==============
 
-  fs.expose.sftp:  expose an FS object over SFTP (via paramiko).
+Expose an FS object over SFTP (via paramiko).
 
 This module provides the necessary interfaces to expose an FS object over
 SFTP, plugging into the infratructure provided by the 'paramiko' module.
 
 For simple usage, the class 'BaseSFTPServer' provides an all-in-one server
-class based on the standard SocketServer module.  Use it like so:
+class based on the standard SocketServer module.  Use it like so::
 
     server = BaseSFTPServer((hostname,port),fs)
     server.serve_forever()
@@ -227,7 +229,7 @@ class BaseSFTPServer(sockserv.TCPServer,paramiko.ServerInterface):
     needed to expose an FS via SFTP.
 
     Operation is in the standard SocketServer style.  The target FS object
-    can be passed into the constructor, or set as an attribute on the server:
+    can be passed into the constructor, or set as an attribute on the server::
 
         server = BaseSFTPServer((hostname,port),fs)
         server.serve_forever()
@@ -240,10 +242,10 @@ class BaseSFTPServer(sockserv.TCPServer,paramiko.ServerInterface):
     FS.  This is intentional, since we can't guess what your authentication
     needs are.  To protect the exposed FS, override the following methods:
 
-        get_allowed_auths:      determine the allowed auth modes
-        check_auth_none:        check auth with no credentials
-        check_auth_password:    check auth with a password
-        check_auth_publickey:   check auth with a public key
+        * get_allowed_auths Determine the allowed auth modes
+        * check_auth_none Check auth with no credentials
+        * check_auth_password Check auth with a password
+        * check_auth_publickey Check auth with a public key
 
     """
 

fs/expose/xmlrpc.py

 """
+fs.expose.xmlrpc
+================
 
-  fs.expose.xmlrpc:  server to expose an FS via XML-RPC
+Server to expose an FS via XML-RPC
 
 This module provides the necessary infrastructure to expose an FS object
 over XML-RPC.  The main class is 'RPCFSServer', a SimpleXMLRPCServer subclass
@@ -136,7 +138,7 @@ class RPCFSServer(SimpleXMLRPCServer):
 
     This class takes as its first argument an FS instance, and as its second
     argument a (hostname,port) tuple on which to listen for XML-RPC requests.
-    Example:
+    Example::
 
         fs = OSFS('/var/srv/myfiles')
         s = RPCFSServer(fs,("",8080))

fs/memoryfs.py

 #!/usr/bin/env python
 """
+fs.memoryfs
+===========
 
-  fs.memoryfs:  A filesystem that exists only in memory
+A Filesystem that exists in memory only.
 
-Obviously that makes this particular filesystem very fast...
+File objects returned by MemoryFS.objects use StringIO objects for storage.
 
 """
 
 import datetime
 from fs.path import iteratepath
 from fs.base import *
+from fs import _thread_syncronize_default
 
 try:
     from cStringIO import StringIO
@@ -166,12 +169,18 @@ class DirEntry(object):
 
 
 class MemoryFS(FS):
+    
+    """ An in-memory filesystem.
+    
+    MemoryFS objects are very fast, but non-permantent. They are useful for creating a directory structure prior to writing it somewhere permanent.
+    
+    """
 
     def _make_dir_entry(self, *args, **kwargs):
         return self.dir_entry_factory(*args, **kwargs)
 
     def __init__(self, file_factory=None):
-        FS.__init__(self, thread_synchronize=True)
+        FS.__init__(self, thread_synchronize=_thread_syncronize_default)
         self.dir_entry_factory = DirEntry
         self.file_factory = file_factory or MemoryFile
 

fs/mountfs.py

-#!/usr/bin/env python
+"""
+fs.mountfs
+==========
+
+Contains MountFS class which is a virtual Filesystem which can have other Filesystems linked as branched directories, much like a symlink in Linux
+
+"""
 
 from fs.base import *
 from fs.objecttree import ObjectTree
+from fs import _thread_syncronize_default
 
 
 class DirMount(object):
@@ -27,7 +34,7 @@ class MountFS(FS):
     DirMount = DirMount
     FileMount = FileMount
 
-    def __init__(self, thread_synchronize=True):
+    def __init__(self, thread_synchronize=_thread_syncronize_default):
         FS.__init__(self, thread_synchronize=thread_synchronize)
         self.mount_tree = ObjectTree()
 
@@ -223,10 +230,10 @@ class MountFS(FS):
 
     @synchronize
     def mountdir(self, path, fs):
-        """Mounts a directory on a given path.
+        """Mounts a host FS object on a given path.
 
-        path -- A path within the MountFS
-        fs -- A filesystem object to mount
+        :param path: A path within the MountFS
+        :param fs: A filesystem object to mount
 
         """
         path = normpath(path)
@@ -235,11 +242,17 @@ class MountFS(FS):
 
     @synchronize
     def mountfile(self, path, open_callable=None, info_callable=None):
+        """Mounts a single file path. """
         path = normpath(path)
         self.mount_tree[path] = MountFS.FileMount(path, callable, info_callable)
 
     @synchronize
-    def unmount(self,path):
+    def unmount(self, path):
+        """Unmounds a path.
+        
+        :param path: Path to unmount
+        
+        """
         path = normpath(path)
         del self.mount_tree[path]
 

fs/multifs.py

-#!/usr/in/env python
+"""
+fs.multifs
+==========
+"""
 
 from fs.base import FS, FSError
 from fs.path import *
+from fs import _thread_syncronize_default
 
 
 class MultiFS(FS):
@@ -14,7 +18,7 @@ class MultiFS(FS):
     """
 
     def __init__(self):
-        FS.__init__(self, thread_synchronize=True)
+        FS.__init__(self, thread_synchronize=_thread_syncronize_default)
 
         self.fs_sequence = []
         self.fs_lookup =  {}
@@ -26,15 +30,15 @@ class MultiFS(FS):
     __repr__ = __str__
 
     def __unicode__(self):
-        return unicode(self.__str__())
+        return u"<MultiFS: %s>" % ", ".join(unicode(fs) for fs in self.fs_sequence)
 
 
     @synchronize
     def addfs(self, name, fs):
         """Adds a filesystem to the MultiFS.
 
-        name -- A unique name to refer to the filesystem being added
-        fs -- The filesystem to add
+        :param name: A unique name to refer to the filesystem being added
+        :param fs: The filesystem to add
 
         """
         if name in self.fs_lookup:
@@ -47,7 +51,7 @@ class MultiFS(FS):
     def removefs(self, name):
         """Removes a filesystem from the sequence.
 
-        name -- The name of the filesystem, as used in addfs
+        :param name: The name of the filesystem, as used in addfs
 
         """
         if name not in self.fs_lookup:
@@ -75,7 +79,7 @@ class MultiFS(FS):
         """Retrieves the filesystem that a given path would delegate to.
         Returns a tuple of the filesystem's name and the filesystem object itself.
 
-        path -- A path in MultiFS
+        :param path: A path in MultiFS
 
         """
         for fs in self:

fs/objecttree.py

@@ -94,7 +94,7 @@ class ObjectTree(object):
         return self.root.keys()
 
     def iterkeys(self):
-        return self.root.keys()
+        return self.root.iterkeys()
 
     def items(self):
         return self.root.items()

fs/osfs.py

-#!/usr/bin/env python
+"""
+fs.osfs
+=======
+
+Exposes the OS Filesystem as an FS object.
+
+For example, to print all the files and directories in the OS root::
+
+    >>> from fs.osfs import OSFS
+    >>> home_fs = OSFS('/')
+    >>> print home_fs.listdir()
+
+"""
+
 
 import os
 import sys
@@ -6,6 +19,7 @@ import errno
 
 from fs.base import *
 from fs.path import *
+from fs import _thread_syncronize_default
 
 try:
     import xattr
@@ -27,7 +41,18 @@ class OSFS(FS):
     methods in the os and os.path modules.
     """
 
-    def __init__(self, root_path, dir_mode=0700, thread_synchronize=True, encoding=None):
+    def __init__(self, root_path, dir_mode=0700, thread_synchronize=_thread_syncronize_default, encoding=None):
+        
+        """
+        Creates an FS object that represents the OS Filesystem under a given root path
+        
+        :param root_path: The root OS path
+        :param dir_mode: srt
+        :param thread_syncronize: If True, this object will be thread-safe by use of a threading.Lock object
+        :param encoding: The encoding method for path strings
+        
+        """
+        
         FS.__init__(self, thread_synchronize=thread_synchronize)
         self.encoding = encoding
         root_path = os.path.expanduser(os.path.expandvars(root_path))
@@ -45,6 +70,9 @@ class OSFS(FS):
 
     def __str__(self):
         return "<OSFS: %s>" % self.root_path
+    
+    def __unicode__(self):
+        return u"<OSFS: %s>" % self.root_path
 
     def getsyspath(self, path, allow_none=False):
         path = relpath(normpath(path)).replace("/",os.sep)
@@ -113,17 +141,17 @@ class OSFS(FS):
             raise
 
     @convert_os_errors
-    def removedir(self, path, recursive=False,force=False):
+    def removedir(self, path, recursive=False, force=False):
         sys_path = self.getsyspath(path)
         if force:
-            for path2 in self.listdir(path,absolute=True,files_only=True):
+            for path2 in self.listdir(path, absolute=True, files_only=True):
                 try:
                     self.remove(path2)
                 except ResourceNotFoundError:
                     pass
-            for path2 in self.listdir(path,absolute=True,dirs_only=True):
+            for path2 in self.listdir(path, absolute=True, dirs_only=True):
                 try:
-                    self.removedir(path2,force=True)
+                    self.removedir(path2, force=True)
                 except ResourceNotFoundError:
                     pass
         #  Don't remove the root directory of this FS

fs/path.py

 """
-
-  fs.path: useful functions for FS path manipulation.
+Useful functions for FS path manipulation.
 
 This is broadly similar to the standard 'os.path' module but works with
 paths in the canonical format expected by all FS objects (backslash-separated,
@@ -16,6 +15,8 @@ def normpath(path):
     duplicate slashes, replaces forward with backward slashes, and generally
     tries very hard to return a new path string the canonical FS format.
     If the path is invalid, ValueError will be raised.
+    
+    :param path: Path to normalize
 
     >>> normpath(r"foo\\bar\\baz")
     'foo/bar/baz'
@@ -47,20 +48,41 @@ def normpath(path):
         if not components:
             components = [""]
         components.insert(0,"")
-    return "/".join(components)
+    if isinstance(path, unicode):
+        return u"/".join(components)
+    else:
+        return '/'.join(components)
 
 
 def iteratepath(path, numsplits=None):
-    """Iterate over the individual components of a path."""
+    """Iterate over the individual components of a path.
+    
+    :param path: Path to iterate over
+    :numsplits: Maximum number of splits
+    
+    """
     path = relpath(normpath(path))
     if not path:
         return []
     if numsplits == None:
-        return path.split('/')
+        return map(None, path.split('/'))
     else:
-        return path.split('/', numsplits)
-
-
+        return map(None, path.split('/', numsplits))
+        
+def recursepath(path, reverse=False):
+    """Iterate from root to path, returning intermediate paths"""
+    
+    paths = list(iteratepath(path))
+    
+    if reverse:
+        paths = []
+        while path.lstrip('/'):
+            paths.append(path)
+            path = dirname(path)
+        return paths
+    else:   
+        return [u'/'.join(paths[:i+1]) for i in xrange(len(paths))]
+    
 def abspath(path):
     """Convert the given path to an absolute path.
 
@@ -69,9 +91,9 @@ def abspath(path):
 
     """
     if not path:
-        return "/"
-    if path[0] != "/":
-        return "/" + path
+        return u'/'
+    if not path.startswith('/'):
+        return u'/' + path
     return path
 
 
@@ -80,6 +102,8 @@ def relpath(path):
 
     This is the inverse of abspath(), stripping a leading '/' from the
     path if it is present.
+    
+    :param path: Path to adjust
 
     """
     while path and path[0] == "/":
@@ -89,6 +113,8 @@ def relpath(path):
 
 def pathjoin(*paths):
     """Joins any number of paths together, returning a new path string.
+    
+    :param paths: Paths to join are given in positional arguments
 
     >>> pathjoin('foo', 'bar', 'baz')
     'foo/bar/baz'
@@ -111,7 +137,7 @@ def pathjoin(*paths):
 
     path = normpath("/".join(relpaths))
     if absolute and not path.startswith("/"):
-        path = "/" + path
+        path = u"/" + path
     return path
 
 # Allow pathjoin() to be used as fs.path.join()
@@ -123,6 +149,8 @@ def pathsplit(path):
 
     This function splits a path into a pair (head,tail) where 'tail' is the
     last pathname component and 'head' is all preceeding components.
+    
+    :param path: Path to split
 
     >>> pathsplit("foo/bar")
     ('foo', 'bar')
@@ -133,7 +161,7 @@ def pathsplit(path):
     """
     split = normpath(path).rsplit('/', 1)
     if len(split) == 1:
-        return ('', split[0])
+        return (u'', split[0])
     return tuple(split)
 
 # Allow pathsplit() to be used as fs.path.split()
@@ -145,6 +173,8 @@ def dirname(path):
 
     This is always equivalent to the 'head' component of the value returned
     by pathsplit(path).
+    
+    :param path: A FS path
 
     >>> dirname('foo/bar/baz')
     'foo/bar'
@@ -158,6 +188,8 @@ def basename(path):
 
     This is always equivalent to the 'head' component of the value returned
     by pathsplit(path).
+    
+    :param path: A FS path
 
     >>> basename('foo/bar/baz')
     'baz'
@@ -168,6 +200,9 @@ def basename(path):
 
 def issamedir(path1, path2):
     """Return true if two paths reference a resource in the same directory.
+    
+    :param path1: An FS path
+    :param path2: An FS path
 
     >>> issamedir("foo/bar/baz.txt", "foo/bar/spam.txt")
     True
@@ -180,7 +215,10 @@ def issamedir(path1, path2):
 
 def isprefix(path1, path2):
     """Return true is path1 is a prefix of path2.
-
+    
+    :param path1: An FS path
+    :param path2: An FS path
+    
     >>> isprefix("foo/bar", "foo/bar/spam.txt")
     True
     >>> isprefix("foo/bar/", "foo/bar")
@@ -204,6 +242,8 @@ def isprefix(path1, path2):
 
 def forcedir(path):
     """Ensure the path ends with a trailing /
+    
+    :param path: An FS path
 
     >>> forcedir("foo/bar")
     'foo/bar/'

fs/remote.py

 """
 
-  fs.remote:  utilities for interfacing with remote filesystems
+fs.remote
+=========
+
+Utilities for interfacing with remote filesystems
 
 
 This module provides reusable utility functions that can be used to construct
 FS subclasses interfacing with a remote filesystem.  These include:
 
-  RemoteFileBuffer:  a file-like object that locally buffers the contents
+  * RemoteFileBuffer:  a file-like object that locally buffers the contents
                      of a remote file, writing them back on flush() or close().
 
-  ConnectionManagerFS:  a WrapFS subclass that tracks the connection state
+  * ConnectionManagerFS:  a WrapFS subclass that tracks the connection state
                         of a remote FS, and allows client code to wait for
                         a connection to be re-established.
 
-  CacheFS:  a WrapFS subclass that caces file and directory meta-data in
+   * CacheFS:  a WrapFS subclass that caces file and directory meta-data in
             memory, to speed access to a remote FS.
 
 """
@@ -44,7 +47,7 @@ class RemoteFileBuffer(object):
     The intended use-case is for a remote filesystem (e.g. S3FS) to return
     instances of this class from its open() method, and to provide the
     file-uploading logic in its setcontents() method, as in the following
-    pseudo-code:
+    pseudo-code::
 
         def open(self,path,mode="r"):
             rf = self._get_remote_file(path)
@@ -176,7 +179,7 @@ class ConnectionManagerFS(LazyFS):
     Since some remote FS classes can raise RemoteConnectionError during
     initialisation, this class makes use of lazy initialization. The
     remote FS can be specified as an FS instance, an FS subclass, or a
-    (class,args) or (class,args,kwds) tuple. For example:
+    (class,args) or (class,args,kwds) tuple. For example::
 
         >>> fs = ConnectionManagerFS(MyRemoteFS("http://www.example.com/"))
         Traceback (most recent call last):

fs/s3fs.py

 """
+fs.s3fs
+=======
 
-  fs.s3fs:  FS subclass accessing files in Amazon S3
+FS subclass accessing files in Amazon S3
 
 This module provides the class 'S3FS', which implements the FS filesystem
 interface for objects stored in Amazon Simple Storage Service (S3).

fs/sftpfs.py

 """
+fs.sftpfs
+=========
 
-  fs.sftpfs:  Filesystem accesing an SFTP server (via paramiko)
+Filesystem accessing an SFTP server (via paramiko)
 
 """
 
@@ -64,6 +66,8 @@ class SFTPFS(FS):
         self._owns_transport = False
         self._credentials = credentials
         self._tlocal = thread_local()
+        self._transport = None
+        self._client = None
         if isinstance(connection,paramiko.Channel):
             self._transport = None
             self._client = paramiko.SFTPClient(connection)

fs/tempfs.py

-#!/usr/bin/env python
+"""
+fs.tempfs
+=========
+
+Make a temporary file system that exists in a folder provided by the OS. All files contained in a TempFS are removed when the `close` method is called (or when the TempFS is cleaned up by Python).
+
+"""
 
 import os
 import time
@@ -6,13 +12,14 @@ import tempfile
 
 from fs.osfs import OSFS
 from fs.errors import *
+from fs import _thread_syncronize_default
 
 class TempFS(OSFS):
 
     """Create a Filesystem in a tempory directory (with tempfile.mkdtemp),
     and removes it when the TempFS object is cleaned up."""
 
-    def __init__(self, identifier=None, temp_dir=None, dir_mode=0700, thread_synchronize=True):
+    def __init__(self, identifier=None, temp_dir=None, dir_mode=0700, thread_synchronize=_thread_syncronize_default):
         """Creates a temporary Filesystem
 
         identifier -- A string that is included in the name of the temporary directory,
@@ -29,13 +36,14 @@ class TempFS(OSFS):
     __repr__ = __str__
 
     def __unicode__(self):
-        return unicode(self.__str__())
+        return u'<TempFS: %s>' % self._temp_dir
 
     def close(self):
         """Removes the temporary directory.
 
         This will be called automatically when the object is cleaned up by
-        Python. Note that once this method has been called, the FS object may
+        Python, although it is advisable to call it manually.
+        Note that once this method has been called, the FS object may
         no longer be used.
         """
         #  Depending on how resources are freed by the OS, there could

fs/tests/__init__.py

@@ -9,7 +9,7 @@
 #  be captured by nose and reported appropriately
 import sys
 import logging
-logging.basicConfig(level=logging.ERROR,stream=sys.stdout)
+logging.basicConfig(level=logging.ERROR, stream=sys.stdout)
 
 from fs.base import *
 
@@ -297,7 +297,7 @@ class FSTestCases:
         check = self.check
         contents = "If the implementation is hard to explain, it's a bad idea."
         def makefile(path):
-            self.fs.createfile(path,contents)
+            self.fs.createfile(path, contents)
 
         self.fs.makedir("a")
         self.fs.makedir("b")
@@ -700,7 +700,7 @@ class ThreadingTestCases:
         self._runThreads(makedir,removedir)
         if self.fs.isdir("testdir"):
             self.assertEquals(len(errors),1)
-	    self.assertFalse(isinstance(errors[0],DestinationExistsError))
+            self.assertFalse(isinstance(errors[0],DestinationExistsError))
             self.fs.removedir("testdir")
         else:
             self.assertEquals(len(errors),0)

fs/tests/ftpserver.py

+try:
+    from pyftpdlib import ftpserver
+except ImportError:
+    print "Requires pyftpdlib <http://code.google.com/p/pyftpdlib/>"
+    raise
+    
+import sys
+
+authorizer = ftpserver.DummyAuthorizer()
+authorizer.add_user("user", "12345", sys.argv[1], perm="elradfmw")
+authorizer.add_anonymous(sys.argv[1])
+
+handler = ftpserver.FTPHandler
+handler.authorizer = authorizer
+address = ("127.0.0.1", 21)
+
+ftpd = ftpserver.FTPServer(address, handler)
+ftpd.serve_forever()
+

fs/tests/test_fs.py

@@ -31,7 +31,6 @@ class TestOSFS(unittest.TestCase,FSTestCases,ThreadingTestCases):
         return os.path.exists(os.path.join(self.temp_dir, relpath(p)))
 
 
-
 class TestSubFS(unittest.TestCase,FSTestCases,ThreadingTestCases):
 
     def setUp(self):
@@ -89,4 +88,18 @@ class TestTempFS(unittest.TestCase,FSTestCases,ThreadingTestCases):
     def check(self, p):
         td = self.fs._temp_dir
         return os.path.exists(os.path.join(td, relpath(p)))
+     
+        
+from fs import wrapfs
+class TestWrapFS(unittest.TestCase, FSTestCases, ThreadingTestCases):
+    
+    def setUp(self):
+        sys.setcheckinterval(1)
+        self.temp_dir = tempfile.mkdtemp(u"fstest")
+        self.fs = wrapfs.WrapFS(osfs.OSFS(self.temp_dir))
 
+    def tearDown(self):
+        shutil.rmtree(self.temp_dir)
+
+    def check(self, p):
+        return os.path.exists(os.path.join(self.temp_dir, relpath(p)))

fs/tests/test_ftpfs.py

+
+from fs.tests import FSTestCases, ThreadingTestCases
+
+import unittest
+
+import os
+import sys
+import shutil
+import tempfile
+import subprocess
+import time
+from os.path import abspath
+
+try:
+    from pyftpdlib import ftpserver
+except ImportError:
+    raise ImportError("Requires pyftpdlib <http://code.google.com/p/pyftpdlib/>")
+
+from fs.path import *
+
+from fs import ftpfs
+
+ftp_port = 30000
+class TestFTPFS(unittest.TestCase, FSTestCases, ThreadingTestCases):
+    
+    def setUp(self):
+        global ftp_port
+        #ftp_port += 1
+        use_port = str(ftp_port)
+        #ftp_port = 10000
+                
+        sys.setcheckinterval(1)
+        self.temp_dir = tempfile.mkdtemp(u"ftpfstests")
+        
+        self.ftp_server = subprocess.Popen(['python', abspath(__file__), self.temp_dir, str(use_port)])
+        # Need to sleep to allow ftp server to start
+        time.sleep(.2)        
+        self.fs = ftpfs.FTPFS('127.0.0.1', 'user', '12345', port=use_port, timeout=5.0)
+        
+    
+    def tearDown(self):
+        
+        if sys.platform == 'win32':
+            import win32api
+            win32api.TerminateProcess(int(process._handle), -1)
+        else:
+            os.system('kill '+str(self.ftp_server.pid))
+        shutil.rmtree(self.temp_dir)
+        
+    def check(self, p):
+        return os.path.exists(os.path.join(self.temp_dir, relpath(p)))
+        
+    
+if __name__ == "__main__":
+    
+    # Run an ftp server that exposes a given directory    
+    import sys
+    authorizer = ftpserver.DummyAuthorizer()
+    authorizer.add_user("user", "12345", sys.argv[1], perm="elradfmw")
+    authorizer.add_anonymous(sys.argv[1])
+    
+    def nolog(*args):
+        pass
+    ftpserver.log = nolog
+    ftpserver.logline = nolog
+    
+    handler = ftpserver.FTPHandler
+    handler.authorizer = authorizer
+    address = ("127.0.0.1", int(sys.argv[2]))
+    #print address
+    
+    ftpd = ftpserver.FTPServer(address, handler)
+    
+    ftpd.serve_forever()
+
+
+
+

fs/wrapfs.py

 """
+fs.wrapfs
+=========
 
-  fs.wrapfs:  class for wrapping an existing FS object with added functionality
+A class for wrapping an existing FS object with additional functionality.
 
 This module provides the class WrapFS, a base class for objects that wrap
 another FS object and provide some transformation of its contents.  It could
@@ -121,10 +123,10 @@ class WrapFS(FS):
         return self.wrapped_fs.hassyspath(self._encode(path))
 
     @rewrite_errors
-    def open(self,path,mode="r"):
-        (mode,wmode) = self._adjust_mode(mode)
-        f = self.wrapped_fs.open(self._encode(path),wmode)
-        return self._file_wrap(f,mode)
+    def open(self, path, mode="r", **kwargs):
+        (mode, wmode) = self._adjust_mode(mode)
+        f = self.wrapped_fs.open(self._encode(path), wmode, **kwargs)
+        return self._file_wrap(f, mode)
 
     @rewrite_errors
     def exists(self,path):
@@ -509,3 +511,41 @@ class LimitSizeFile(object):
     def __iter__(self):
         return iter(self.file)
 
+
+class ReadOnlyFS(WrapFS):
+    """ Makes a FS object read only. Any operation that could potentially modify
+    the underlying file system will throw an UnsupportedError
+    
+    Note that this isn't a secure sandbox, untrusted code could work around the
+    read-only restrictions by getting the base class. Its main purpose is to
+    provide a degree of safety if you want to protect an FS object from
+    modification.
+    
+    """
+        
+    def getsyspath(self, path, allow_none=False):
+        """ Doesn't technically modify the filesystem but could be used to work
+        around read-only restrictions. """
+        if allow_none:
+            return None
+        raise NoSysPathError(path)
+    
+    def open(self, path, mode='r', **kwargs):
+        """ Only permit read access """
+        if 'w' in mode or 'a' in mode:
+            raise UnsupportedError('write')
+        return super(ReadOnlyFS, self).open(path, mode, **kwargs)
+        
+    def _no_can_do(self, *args, **kwargs):
+        """ Replacement method for methods that can modify the file system """
+        raise UnsupportedError('write')
+      
+    move = _no_can_do
+    movedir = _no_can_do
+    copy = _no_can_do
+    copydir = _no_can_do
+    makedir = _no_can_do
+    rename = _no_can_do
+    setxattr = _no_can_do
+    delattr = _no_can_do
+    
\ No newline at end of file

fs/xattrs.py

 """
+fs.xattrs
+=========
 
-  fs.xattrs:  extended attribute support for FS
+Extended attribute support for FS
 
 This module defines a standard interface for FS subclasses that want to
 support extended file attributes, and a WrapFS subclass that can simulate
-extended attributes on top of an ordinery FS.
+extended attributes on top of an ordinary FS.
 
 FS instances offering extended attribute support must provide the following
 methods:
 
-  getxattr(path,name)        -  get the named attribute for the given path,
-                                or None if it does not exist
-  setxattr(path,name,value)  -  set the named attribute for the given path
-                                to the given value
-  delxattr(path,name)        -  delete the named attribute for the given path,
-                                raising KeyError if it does not exist
-  listxattrs(path)           -  iterator over all stored attribute names for
-                                the given path
+  * getxattr(path,name) Get the named attribute for the given path, or None if it does not exist
+  * setxattr(path,name,value) Set the named attribute for the given path to the given value
+  * delxattr(path,name) Delete the named attribute for the given path, raising KeyError if it does not exist
+  * listxattrs(path) Iterate over all stored attribute names for the given path
 
 If extended attributes are required by FS-consuming code, it should use the
-function 'ensure_xattrs'.  This will interrogate an FS object to determine
+function 'ensure_xattrs'. This will interrogate an FS object to determine
 if it has native xattr support, and return a wrapped version if it does not.
 """
 
@@ -41,6 +39,8 @@ def ensure_xattrs(fs):
     Given an FS object, this function returns an equivalent FS that has support
     for extended attributes.  This may be the original object if they are
     supported natively, or a wrapper class is they must be simulated.
+    
+    :param fs: An FS object that must have xattrs
     """
     try:
         #  This attr doesn't have to exist, None should be returned by default

fs/zipfs.py

-#!/usr/bin/env python
+"""
+fs.zipfs
+========
+
+A FS object that represents the contents of a Zip file
+
+"""
 
 from fs.base import *
 
@@ -52,12 +58,12 @@ class ZipFS(FS):
     def __init__(self, zip_file, mode="r", compression="deflated", allowZip64=False, encoding="CP437", thread_synchronize=True):
         """Create a FS that maps on to a zip file.
 
-        zip_file -- A (system) path, or a file-like object
-        mode -- Mode to open zip file: 'r' for reading, 'w' for writing or 'a' for appending
-        compression -- Can be 'deflated' (default) to compress data or 'stored' to just store date
-        allowZip64 -- Set to True to use zip files greater than 2 MB, default is False
-        encoding --  The encoding to use for unicode filenames
-        thread_synchronize -- Set to True (default) to enable thread-safety
+        :param zip_file: A (system) path, or a file-like object
+        :param mode: Mode to open zip file: 'r' for reading, 'w' for writing or 'a' for appending
+        :param compression: Can be 'deflated' (default) to compress data or 'stored' to just store date
+        :param allowZip64: -- Set to True to use zip files greater than 2 MB, default is False
+        :param encoding: --  The encoding to use for unicode filenames
+        :param thread_synchronize: -- Set to True (default) to enable thread-safety
 
         """
         FS.__init__(self, thread_synchronize=thread_synchronize)