Added pygments_api example

.hgignore

@@ -8,3 +8,4 @@ html
 .project
 .pydevproject
 .settings
+.cache

docs/authenticators.rst

+:mod:`authenticators`
+=====================
+
+.. module:: authenticators
+
+The authenticators module provides a standard set of authentication methods that can be plugged in to a :class:`Resource`, as well as providing a template by which to write custom authentication methods.
+
+The base class
+--------------
+
+All authenticators must subclass the :class:`BaseAuthenticator` class and override it's :func:`authenticate` method.
+
+.. class:: BaseAuthenticator
+
+   .. method:: authenticate(request)
+
+      Authenticate the request and return the authentication context or None.
+
+      The default permission checking on :class:`.Resource` will use the allowed_methods attribute for permissions if the authentication context is not None, and use anon_allowed_methods otherwise.
+
+      The authentication context is passed to the method calls (eg :meth:`.Resource.get`, :meth:`.Resource.post` etc...) in order to allow them to apply any more fine grained permission checking at the point the response is being generated.
+
+      This function must be overridden to be implemented.
+
+Provided authenticators
+-----------------------
+

docs/index.rst

@@ -18,8 +18,9 @@ Some of FlyWheel's features:
 
   resource
   modelresource
-  parsers
   emitters
+  parsers
+  authenticators
   response
 
 

docs/resource.rst

 :mod:`resource` 
 ===============
 
-The :mod:`resource` module is the core of FlyWheel.  It provides the :class:`Resource` base class which handles incoming HTTP requests and maps them to method calls, performing authentication, input deserialization, input validation, output serialization.
+.. module:: resource
+
+The :mod:`resource` module is the core of FlyWheel.  It provides the :class:`Resource` base class which handles incoming HTTP requests and maps them to method calls, performing authentication, input deserialization, input validation and output serialization.
 
 Resources are created by sublassing :class:`Resource`, setting a number of class attributes, and overriding one or more methods.
 
+.. class:: Resource
+
 :class:`Resource` class attributes
 ----------------------------------
 
@@ -26,7 +30,7 @@ The following class attributes determine the behavior of the Resource and are in
 
 .. attribute:: Resource.emitters
 
-  Lists the set of emitters that the Resource supports.  This determines which media types the resource can serialize it's output to.  Clients can specify which media types they accept using standard HTTP content negotiation via the Accept header.  (See `RFC 2616 - Sec 14.1 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html>`_)  Clients can also override this standard content negotiation by specifying a `_format` ...
+  The list of emitters that the Resource supports.  This determines which media types the resource can serialize it's output to.  Clients can specify which media types they accept using standard HTTP content negotiation via the Accept header.  (See `RFC 2616 - Sec 14.1 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html>`_)  Clients can also override this standard content negotiation by specifying a `_format` ...
 
   The :mod:`emitters` module provides the :class:`BaseEmitter` class and a set of default emitters, including emitters for JSON and XML, as well as emitters for HTML and Plain Text which provide for a self documenting API.
 
@@ -36,12 +40,18 @@ The following class attributes determine the behavior of the Resource and are in
 
 .. attribute:: Resource.parsers
 
-  Lists the set of parsers that the Resource supports. This determines which media types the resource can accept as input for incoming HTTP requests.  (Typically PUT and POST requests).
+  The list of parsers that the Resource supports. This determines which media types the resource can accept as input for incoming HTTP requests.  (Typically PUT and POST requests).
 
   The ordering of the Parsers may be considered informative of preference but is not used ...
 
   Default: ``(parsers.JSONParser, parsers.XMLParser, parsers.FormParser)``
 
+.. attribute:: Resource.authenticators
+
+  The list of authenticators that the Resource supports.  This determines which authentication methods (eg Basic, Digest, OAuth) are used to authenticate requests.
+
+  Default: ``(authenticators.UserLoggedInAuthenticator, authenticators.BasicAuthenticator)`` 
+
 .. attribute:: Resource.form
 
   If not None, this attribute should be a Django form which will be used to validate any request data.
@@ -76,8 +86,8 @@ The following class attributes determine the behavior of the Resource and are in
 .. method:: Resource.emitted_media_types
 .. method:: Resource.parsed_media_types
 
-:class:`Resource` reserved parameters
--------------------------------------
+:class:`Resource` reserved form and query parameters
+----------------------------------------------------
 
 .. attribute:: Resource.ACCEPT_QUERY_PARAM
 
@@ -85,7 +95,7 @@ The following class attributes determine the behavior of the Resource and are in
 
   Set to None to disable, or to another string value to use another name for the reserved URL query parameter.
 
-  Default: ``_accept``
+  Default: ``"_accept"``
 
 .. attribute:: Resource.METHOD_PARAM
 
@@ -93,7 +103,7 @@ The following class attributes determine the behavior of the Resource and are in
 
   Set to None to disable, or to another string value to use another name for the reserved form field.
 
-  Default: ``_method``
+  Default: ``"_method"``
 
 .. attribute:: Resource.CONTENTTYPE_PARAM
 
@@ -103,7 +113,7 @@ The following class attributes determine the behavior of the Resource and are in
   
   Set to None to disable, or to another string value to use another name for the reserved form field.
 
-  Default: ``_contenttype``
+  Default: ``"_contenttype"``
 
 .. attribute:: Resource.CONTENT_PARAM
 
@@ -111,7 +121,7 @@ The following class attributes determine the behavior of the Resource and are in
 
   Set to None to disable, or to another string value to use another name for the reserved form field.
 
-  Default: ``_content``
+  Default: ``"_content"``
 
 .. attribute:: Resource.CSRF_PARAM
 
@@ -119,7 +129,8 @@ The following class attributes determine the behavior of the Resource and are in
 
   Setting to None does not disable Django's CSRF middleware, but it does mean that the field name will not be treated as reserved by FlyWheel, so for example the default :class:`FormParser` will return fields with this as part of the request content, rather than ignoring them.
 
-  Default:: ``csrfmiddlewaretoken``
+  Default:: ``"csrfmiddlewaretoken"``
 
 reserved params
 internal methods
+

examples/pygments_api/forms.py

+from django import forms
+
+from pygments.lexers import get_all_lexers
+from pygments.styles import get_all_styles
+
+import httplib2 as httplib
+
+
+LEXER_CHOICES = sorted([(item[1][0], item[0]) for item in get_all_lexers()])
+STYLE_CHOICES = sorted((item, item) for item in list(get_all_styles()))
+
+
+class PygmentsForm(forms.Form):
+    """A simple form with some of the most important pygments settings.
+    The code to be highlighted can be specified either in a text field, or by URL.
+    We do some additional form validation to ensure clients see helpful error responses."""
+
+    code_url = forms.URLField(required=False, label='Code URL',
+                              help_text='eg. https://bitbucket.org/tomchristie/flywheel/raw/cc266285d879/flywheel/resource.py')
+    code_text = forms.CharField(widget=forms.Textarea, required=False, label='Code Text',
+                                help_text='Either supply a URL for the code to be highlighted or copy and paste the code text here.')
+    title = forms.CharField(required=False, help_text='(Optional)')
+    linenos = forms.BooleanField(label='Show Line Numbers', required=False)
+    lexer = forms.ChoiceField(choices=LEXER_CHOICES, initial='python')
+    style = forms.ChoiceField(choices=STYLE_CHOICES, initial='friendly')
+
+    
+    def clean_code_url(self):
+        """Custom field validation.
+        Ensure that code URLs really are valid, and return the content they point to in the cleaned_data,
+        rather than returning the URL itself."""
+        cleaned_data = self.cleaned_data
+        url = cleaned_data.get('code_url')
+        if not url:
+            return ''
+
+        try:
+            http = httplib.Http('.cache')
+            resp, content = http.request(url)
+        except:
+            raise forms.ValidationError('The URL supplied cannot be reached')
+
+        if int(resp.status/100) != 2:
+            raise forms.ValidationError('The URL supplied does not return successfully')
+        if not content:
+            raise forms.ValidationError('The URL supplied returns no content')
+        
+        return content
+        
+
+    def clean(self):
+        """Custom form validation.
+        Ensure that only one of code_url and code_text is set, and return the content of whichever is set in 'code'."""
+        cleaned_data = self.cleaned_data
+        code_url = cleaned_data.get('code_url')
+        code_text = cleaned_data.get('code_text')
+
+        if not code_url and not code_text:
+            raise forms.ValidationError('Either the URL or the code text must be supplied')
+        if code_url and code_text:
+            raise forms.ValidationError('You may not specify both the URL and the code text')
+
+        if code_url:
+            cleaned_data['code'] = code_url
+            del cleaned_data['code_url']
+        else:
+            cleaned_data['code'] = code_text
+            del cleaned_data['code_text']
+
+        return cleaned_data
+

examples/pygments_api/urls.py

+from django.conf.urls.defaults import patterns
+
+urlpatterns = patterns('pygments_api.views',
+    (r'^$', 'PygmentsRoot'), 
+    (r'^([a-zA-Z0-9]+)/$', 'PygmentsInstance'),
+)

examples/pygments_api/views.py

+from django.conf import settings
+
+from flywheel.resource import Resource
+from flywheel.response import Response, status
+from flywheel.emitters import BaseEmitter
+
+from pygments.formatters import HtmlFormatter
+from pygments.lexers import get_lexer_by_name
+from pygments import highlight
+
+from forms import PygmentsForm
+
+import os
+import hashlib
+
+# We need somewhere to store the code that we highlight
+HIGHLIGHTED_CODE_DIR = os.path.join(settings.MEDIA_ROOT, 'pygments')
+
+
+class HTMLEmitter(BaseEmitter):
+    """Basic emitter which just returns the content without any further serialization."""
+    media_type = 'text/html'
+
+
+class PygmentsRoot(Resource):
+    """This example demonstrates a simple RESTful Web API aound the awesome pygments library.
+    This top level resource is used to create  """
+    form = PygmentsForm
+    allowed_methods = anon_allowed_methods = ('POST',)
+
+    def post(self, request, auth, content):
+        # Generate a unique id by hashing the input
+        input_str = ''.join(['%s%s' % (key, content[key]) for key in sorted(content.keys())])
+        hash = hashlib.md5()
+        hash.update(input_str)
+        unique_id = hash.hexdigest()
+        pathname = os.path.join(HIGHLIGHTED_CODE_DIR, unique_id)
+
+        if not os.path.exists(pathname):
+            # We only need to generate the file if it doesn't already exist.
+            title = content['title'] if content['title'] else None
+            linenos = 'table' if content['linenos'] else False
+            lexer = get_lexer_by_name(content['lexer'])
+            formatter = HtmlFormatter(style=content['style'], linenos=linenos, full=True, title=title)
+            
+            with open(pathname, 'w') as outfile:
+                highlight(content['code'], lexer, formatter, outfile)
+            
+        return Response(status.HTTP_303_SEE_OTHER, headers={'Location': self.reverse(PygmentsInstance, unique_id)})
+
+
+class PygmentsInstance(Resource):
+    """Simply return the stored highlighted HTML file with the correct mime type.
+    This Resource only emits HTML and uses a standard HTML emitter rather than FlyWheel's DocumentingHTMLEmitter class."""
+    allowed_methods = anon_allowed_methods = ('GET',)
+    emitters = (HTMLEmitter,)
+
+    def get(self, request, auth, unique_id):
+        pathname = os.path.join(HIGHLIGHTED_CODE_DIR, unique_id)
+        if not os.path.exists(pathname):
+            return Resource(status.HTTP_404_NOT_FOUND)
+        return open(pathname, 'r').read()
+
+

examples/settings.py

@@ -92,5 +92,6 @@ INSTALLED_APPS = (
     'django.contrib.admin',
     'flywheel',
     'blogpost',
-    'objectstore'
+    'objectstore',
+    'pygments_api'
 )

examples/urls.py

@@ -4,8 +4,9 @@ from django.contrib import admin
 admin.autodiscover()
 
 urlpatterns = patterns('',
-    (r'^blog-post-example/', include('blogpost.urls')),
-    (r'^object-store-example/', include('objectstore.urls')),
+    (r'^blog-post-api/', include('blogpost.urls')),
+    (r'^object-store-api/', include('objectstore.urls')),
+    (r'^pygments-api/', include('pygments_api.urls')),
     (r'^admin/doc/', include('django.contrib.admindocs.urls')),
     (r'^admin/', include(admin.site.urls)),
 )

flywheel/emitters.py

@@ -17,7 +17,20 @@ class BaseEmitter(object):
         self.resource = resource
 
     def emit(self, output=NoContent, verbose=False):
-        raise Exception('emit() function on a subclass of BaseEmitter must be implemented')
+        if output is NoContent:
+            return ''
+        
+        return output
+
+class TemplateEmitter(BaseEmitter):
+    media_type = None
+    template = None
+
+    def emit(self, output=NoContent, verbose=False):
+        if output is NoContent:
+            return ''
+
+        return self.template.render(Context(output))
         
 
 from django import forms

flywheel/parsers.py

@@ -8,13 +8,21 @@ except ImportError:
 # TODO: Make all parsers only list a single media_type, rather than a list
 
 class BaseParser(object):
+    """All parsers should extend BaseParser, specifing a media_type attribute,
+    and overriding the parse() method."""
+
     media_types = ()
 
     def __init__(self, resource):
+        """Initialise the parser with the Resource instance as state,
+        in case the parser needs to access any metadata on the Resource object."""
         self.resource = resource
     
     def parse(self, input):
-        return {}
+        """Given some serialized input, return the deserialized output.
+        The input will be the raw request content body.  The return value may be of
+        any type, but for many parsers/inputs it might typically be a dict."""
+        return input
 
 
 class JSONParser(BaseParser):
@@ -26,6 +34,7 @@ class JSONParser(BaseParser):
         except ValueError, exc:
             raise ResponseException(status.HTTP_400_BAD_REQUEST, {'detail': 'JSON parse error - %s' % str(exc)})
 
+
 class XMLParser(BaseParser):
     media_types = ('application/xml',)
 

flywheel/resource.py

@@ -9,7 +9,6 @@ from decimal import Decimal
 import re
 from itertools import chain
 
-# TODO: Authentication
 # TODO: Display user login in top panel: http://stackoverflow.com/questions/806835/django-redirect-to-previous-page-after-login
 # TODO: Figure how out references and named urls need to work nicely
 # TODO: POST on existing 404 URL, PUT on existing 404 URL
@@ -73,6 +72,7 @@ class Resource(object):
         """Make the class callable so it can be used as a Django view."""
         self = object.__new__(cls)
         self.__init__(request)
+        # TODO: Remove this debugging code
         try:
             return self._handle_request(request, *args, **kwargs)
         except:
@@ -120,22 +120,16 @@ class Resource(object):
         (This emitter is used if the client does not send and Accept: header, or sends Accept: */*)"""
         return self.emitters[0]
 
-    # TODO:
-    
+    #@property
     #def parsed_media_types(self):
     #    """Return an list of all the media types that this resource can emit."""
     #    return [parser.media_type for parser in self.parsers]
-    
+    #
+    #@property
     #def default_parser(self):
     #    return self.parsers[0]
 
 
-    def reverse(self, view, *args, **kwargs):
-        """Return a fully qualified URI for a given view or resource.
-        Add the domain using the Sites framework if possible, otherwise fallback to using the current request."""
-        return self.add_domain(reverse(view, args=args, kwargs=kwargs))
-
-
     def get(self, request, auth, *args, **kwargs):
         """Must be subclassed to be implemented."""
         self.not_implemented('GET')
@@ -156,6 +150,12 @@ class Resource(object):
         self.not_implemented('DELETE')
 
 
+    def reverse(self, view, *args, **kwargs):
+        """Return a fully qualified URI for a given view or resource.
+        Add the domain using the Sites framework if possible, otherwise fallback to using the current request."""
+        return self.add_domain(reverse(view, args=args, kwargs=kwargs))
+
+
     def not_implemented(self, operation):
         """Return an HTTP 500 server error if an operation is called which has been allowed by
         allowed_methods, but which has not been implemented."""
@@ -191,22 +191,21 @@ class Resource(object):
 
 
     def authenticate(self, request):
-        """Attempt to authenticate the request, returning an authentication context or None"""
+        """Attempt to authenticate the request, returning an authentication context or None.
+        An authentication context may be any object, although in many cases it will be a User instance."""
+        
+        # Attempt authentication against each authenticator in turn,
+        # and return None if no authenticators succeed in authenticating the request.
         for authenticator in self.authenticators:
             auth_context = authenticator(self).authenticate(request)
             if auth_context:
                 return auth_context
+
         return None
 
 
     def check_method_allowed(self, method, auth):
-        """Ensure the request method is acceptable for this resource."""
-
-        # If anonoymous check permissions and bail with no further info if disallowed
-        if auth is None and not method in self.anon_allowed_methods:
-            raise ResponseException(status.HTTP_403_FORBIDDEN,
-                                    {'detail': 'You do not have permission to access this resource. ' +
-                                     'You may need to login or otherwise authenticate the request.'})
+        """Ensure the request method is permitted for this resource, raising a ResourceException if it is not."""
 
         if not method in self.callmap.keys():
             raise ResponseException(status.HTTP_501_NOT_IMPLEMENTED,
@@ -216,13 +215,17 @@ class Resource(object):
             raise ResponseException(status.HTTP_405_METHOD_NOT_ALLOWED,
                                     {'detail': 'Method \'%s\' not allowed on this resource.' % method})
 
+        if auth is None and not method in self.anon_allowed_methods:
+            raise ResponseException(status.HTTP_403_FORBIDDEN,
+                                    {'detail': 'You do not have permission to access this resource. ' +
+                                     'You may need to login or otherwise authenticate the request.'})
 
     def get_form(self, data=None):
         """Optionally return a Django Form instance, which may be used for validation
         and/or rendered by an HTML/XHTML emitter.
         
-        If data is not None the form will be bound to data.  is_response indicates if data should be
-        treated as the input data (bind to client input) or the response data (bind to an existing object)."""
+        If data is not None the form will be bound to data."""
+
         if self.form:
             if data:
                 return self.form(data)
@@ -238,6 +241,7 @@ class Resource(object):
         Returns a tuple containing the cleaned up data, and optionally a form bound to that data.
         
         By default this uses form validation to filter the basic input into the required types."""
+
         if form_instance is None:
             return data
         
@@ -253,7 +257,7 @@ class Resource(object):
                 
             else:
                 # Add standard field errors
-                details = dict((key, map(unicode, val)) for (key, val) in form_instance.errors.iteritems())
+                details = dict((key, map(unicode, val)) for (key, val) in form_instance.errors.iteritems() if key != '__all__')
 
                 # Add any non-field errors
                 if form_instance.non_field_errors():
@@ -363,7 +367,8 @@ class Resource(object):
 
 
     def _handle_request(self, request, *args, **kwargs):
-        """
+        """This method is the core of Resource, through which all requests are passed.
+
         Broadly this consists of the following procedure:
 
         0. ensure the operation is permitted

flywheel/templates/emitter.html

@@ -48,6 +48,7 @@
 		<div class='action'>
 			<form action="{{ request.path }}" method="post">
 			    {% csrf_token %}
+			    {{ form.non_field_errors }}
 				{% for field in form %}
 				<div>
 				    {{ field.label_tag }}:
@@ -67,6 +68,7 @@
 			<form action="{{ request.path }}" method="post">
 				<input type="hidden" name="{{ resource.METHOD_PARAM }}" value="PUT" />
 				{% csrf_token %}
+				{{ form.non_field_errors }}
 				{% for field in form %}
 				<div>
 				    {{ field.label_tag }}:

flywheel/templates/emitter.xhtml

-HTML:
-
-{{ content }}
\ No newline at end of file