Merge remote-tracking branch 'upstream/master' into null-one-to-one

README.md

@@ -87,10 +87,9 @@ To run the tests.
 
 * Added `PATCH` support.
 * Added `RetrieveUpdateAPIView`.
-* Relation changes are now persisted in `save` instead of in `.restore_object`.
+* Relation changes are now persisted in `.save` instead of in `.restore_object`.
 * Remove unused internal `save_m2m` flag on `ModelSerializer.save()`.
 * Tweak behavior of hyperlinked fields with an explicit format suffix.
-* Relation changes are now persisted in `.save()` instead of in `.restore_object()`.
 * Bugfix: Fix issue with FileField raising exception instead of validation error when files=None.
 * Bugfix: Partial updates should not set default values if field is not included.
 

docs/api-guide/pagination.md

@@ -97,6 +97,8 @@ You can also set the pagination style on a per-view basis, using the `ListAPIVie
         paginate_by = 10
         paginate_by_param = 'page_size'
 
+Note that using a `paginate_by` value of `None` will turn off pagination for the view.
+
 For more complex requirements such as serialization that differs depending on the requested media type you can override the `.get_paginate_by()` and `.get_pagination_serializer_class()` methods.
 
 ---

docs/api-guide/parsers.md

@@ -159,4 +159,17 @@ For example:
             files = {name: uploaded}
             return DataAndFiles(data, files)
 
+---
+
+# Third party packages
+
+The following third party packages are also available.
+
+## MessagePack
+
+[MessagePack][messagepack] is a fast, efficient binary serialization format.  [Juan Riaza][juanriaza] maintains the `djangorestframework-msgpack` package which provides MessagePack renderer and parser support for REST framework.  Documentation is [available here][djangorestframework-msgpack].
+
 [cite]: https://groups.google.com/d/topic/django-developers/dxI4qVzrBY4/discussion
+[messagepack]: https://github.com/juanriaza/django-rest-framework-msgpack
+[juanriaza]: https://github.com/juanriaza
+[djangorestframework-msgpack]: https://github.com/juanriaza/django-rest-framework-msgpack
\ No newline at end of file

docs/api-guide/renderers.md

@@ -271,6 +271,15 @@ Exceptions raised and handled by an HTML renderer will attempt to render using o
 
 Templates will render with a `RequestContext` which includes the `status_code` and `details` keys.
 
+---
+
+# Third party packages
+
+The following third party packages are also available.
+
+## MessagePack
+
+[MessagePack][messagepack] is a fast, efficient binary serialization format.  [Juan Riaza][juanriaza] maintains the `djangorestframework-msgpack` package which provides MessagePack renderer and parser support for REST framework.  Documentation is [available here][djangorestframework-msgpack].
 
 [cite]: https://docs.djangoproject.com/en/dev/ref/template-response/#the-rendering-process
 [conneg]: content-negotiation.md
@@ -281,3 +290,6 @@ Templates will render with a `RequestContext` which includes the `status_code` a
 [application/vnd.github+json]: http://developer.github.com/v3/media/
 [application/vnd.collection+json]: http://www.amundsen.com/media-types/collection/
 [django-error-views]: https://docs.djangoproject.com/en/dev/topics/http/views/#customizing-error-views
+[messagepack]: https://github.com/juanriaza/django-rest-framework-msgpack
+[juanriaza]: https://github.com/juanriaza
+[djangorestframework-msgpack]: https://github.com/juanriaza/django-rest-framework-msgpack
\ No newline at end of file

docs/api-guide/settings.md

@@ -65,7 +65,7 @@ Default:
 
     (
         'rest_framework.authentication.SessionAuthentication',
-        'rest_framework.authentication.UserBasicAuthentication'
+        'rest_framework.authentication.BasicAuthentication'
     )
 
 ## DEFAULT_PERMISSION_CLASSES
@@ -106,7 +106,7 @@ The default page size to use for pagination.  If set to `None`, pagination is di
 
 Default: `None`
 
-## PAGINATE_BY_KWARG
+## PAGINATE_BY_PARAM
 
 The name of a query parameter, which can be used by the client to overide the default page size to use for pagination.  If set to `None`, clients may not override the default page size.
 

docs/topics/credits.md

@@ -85,6 +85,9 @@ The following people have helped make REST framework great.
 * Toran Billups - [toranb]
 * Sébastien Béal - [sebastibe]
 * Andrew Hankinson - [ahankinson]
+* Juan Riaza - [juanriaza]
+* Michael Mior - [michaelmior]
+* Marc Tamlyn - [mjtamlyn]
 
 Many thanks to everyone who's contributed to the project.
 
@@ -205,3 +208,6 @@ You can also contact [@_tomchristie][twitter] directly on twitter.
 [toranb]: https://github.com/toranb
 [sebastibe]: https://github.com/sebastibe
 [ahankinson]: https://github.com/ahankinson
+[juanriaza]: https://github.com/juanriaza
+[michaelmior]: https://github.com/michaelmior
+[mjtamlyn]: https://github.com/mjtamlyn

docs/topics/release-notes.md

@@ -16,13 +16,18 @@ Major version numbers (x.0.0) are reserved for project milestones.  No major poi
 
 ## 2.1.x series
 
+### Master
+
+* Deprecate django.utils.simplejson in favor of Python 2.6's built-in json module.
+* Bugfix: Validation errors instead of exceptions when serializers receive incorrect types.
+* Bugfix: Validation errors instead of exceptions when related fields receive incorrect types.
+
 ### 2.1.15
 
 **Date**: 3rd Jan 2013
 
 * Added `PATCH` support.
 * Added `RetrieveUpdateAPIView`.
-* Relation changes are now persisted in `save` instead of in `.restore_object`.
 * Remove unused internal `save_m2m` flag on `ModelSerializer.save()`.
 * Tweak behavior of hyperlinked fields with an explicit format suffix.
 * Relation changes are now persisted in `.save()` instead of in `.restore_object()`.
@@ -37,9 +42,9 @@ Major version numbers (x.0.0) are reserved for project milestones.  No major poi
 * Bugfix: Model fields with `blank=True` are now `required=False` by default.
 * Bugfix: Nested serializers now support nullable relationships.
 
-**Note**: From 2.1.14 onwards, relational fields move out of the `fields.py` module and into the new `relations.py` module, in order to seperate them from regular data type fields, such as `CharField` and `IntegerField`.
+**Note**: From 2.1.14 onwards, relational fields move out of the `fields.py` module and into the new `relations.py` module, in order to separate them from regular data type fields, such as `CharField` and `IntegerField`.
 
-This change will not affect user code, so long as it's following the recommended import style of `from rest_framework import serializers` and refering to fields using the style `serializers.PrimaryKeyRelatedField`.
+This change will not affect user code, so long as it's following the recommended import style of `from rest_framework import serializers` and referring to fields using the style `serializers.PrimaryKeyRelatedField`.
 
 
 ### 2.1.13

rest_framework/authtoken/views.py

@@ -12,10 +12,11 @@ class ObtainAuthToken(APIView):
     permission_classes = ()
     parser_classes = (parsers.FormParser, parsers.MultiPartParser, parsers.JSONParser,)
     renderer_classes = (renderers.JSONRenderer,)
+    serializer_class = AuthTokenSerializer
     model = Token
 
     def post(self, request):
-        serializer = AuthTokenSerializer(data=request.DATA)
+        serializer = self.serializer_class(data=request.DATA)
         if serializer.is_valid():
             token, created = Token.objects.get_or_create(user=serializer.object['user'])
             return Response({'token': token.key})

rest_framework/parsers.py

@@ -8,11 +8,11 @@ on the request, such as form content or json encoded data.
 from django.http import QueryDict
 from django.http.multipartparser import MultiPartParser as DjangoMultiPartParser
 from django.http.multipartparser import MultiPartParserError
-from django.utils import simplejson as json
 from rest_framework.compat import yaml, ETParseError
 from rest_framework.exceptions import ParseError
 from xml.etree import ElementTree as ET
 from xml.parsers.expat import ExpatError
+import json
 import datetime
 import decimal
 

rest_framework/relations.py

@@ -4,6 +4,7 @@ from django import forms
 from django.forms import widgets
 from django.forms.models import ModelChoiceIterator
 from django.utils.encoding import smart_unicode
+from django.utils.translation import ugettext_lazy as _
 from rest_framework.fields import Field, WritableField
 from rest_framework.reverse import reverse
 from urlparse import urlparse
@@ -171,6 +172,11 @@ class PrimaryKeyRelatedField(RelatedField):
     default_read_only = False
     form_field_class = forms.ChoiceField
 
+    default_error_messages = {
+        'does_not_exist': _("Invalid pk '%s' - object does not exist."),
+        'invalid': _('Invalid value.'),
+    }
+
     # TODO: Remove these field hacks...
     def prepare_value(self, obj):
         return self.to_native(obj.pk)
@@ -196,7 +202,10 @@ class PrimaryKeyRelatedField(RelatedField):
         try:
             return self.queryset.get(pk=data)
         except ObjectDoesNotExist:
-            msg = "Invalid pk '%s' - object does not exist." % smart_unicode(data)
+            msg = self.error_messages['does_not_exist'] % smart_unicode(data)
+            raise ValidationError(msg)
+        except (TypeError, ValueError):
+            msg = self.error_messages['invalid']
             raise ValidationError(msg)
 
     def field_to_native(self, obj, field_name):
@@ -221,6 +230,11 @@ class ManyPrimaryKeyRelatedField(ManyRelatedField):
     default_read_only = False
     form_field_class = forms.MultipleChoiceField
 
+    default_error_messages = {
+        'does_not_exist': _("Invalid pk '%s' - object does not exist."),
+        'invalid': _('Invalid value.'),
+    }
+
     def prepare_value(self, obj):
         return self.to_native(obj.pk)
 
@@ -255,7 +269,10 @@ class ManyPrimaryKeyRelatedField(ManyRelatedField):
         try:
             return self.queryset.get(pk=data)
         except ObjectDoesNotExist:
-            msg = "Invalid pk '%s' - object does not exist." % smart_unicode(data)
+            msg = self.error_messages['does_not_exist'] % smart_unicode(data)
+            raise ValidationError(msg)
+        except (TypeError, ValueError):
+            msg = self.error_messages['invalid']
             raise ValidationError(msg)
 
 ### Slug relationships
@@ -265,6 +282,11 @@ class SlugRelatedField(RelatedField):
     default_read_only = False
     form_field_class = forms.ChoiceField
 
+    default_error_messages = {
+        'does_not_exist': _("Object with %s=%s does not exist."),
+        'invalid': _('Invalid value.'),
+    }
+
     def __init__(self, *args, **kwargs):
         self.slug_field = kwargs.pop('slug_field', None)
         assert self.slug_field, 'slug_field is required'
@@ -280,8 +302,11 @@ class SlugRelatedField(RelatedField):
         try:
             return self.queryset.get(**{self.slug_field: data})
         except ObjectDoesNotExist:
-            raise ValidationError('Object with %s=%s does not exist.' %
+            raise ValidationError(self.error_messages['does_not_exist'] %
                                   (self.slug_field, unicode(data)))
+        except (TypeError, ValueError):
+            msg = self.error_messages['invalid']
+            raise ValidationError(msg)
 
 
 class ManySlugRelatedField(ManyRelatedMixin, SlugRelatedField):
@@ -300,6 +325,14 @@ class HyperlinkedRelatedField(RelatedField):
     default_read_only = False
     form_field_class = forms.ChoiceField
 
+    default_error_messages = {
+        'no_match': _('Invalid hyperlink - No URL match'),
+        'incorrect_match': _('Invalid hyperlink - Incorrect URL match'),
+        'configuration_error': _('Invalid hyperlink due to configuration error'),
+        'does_not_exist': _("Invalid hyperlink - object does not exist."),
+        'invalid': _('Invalid value.'),
+    }
+
     def __init__(self, *args, **kwargs):
         try:
             self.view_name = kwargs.pop('view_name')
@@ -336,21 +369,21 @@ class HyperlinkedRelatedField(RelatedField):
         slug = getattr(obj, self.slug_field, None)
 
         if not slug:
-            raise ValidationError('Could not resolve URL for field using view name "%s"' % view_name)
+            raise Exception('Could not resolve URL for field using view name "%s"' % view_name)
 
         kwargs = {self.slug_url_kwarg: slug}
         try:
-            return reverse(self.view_name, kwargs=kwargs, request=request, format=format)
+            return reverse(view_name, kwargs=kwargs, request=request, format=format)
         except:
             pass
 
         kwargs = {self.pk_url_kwarg: obj.pk, self.slug_url_kwarg: slug}
         try:
-            return reverse(self.view_name, kwargs=kwargs, request=request, format=format)
+            return reverse(view_name, kwargs=kwargs, request=request, format=format)
         except:
             pass
 
-        raise ValidationError('Could not resolve URL for field using view name "%s"' % view_name)
+        raise Exception('Could not resolve URL for field using view name "%s"' % view_name)
 
     def from_native(self, value):
         # Convert URL -> model instance pk
@@ -358,7 +391,13 @@ class HyperlinkedRelatedField(RelatedField):
         if self.queryset is None:
             raise Exception('Writable related fields must include a `queryset` argument')
 
-        if value.startswith('http:') or value.startswith('https:'):
+        try:
+            http_prefix = value.startswith('http:') or value.startswith('https:')
+        except AttributeError:
+            msg = self.error_messages['invalid']
+            raise ValidationError(msg)
+
+        if http_prefix:
             # If needed convert absolute URLs to relative path
             value = urlparse(value).path
             prefix = get_script_prefix()
@@ -368,10 +407,10 @@ class HyperlinkedRelatedField(RelatedField):
         try:
             match = resolve(value)
         except:
-            raise ValidationError('Invalid hyperlink - No URL match')
+            raise ValidationError(self.error_messages['no_match'])
 
-        if match.url_name != self.view_name:
-            raise ValidationError('Invalid hyperlink - Incorrect URL match')
+        if match.view_name != self.view_name:
+            raise ValidationError(self.error_messages['incorrect_match'])
 
         pk = match.kwargs.get(self.pk_url_kwarg, None)
         slug = match.kwargs.get(self.slug_url_kwarg, None)
@@ -383,14 +422,18 @@ class HyperlinkedRelatedField(RelatedField):
         elif slug is not None:
             slug_field = self.get_slug_field()
             queryset = self.queryset.filter(**{slug_field: slug})
-        # If none of those are defined, it's an error.
+        # If none of those are defined, it's probably a configuation error.
         else:
-            raise ValidationError('Invalid hyperlink')
+            raise ValidationError(self.error_messages['configuration_error'])
 
         try:
             obj = queryset.get()
         except ObjectDoesNotExist:
-            raise ValidationError('Invalid hyperlink - object does not exist.')
+            raise ValidationError(self.error_messages['does_not_exist'])
+        except (TypeError, ValueError):
+            msg = self.error_messages['invalid']
+            raise ValidationError(msg)
+
         return obj
 
 
@@ -449,18 +492,18 @@ class HyperlinkedIdentityField(Field):
         slug = getattr(obj, self.slug_field, None)
 
         if not slug:
-            raise ValidationError('Could not resolve URL for field using view name "%s"' % view_name)
+            raise Exception('Could not resolve URL for field using view name "%s"' % view_name)
 
         kwargs = {self.slug_url_kwarg: slug}
         try:
-            return reverse(self.view_name, kwargs=kwargs, request=request, format=format)
+            return reverse(view_name, kwargs=kwargs, request=request, format=format)
         except:
             pass
 
         kwargs = {self.pk_url_kwarg: obj.pk, self.slug_url_kwarg: slug}
         try:
-            return reverse(self.view_name, kwargs=kwargs, request=request, format=format)
+            return reverse(view_name, kwargs=kwargs, request=request, format=format)
         except:
             pass
 
-        raise ValidationError('Could not resolve URL for field using view name "%s"' % view_name)
+        raise Exception('Could not resolve URL for field using view name "%s"' % view_name)

rest_framework/renderers.py

@@ -8,10 +8,10 @@ REST framework also provides an HTML renderer the renders the browsable API.
 """
 import copy
 import string
+import json
 from django import forms
 from django.http.multipartparser import parse_header
 from django.template import RequestContext, loader, Template
-from django.utils import simplejson as json
 from rest_framework.compat import yaml
 from rest_framework.exceptions import ConfigurationError
 from rest_framework.settings import api_settings

rest_framework/serializers.py

@@ -208,6 +208,11 @@ class BaseSerializer(Field):
         Converts a dictionary of data into a dictionary of deserialized fields.
         """
         reverted_data = {}
+
+        if data is not None and not isinstance(data, dict):
+            self._errors['non_field_errors'] = [u'Invalid data']
+            return None
+
         for field_name, field in self.fields.items():
             field.initialize(parent=self, field_name=field_name)
             try:
@@ -276,7 +281,7 @@ class BaseSerializer(Field):
         """
         if hasattr(data, '__iter__') and not isinstance(data, dict):
             # TODO: error data when deserializing lists
-            return (self.from_native(item) for item in data)
+            return [self.from_native(item, None) for item in data]
 
         self._errors = {}
         if data is not None or files is not None:
@@ -428,7 +433,7 @@ class ModelSerializer(Serializer):
         # TODO: filter queryset using:
         # .using(db).complex_filter(self.rel.limit_choices_to)
         kwargs = {
-            'null': model_field.null,
+            'null': model_field.null or model_field.blank,
             'queryset': model_field.rel.to._default_manager
         }
 

rest_framework/settings.py

@@ -115,8 +115,8 @@ def import_from_string(val, setting_name):
         module_path, class_name = '.'.join(parts[:-1]), parts[-1]
         module = importlib.import_module(module_path)
         return getattr(module, class_name)
-    except:
-        msg = "Could not import '%s' for API setting '%s'" % (val, setting_name)
+    except ImportError as e:
+        msg = "Could not import '%s' for API setting '%s'. %s: %s." % (val, setting_name, e.__class__.__name__, e)
         raise ImportError(msg)
 
 

rest_framework/templatetags/rest_framework.py

@@ -23,7 +23,7 @@ register = template.Library()
 # conflicts with this rest_framework template tag module.
 
 try:  # Django 1.5+
-    from django.contrib.staticfiles.templatetags import StaticFilesNode
+    from django.contrib.staticfiles.templatetags.staticfiles import StaticFilesNode
 
     @register.tag('static')
     def do_static(parser, token):

rest_framework/tests/authentication.py

 from django.contrib.auth.models import User
 from django.http import HttpResponse
 from django.test import Client, TestCase
-from django.utils import simplejson as json
 
 from rest_framework import permissions
 from rest_framework.authtoken.models import Token
@@ -9,6 +8,7 @@ from rest_framework.authentication import TokenAuthentication
 from rest_framework.compat import patterns
 from rest_framework.views import APIView
 
+import json
 import base64
 
 

rest_framework/tests/extras/bad_import.py

+raise ValueError

rest_framework/tests/generics.py

+import json
 from django.db import models
 from django.test import TestCase
-from django.utils import simplejson as json
 from rest_framework import generics, serializers, status
 from rest_framework.tests.utils import RequestFactory
 from rest_framework.tests.models import BasicModel, Comment, SlugBasedModel

rest_framework/tests/hyperlinkedserializers.py

+import json
 from django.test import TestCase
 from django.test.client import RequestFactory
-from django.utils import simplejson as json
 from rest_framework import generics, status, serializers
 from rest_framework.compat import patterns, url
 from rest_framework.tests.models import Anchor, BasicModel, ManyToManyModel, BlogPost, BlogPostComment, Album, Photo, OptionalRelationModel

rest_framework/tests/pagination.py

@@ -181,10 +181,10 @@ class UnitTestPagination(TestCase):
         """
         Ensure context gets passed through to the object serializer.
         """
-        serializer = PassOnContextPaginationSerializer(self.first_page)
+        serializer = PassOnContextPaginationSerializer(self.first_page, context={'foo': 'bar'})
         serializer.data
         results = serializer.fields[serializer.results_field]
-        self.assertTrue(serializer.context is results.context)
+        self.assertEquals(serializer.context, results.context)
 
 
 class TestUnpaginated(TestCase):

rest_framework/tests/relations.py

+"""
+General tests for relational fields.
+"""
+
+from django.db import models
+from django.test import TestCase
+from rest_framework import serializers
+
+
+class NullModel(models.Model):
+    pass
+
+
+class FieldTests(TestCase):
+    def test_pk_related_field_with_empty_string(self):
+        """
+        Regression test for #446
+
+        https://github.com/tomchristie/django-rest-framework/issues/446
+        """
+        field = serializers.PrimaryKeyRelatedField(queryset=NullModel.objects.all())
+        self.assertRaises(serializers.ValidationError, field.from_native, '')
+        self.assertRaises(serializers.ValidationError, field.from_native, [])
+
+    def test_hyperlinked_related_field_with_empty_string(self):
+        field = serializers.HyperlinkedRelatedField(queryset=NullModel.objects.all(), view_name='')
+        self.assertRaises(serializers.ValidationError, field.from_native, '')
+        self.assertRaises(serializers.ValidationError, field.from_native, [])
+
+    def test_slug_related_field_with_empty_string(self):
+        field = serializers.SlugRelatedField(queryset=NullModel.objects.all(), slug_field='pk')
+        self.assertRaises(serializers.ValidationError, field.from_native, '')
+        self.assertRaises(serializers.ValidationError, field.from_native, [])

rest_framework/tests/request.py

 """
 Tests for content parsing, and form-overloaded content parsing.
 """
+import json
 from django.contrib.auth.models import User
 from django.contrib.auth import authenticate, login, logout
 from django.contrib.sessions.middleware import SessionMiddleware
 from django.test import TestCase, Client
 from django.test.client import RequestFactory
-from django.utils import simplejson as json
 from rest_framework import status
 from rest_framework.authentication import SessionAuthentication
 from rest_framework.compat import patterns

rest_framework/tests/serializer.py

@@ -69,6 +69,7 @@ class AlbumsSerializer(serializers.ModelSerializer):
         model = Album
         fields = ['title']  # lists are also valid options
 
+
 class PositiveIntegerAsChoiceSerializer(serializers.ModelSerializer):
     class Meta:
         model = HasPositiveIntegerAsChoice
@@ -240,6 +241,25 @@ class ValidationTests(TestCase):
         self.assertFalse(serializer.is_valid())
         self.assertEquals(serializer.errors, {'content': [u'Test not in value']})
 
+    def test_bad_type_data_is_false(self):
+        """
+        Data of the wrong type is not valid.
+        """
+        data = ['i am', 'a', 'list']
+        serializer = CommentSerializer(self.comment, data=data)
+        self.assertEquals(serializer.is_valid(), False)
+        self.assertEquals(serializer.errors, {'non_field_errors': [u'Invalid data']})
+
+        data = 'and i am a string'
+        serializer = CommentSerializer(self.comment, data=data)
+        self.assertEquals(serializer.is_valid(), False)
+        self.assertEquals(serializer.errors, {'non_field_errors': [u'Invalid data']})
+
+        data = 42
+        serializer = CommentSerializer(self.comment, data=data)
+        self.assertEquals(serializer.is_valid(), False)
+        self.assertEquals(serializer.errors, {'non_field_errors': [u'Invalid data']})
+
     def test_cross_field_validation(self):
 
         class CommentSerializerWithCrossFieldValidator(CommentSerializer):

rest_framework/tests/settings.py

+"""Tests for the settings module"""
+from django.test import TestCase
+
+from rest_framework.settings import APISettings, DEFAULTS, IMPORT_STRINGS
+
+
+class TestSettings(TestCase):
+    """Tests relating to the api settings"""
+
+    def test_non_import_errors(self):
+        """Make sure other errors aren't suppressed."""
+        settings = APISettings({'DEFAULT_MODEL_SERIALIZER_CLASS': 'rest_framework.tests.extras.bad_import.ModelSerializer'}, DEFAULTS, IMPORT_STRINGS)
+        with self.assertRaises(ValueError):
+            settings.DEFAULT_MODEL_SERIALIZER_CLASS
+
+    def test_import_error_message_maintained(self):
+        """Make sure real import errors are captured and raised sensibly."""
+        settings = APISettings({'DEFAULT_MODEL_SERIALIZER_CLASS': 'rest_framework.tests.extras.not_here.ModelSerializer'}, DEFAULTS, IMPORT_STRINGS)
+        with self.assertRaises(ImportError) as cm:
+            settings.DEFAULT_MODEL_SERIALIZER_CLASS
+        self.assertTrue('ImportError' in str(cm.exception))

rest_framework/utils/encoders.py

@@ -4,7 +4,7 @@ Helper classes for parsers.
 import datetime
 import decimal
 import types
-from django.utils import simplejson as json
+import json
 from django.utils.datastructures import SortedDict
 from rest_framework.compat import timezone
 from rest_framework.serializers import DictWithMetadata, SortedDictWithMetadata