Merge remote-tracking branch 'reference/master' into feature/django_1_7

docs/api-guide/exceptions.md

@@ -18,7 +18,7 @@ The handled exceptions are:
 
 In each case, REST framework will return a response with an appropriate status code and content-type.  The body of the response will include any additional details regarding the nature of the error.
 
-By default all error responses will include a key `details` in the body of the response, but other keys may also be included.
+By default all error responses will include a key `detail` in the body of the response, but other keys may also be included.
 
 For example, the following request:
 
@@ -86,7 +86,7 @@ Note that the exception handler will only be called for responses generated by r
 
 The **base class** for all exceptions raised inside REST framework.
 
-To provide a custom exception, subclass `APIException` and set the `.status_code` and `.detail` properties on the class.
+To provide a custom exception, subclass `APIException` and set the `.status_code` and `.default_detail` properties on the class.
 
 For example, if your API relies on a third party service that may sometimes be unreachable, you might want to implement an exception for the "503 Service Unavailable" HTTP response code.  You could do this like so:
 
@@ -94,7 +94,7 @@ For example, if your API relies on a third party service that may sometimes be u
 
     class ServiceUnavailable(APIException):
         status_code = 503
-        detail = 'Service temporarily unavailable, try again later.'
+        default_detail = 'Service temporarily unavailable, try again later.'
 
 ## ParseError
 

docs/api-guide/fields.md

@@ -104,6 +104,7 @@ A serializer definition that looked like this:
         expired = serializers.Field(source='has_expired')
         
         class Meta:
+            model = Account
             fields = ('url', 'owner', 'name', 'expired')
 
 Would produce output similar to:
@@ -125,7 +126,7 @@ A field that supports both read and write operations.  By itself `WritableField`
 
 ## ModelField
 
-A generic field that can be tied to any arbitrary model field.  The `ModelField` class delegates the task of serialization/deserialization to it's associated model field.  This field can be used to create serializer fields for custom model fields, without having to create a new custom serializer field.
+A generic field that can be tied to any arbitrary model field.  The `ModelField` class delegates the task of serialization/deserialization to its associated model field.  This field can be used to create serializer fields for custom model fields, without having to create a new custom serializer field.
 
 The `ModelField` class is generally intended for internal use, but can be used by your API if needed.  In order to properly instantiate a `ModelField`, it must be passed a field that is attached to an instantiated model.  For example: `ModelField(model_field=MyModel()._meta.get_field('custom_field'))`
 
@@ -307,7 +308,7 @@ Django's regular [FILE_UPLOAD_HANDLERS] are used for handling uploaded files.
 
 If you want to create a custom field, you'll probably want to override either one or both of the `.to_native()` and `.from_native()` methods.  These two methods are used to convert between the initial datatype, and a primitive, serializable datatype.  Primitive datatypes may be any of a number, string, date/time/datetime or None.  They may also be any list or dictionary like object that only contains other primitive objects.
 
-The `.to_native()` method is called to convert the initial datatype into a primitive, serializable datatype.  The `from_native()` method is called to restore a primitive datatype into it's initial representation.
+The `.to_native()` method is called to convert the initial datatype into a primitive, serializable datatype.  The `from_native()` method is called to restore a primitive datatype into its initial representation.
 
 ## Examples
 

docs/api-guide/generic-views.md

@@ -119,7 +119,7 @@ For example:
         self.check_object_permissions(self.request, obj)
         return obj
 
-Note that if your API doesn't include any object level permissions, you may optionally exclude the ``self.check_object_permissions, and simply return the object from the `get_object_or_404` lookup.
+Note that if your API doesn't include any object level permissions, you may optionally exclude the `self.check_object_permissions`, and simply return the object from the `get_object_or_404` lookup.
 
 #### `get_filter_backends(self)`
 

docs/api-guide/testing.md

@@ -218,12 +218,12 @@ You can use any of REST framework's test case classes as you would for the regul
 
 When checking the validity of test responses it's often more convenient to inspect the data that the response was created with, rather than inspecting the fully rendered response.
 
-For example, it's easier to inspect `request.data`:
+For example, it's easier to inspect `response.data`:
 
     response = self.client.get('/users/4/')
     self.assertEqual(response.data, {'id': 4, 'username': 'lauren'})
 
-Instead of inspecting the result of parsing `request.content`:
+Instead of inspecting the result of parsing `response.content`:
 
     response = self.client.get('/users/4/')
     self.assertEqual(json.loads(response.content), {'id': 4, 'username': 'lauren'})

docs/api-guide/throttling.md

@@ -150,7 +150,7 @@ For example, given the following views...
 
     REST_FRAMEWORK = {
         'DEFAULT_THROTTLE_CLASSES': (
-            'rest_framework.throttling.ScopedRateThrottle'
+            'rest_framework.throttling.ScopedRateThrottle',
         ),
         'DEFAULT_THROTTLE_RATES': {
             'contacts': '1000/day',

docs/api-guide/viewsets.md

@@ -225,7 +225,7 @@ To create a base viewset class that provides `create`, `list` and `retrieve` ope
                                     mixins.RetrieveModelMixin,
                                     viewsets.GenericViewSet):
         """
-        A viewset that provides `retrieve`, `update`, and `list` actions.
+        A viewset that provides `retrieve`, `create`, and `list` actions.
 
         To use it, override the class and set the `.queryset` and
         `.serializer_class` attributes.

docs/topics/contributing.md

@@ -60,7 +60,7 @@ To run the tests, clone the repository, and then:
 
     # Setup the virtual environment
     virtualenv env
-    env/bin/activate
+    source env/bin/activate
     pip install -r requirements.txt
     pip install -r optionals.txt
 

docs/topics/credits.md

@@ -182,6 +182,7 @@ The following people have helped make REST framework great.
 * Ian Foote - [ian-foote]
 * Chuck Harmston - [chuckharmston]
 * Philip Forget - [philipforget]
+* Artem Mezhenin - [amezhenin]
 
 Many thanks to everyone who's contributed to the project.
 
@@ -400,3 +401,4 @@ You can also contact [@_tomchristie][twitter] directly on twitter.
 [ian-foote]: https://github.com/ian-foote
 [chuckharmston]: https://github.com/chuckharmston
 [philipforget]: https://github.com/philipforget
+[amezhenin]: https://github.com/amezhenin

rest_framework/authtoken/models.py

-import uuid
-import hmac
+import binascii
+import os
 from hashlib import sha1
 from django.conf import settings
 from django.db import models
@@ -34,8 +34,7 @@ class Token(models.Model):
         return super(Token, self).save(*args, **kwargs)
 
     def generate_key(self):
-        unique = uuid.uuid4()
-        return hmac.new(unique.bytes, digestmod=sha1).hexdigest()
+        return binascii.hexlify(os.urandom(20))
 
     def __unicode__(self):
         return self.key

rest_framework/compat.py

@@ -457,7 +457,7 @@ from django.test.client import RequestFactory as DjangoRequestFactory
 from django.test.client import FakePayload
 try:
     # In 1.5 the test client uses force_bytes
-    from django.utils.encoding import force_bytes_or_smart_bytes
+    from django.utils.encoding import force_bytes as force_bytes_or_smart_bytes
 except ImportError:
     # In 1.3 and 1.4 the test client just uses smart_str
     from django.utils.encoding import smart_str as force_bytes_or_smart_bytes

rest_framework/exceptions.py

@@ -12,7 +12,7 @@ import math
 class APIException(Exception):
     """
     Base class for REST framework exceptions.
-    Subclasses should provide `.status_code` and `.detail` properties.
+    Subclasses should provide `.status_code` and `.default_detail` properties.
     """
     status_code = status.HTTP_500_INTERNAL_SERVER_ERROR
     default_detail = ''

rest_framework/fields.py

@@ -477,7 +477,8 @@ class URLField(CharField):
     type_label = 'url'
 
     def __init__(self, **kwargs):
-        kwargs['validators'] = [validators.URLValidator()]
+        if not 'validators' in kwargs:
+            kwargs['validators'] = [validators.URLValidator()]
         super(URLField, self).__init__(**kwargs)
 
 

rest_framework/renderers.py

@@ -10,6 +10,7 @@ from __future__ import unicode_literals
 
 import copy
 import json
+import django
 from django import forms
 from django.core.exceptions import ImproperlyConfigured
 from django.http.multipartparser import parse_header
@@ -597,7 +598,7 @@ class MultiPartRenderer(BaseRenderer):
     media_type = 'multipart/form-data; boundary=BoUnDaRyStRiNg'
     format = 'multipart'
     charset = 'utf-8'
-    BOUNDARY = 'BoUnDaRyStRiNg'
+    BOUNDARY = 'BoUnDaRyStRiNg' if django.VERSION >= (1, 5) else b'BoUnDaRyStRiNg'
 
     def render(self, data, accepted_media_type=None, renderer_context=None):
         return encode_multipart(self.BOUNDARY, data)

rest_framework/serializers.py

@@ -501,7 +501,7 @@ class BaseSerializer(WritableField):
             else:
                 many = hasattr(data, '__iter__') and not isinstance(data, (Page, dict, six.text_type))
                 if many:
-                    warnings.warn('Implict list/queryset serialization is deprecated. '
+                    warnings.warn('Implicit list/queryset serialization is deprecated. '
                                   'Use the `many=True` flag when instantiating the serializer.',
                                   DeprecationWarning, stacklevel=3)
 
@@ -563,7 +563,7 @@ class BaseSerializer(WritableField):
             else:
                 many = hasattr(obj, '__iter__') and not isinstance(obj, (Page, dict))
                 if many:
-                    warnings.warn('Implict list/queryset serialization is deprecated. '
+                    warnings.warn('Implicit list/queryset serialization is deprecated. '
                                   'Use the `many=True` flag when instantiating the serializer.',
                                   DeprecationWarning, stacklevel=2)
 
@@ -893,6 +893,7 @@ class ModelSerializer(Serializer):
             field_name = field.source or field_name
             if field_name in exclusions \
                 and not field.read_only \
+                and field.required \
                 and not isinstance(field, Serializer):
                 exclusions.remove(field_name)
         return exclusions

rest_framework/templatetags/rest_framework.py

@@ -6,7 +6,7 @@ from django.utils.encoding import iri_to_uri
 from django.utils.html import escape
 from django.utils.safestring import SafeData, mark_safe
 from rest_framework.compat import urlparse, force_text, six, smart_urlquote
-import re, string
+import re
 
 register = template.Library()
 
@@ -189,6 +189,17 @@ simple_url_2_re = re.compile(r'^www\.|^(?!http)\w[^@]+\.(com|edu|gov|int|mil|net
 simple_email_re = re.compile(r'^\S+@\S+\.\S+$')
 
 
+def smart_urlquote_wrapper(matched_url):
+    """
+    Simple wrapper for smart_urlquote. ValueError("Invalid IPv6 URL") can
+    be raised here, see issue #1386
+    """
+    try:
+        return smart_urlquote(matched_url)
+    except ValueError:
+        return None
+
+
 @register.filter
 def urlize_quoted_links(text, trim_url_limit=None, nofollow=True, autoescape=True):
     """
@@ -211,7 +222,6 @@ def urlize_quoted_links(text, trim_url_limit=None, nofollow=True, autoescape=Tru
     safe_input = isinstance(text, SafeData)
     words = word_split_re.split(force_text(text))
     for i, word in enumerate(words):
-        match = None
         if '.' in word or '@' in word or ':' in word:
             # Deal with punctuation.
             lead, middle, trail = '', word, ''
@@ -233,9 +243,9 @@ def urlize_quoted_links(text, trim_url_limit=None, nofollow=True, autoescape=Tru
             url = None
             nofollow_attr = ' rel="nofollow"' if nofollow else ''
             if simple_url_re.match(middle):
-                url = smart_urlquote(middle)
+                url = smart_urlquote_wrapper(middle)
             elif simple_url_2_re.match(middle):
-                url = smart_urlquote('http://%s' % middle)
+                url = smart_urlquote_wrapper('http://%s' % middle)
             elif not ':' in middle and simple_email_re.match(middle):
                 local, domain = middle.rsplit('@', 1)
                 try:

rest_framework/tests/test_fields.py

@@ -860,7 +860,9 @@ class SlugFieldTests(TestCase):
 
 class URLFieldTests(TestCase):
     """
-    Tests for URLField attribute values
+    Tests for URLField attribute values.
+
+    (Includes test for #1210, checking that validators can be overridden.)
     """
 
     class URLFieldModel(RESTFrameworkModel):
@@ -902,6 +904,11 @@ class URLFieldTests(TestCase):
         self.assertEqual(getattr(serializer.fields['url_field'],
                          'max_length'), 20)
 
+    def test_validators_can_be_overridden(self):
+        url_field = serializers.URLField(validators=[])
+        validators = url_field.validators
+        self.assertEqual([], validators, 'Passing `validators` kwarg should have overridden default validators')
+
 
 class FieldMetadata(TestCase):
     def setUp(self):

rest_framework/tests/test_serializer.py

@@ -91,6 +91,15 @@ class ActionItemSerializer(serializers.ModelSerializer):
     class Meta:
         model = ActionItem
 
+class ActionItemSerializerOptionalFields(serializers.ModelSerializer):
+    """
+    Intended to test that fields with `required=False` are excluded from validation.
+    """
+    title = serializers.CharField(required=False)
+
+    class Meta:
+        model = ActionItem
+        fields = ('title',)
 
 class ActionItemSerializerCustomRestore(serializers.ModelSerializer):
 
@@ -308,7 +317,13 @@ class BasicTests(TestCase):
         serializer.save()
         self.assertIsNotNone(serializer.data.get('id',None), 'Model is saved. `id` should be set.')
 
-
+    def test_fields_marked_as_not_required_are_excluded_from_validation(self):
+        """
+        Check that fields with `required=False` are included in list of exclusions.
+        """
+        serializer = ActionItemSerializerOptionalFields(self.actionitem)
+        exclusions = serializer.get_validation_exclusions()
+        self.assertTrue('title' in exclusions, '`title` field was marked `required=False` and should be excluded')
 
 
 class DictStyleSerializer(serializers.Serializer):
@@ -1811,14 +1826,14 @@ class SerializerDefaultTrueBoolean(TestCase):
         self.assertEqual(serializer.data['cat'], False)
         self.assertEqual(serializer.data['dog'], False)
 
-        
+
 class BoolenFieldTypeTest(TestCase):
     '''
     Ensure the various Boolean based model fields are rendered as the proper
     field type
-    
+
     '''
-    
+
     def setUp(self):
         '''
         Setup an ActionItemSerializer for BooleanTesting
@@ -1834,11 +1849,11 @@ class BoolenFieldTypeTest(TestCase):
         '''
         bfield = self.serializer.get_fields()['done']
         self.assertEqual(type(bfield), fields.BooleanField)
-    
+
     def test_nullbooleanfield_type(self):
         '''
-        Test that BooleanField is infered from models.NullBooleanField 
-        
+        Test that BooleanField is infered from models.NullBooleanField
+
         https://groups.google.com/forum/#!topic/django-rest-framework/D9mXEftpuQ8
         '''
         bfield = self.serializer.get_fields()['started']

rest_framework/tests/test_templatetags.py

@@ -2,7 +2,7 @@
 from __future__ import unicode_literals
 from django.test import TestCase
 from rest_framework.test import APIRequestFactory
-from rest_framework.templatetags.rest_framework import add_query_param
+from rest_framework.templatetags.rest_framework import add_query_param, urlize_quoted_links
 
 factory = APIRequestFactory()
 
@@ -17,3 +17,35 @@ class TemplateTagTests(TestCase):
         json_url = add_query_param(request, "format", "json")
         self.assertIn("q=%E6%9F%A5%E8%AF%A2", json_url)
         self.assertIn("format=json", json_url)
+
+
+class Issue1386Tests(TestCase):
+    """
+    Covers #1386
+    """
+
+    def test_issue_1386(self):
+        """
+        Test function urlize_quoted_links with different args
+        """
+        correct_urls = [
+            "asdf.com",
+            "asdf.net",
+            "www.as_df.org",
+            "as.d8f.ghj8.gov",
+        ]
+        for i in correct_urls:
+            res = urlize_quoted_links(i)
+            self.assertNotEqual(res, i)
+            self.assertIn(i, res)
+
+        incorrect_urls = [
+            "mailto://asdf@fdf.com",
+            "asdf.netnet",
+        ]
+        for i in incorrect_urls:
+            res = urlize_quoted_links(i)
+            self.assertEqual(i, res)
+
+        # example from issue #1386, this shouldn't raise an exception
+        _ = urlize_quoted_links("asdf:[/p]zxcv.com")

rest_framework/tests/test_testing.py

 # -- coding: utf-8 --
 
 from __future__ import unicode_literals
+from io import BytesIO
+
 from django.contrib.auth.models import User
 from django.test import TestCase
 from rest_framework.compat import patterns, url
@@ -143,3 +145,10 @@ class TestAPIRequestFactory(TestCase):
         force_authenticate(request, user=user)
         response = view(request)
         self.assertEqual(response.data['user'], 'example')
+
+    def test_upload_file(self):
+        # This is a 1x1 black png
+        simple_png = BytesIO(b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01\x08\x06\x00\x00\x00\x1f\x15\xc4\x89\x00\x00\x00\rIDATx\x9cc````\x00\x00\x00\x05\x00\x01\xa5\xf6E@\x00\x00\x00\x00IEND\xaeB`\x82')
+        simple_png.name = 'test.png'
+        factory = APIRequestFactory()
+        factory.post('/', data={'image': simple_png})

rest_framework/views.py

@@ -112,12 +112,13 @@ class APIView(View):
 
     @property
     def default_response_headers(self):
-        # TODO: deprecate?
-        # TODO: Only vary by accept if multiple renderers
-        return {
+        headers = {
             'Allow': ', '.join(self.allowed_methods),
-            'Vary': 'Accept'
         }
+        if len(self.renderer_classes) > 1:
+            headers['Vary'] = 'Accept'
+        return headers
+
 
     def http_method_not_allowed(self, request, *args, **kwargs):
         """