Skip to content

D
django-rest-framework
Commit f19e0d54 by Tom Christie
Options

Fix charset issues

parent 6fcffcc9
Showing with 92 additions and 36 deletions
docs/api-guide/renderers.md
......@@ -67,7 +67,7 @@ If your API includes views that can serve both regular webpages and API response
## JSONRenderer
Renders the request data into `JSON` enforcing ASCII encoding
Renders the request data into `JSON`, using ASCII encoding.
The client may additionally include an `'indent'` media type parameter, in which case the returned `JSON` will be indented. For example `Accept: application/json; indent=4`.
......@@ -75,9 +75,19 @@ The client may additionally include an `'indent'` media type parameter, in which
**.format**: `'.json'`
**.charset**: `iso-8859-1`
## UnicodeJSONRenderer
Same as `JSONRenderer` but doesn't enforce ASCII encoding
Renders the request data into `JSON`, using utf-8 encoding.
The client may additionally include an `'indent'` media type parameter, in which case the returned `JSON` will be indented. For example `Accept: application/json; indent=4`.
**.media_type**: `application/json`
**.format**: `'.json'`
**.charset**: `utf-8`
## JSONPRenderer
......@@ -91,6 +101,8 @@ The javascript callback function must be set by the client including a `callback
**.format**: `'.jsonp'`
**.charset**: `iso-8859-1`
## YAMLRenderer
Renders the request data into `YAML`.
......@@ -101,6 +113,8 @@ Requires the `pyyaml` package to be installed.
**.format**: `'.yaml'`
**.charset**: `utf-8`
## XMLRenderer
Renders REST framework's default style of `XML` response content.
......@@ -113,6 +127,8 @@ If you are considering using `XML` for your API, you may want to consider implem
**.format**: `'.xml'`
**.charset**: `utf-8`
## TemplateHTMLRenderer
Renders data to HTML, using Django's standard template rendering.
......@@ -147,6 +163,8 @@ If you're building websites that use `TemplateHTMLRenderer` along with other ren
**.format**: `'.html'`
**.charset**: `utf-8`
See also: `StaticHTMLRenderer`
## StaticHTMLRenderer
......@@ -167,6 +185,8 @@ You can use `TemplateHTMLRenderer` either to return regular HTML pages using RES
**.format**: `'.html'`
**.charset**: `utf-8`
See also: `TemplateHTMLRenderer`
## BrowsableAPIRenderer
......@@ -177,12 +197,16 @@ Renders data into HTML for the Browsable API. This renderer will determine whic
**.format**: `'.api'`
**.charset**: `utf-8`
---
# Custom renderers
To implement a custom renderer, you should override `BaseRenderer`, set the `.media_type` and `.format` properties, and implement the `.render(self, data, media_type=None, renderer_context=None)` method.
The method should return a bytestring, which wil be used as the body of the HTTP response.
The arguments passed to the `.render()` method are:
### `data`
......@@ -209,14 +233,34 @@ The following is an example plaintext renderer that will return a response with
from rest_framework import renderers
class PlainText(renderers.BaseRenderer):
class PlainTextRenderer(renderers.BaseRenderer):
media_type = 'text/plain'
format = 'txt'
def render(self, data, media_type=None, renderer_context=None):
if isinstance(data, basestring):
return data
return smart_unicode(data)
return data.encode(self.charset)
## Setting the character set
By default renderer classes are assumed to be using the `UTF-8` encoding. To use a different encoding, set the `charset` attribute on the renderer.
class PlainTextRenderer(renderers.BaseRenderer):
media_type = 'text/plain'
format = 'txt'
charset = 'iso-8859-1'
def render(self, data, media_type=None, renderer_context=None):
return data.encode(self.charset)
If the renderer returns a raw bytestring, you should set a charset value of `None`, which will ensure the `Content-Type` header of the response will not have a `charset` value set. Doing so will also ensure that the browsable API will not attempt to display the binary content as a string.
class JPEGRenderer(renderers.BaseRenderer):
media_type = 'image/jpeg'
format = 'jpg'
charset = None
def render(self, data, media_type=None, renderer_context=None):
return data
---
......@@ -286,11 +330,11 @@ Templates will render with a `RequestContext` which includes the `status_code` a
The following third party packages are also available.
## MessagePack
### MessagePack
[MessagePack][messagepack] is a fast, efficient binary serialization format. [Juan Riaza][juanriaza] maintains the [djangorestframework-msgpack][djangorestframework-msgpack] package which provides MessagePack renderer and parser support for REST framework.
## CSV
### CSV
Comma-separated values are a plain-text tabular data format, that can be easily imported into spreadsheet applications. [Mjumbe Poe][mjumbewu] maintains the [djangorestframework-csv][djangorestframework-csv] package which provides CSV renderer support for REST framework.
......
rest_framework/renderers.py
......@@ -9,7 +9,6 @@ REST framework also provides an HTML renderer the renders the browsable API.
from __future__ import unicode_literals
import copy
import string
import json
from django import forms
from django.http.multipartparser import parse_header
......@@ -36,7 +35,7 @@ class BaseRenderer(object):
media_type = None
format = None
charset = None
charset = 'utf-8'
def render(self, data, accepted_media_type=None, renderer_context=None):
raise NotImplemented('Renderer class requires .render() to be implemented')
......@@ -51,6 +50,7 @@ class JSONRenderer(BaseRenderer):
format = 'json'
encoder_class = encoders.JSONEncoder
ensure_ascii = True
charset = 'iso-8859-1'
def render(self, data, accepted_media_type=None, renderer_context=None):
"""
......@@ -74,7 +74,12 @@ class JSONRenderer(BaseRenderer):
except (ValueError, TypeError):
indent = None
return json.dumps(data, cls=self.encoder_class, indent=indent, ensure_ascii=self.ensure_ascii)
ret = json.dumps(data, cls=self.encoder_class,
indent=indent, ensure_ascii=self.ensure_ascii)
if not self.ensure_ascii:
return bytes(ret.encode(self.charset))
return ret
class UnicodeJSONRenderer(JSONRenderer):
......@@ -332,7 +337,7 @@ class BrowsableAPIRenderer(BaseRenderer):
renderer_context['indent'] = 4
content = renderer.render(data, accepted_media_type, renderer_context)
if not all(char in string.printable for char in content):
if renderer.charset is None:
return '[%d bytes of binary content]' % len(content)
return content
......
rest_framework/response.py
......@@ -55,7 +55,11 @@ class Response(SimpleTemplateResponse):
else:
content_type = media_type
self['Content-Type'] = content_type
return renderer.render(self.data, media_type, context)
ret = renderer.render(self.data, media_type, context)
if isinstance(ret, six.text_type):
return bytes(ret.encode(self.charset))
return ret
@property
def status_text(self):
......
rest_framework/tests/renderers.py
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from decimal import Decimal
from django.core.cache import cache
from django.test import TestCase
......@@ -135,7 +137,7 @@ class RendererEndToEndTests(TestCase):
def test_default_renderer_serializes_content(self):
"""If the Accept header is not set the default renderer should serialize the response."""
resp = self.client.get('/')
self.assertEqual(resp['Content-Type'], RendererA.media_type)
self.assertEqual(resp['Content-Type'], RendererA.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_A_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -143,13 +145,13 @@ class RendererEndToEndTests(TestCase):
"""No response must be included in HEAD requests."""
resp = self.client.head('/')
self.assertEqual(resp.status_code, DUMMYSTATUS)
self.assertEqual(resp['Content-Type'], RendererA.media_type)
self.assertEqual(resp['Content-Type'], RendererA.media_type + '; charset=utf-8')
self.assertEqual(resp.content, six.b(''))
def test_default_renderer_serializes_content_on_accept_any(self):
"""If the Accept header is set to */* the default renderer should serialize the response."""
resp = self.client.get('/', HTTP_ACCEPT='*/*')
self.assertEqual(resp['Content-Type'], RendererA.media_type)
self.assertEqual(resp['Content-Type'], RendererA.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_A_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -157,7 +159,7 @@ class RendererEndToEndTests(TestCase):
"""If the Accept header is set the specified renderer should serialize the response.
(In this case we check that works for the default renderer)"""
resp = self.client.get('/', HTTP_ACCEPT=RendererA.media_type)
self.assertEqual(resp['Content-Type'], RendererA.media_type)
self.assertEqual(resp['Content-Type'], RendererA.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_A_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -165,7 +167,7 @@ class RendererEndToEndTests(TestCase):
"""If the Accept header is set the specified renderer should serialize the response.
(In this case we check that works for a non-default renderer)"""
resp = self.client.get('/', HTTP_ACCEPT=RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_B_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -176,7 +178,7 @@ class RendererEndToEndTests(TestCase):
RendererB.media_type
)
resp = self.client.get('/' + param)
self.assertEqual(resp['Content-Type'], RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_B_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -193,7 +195,7 @@ class RendererEndToEndTests(TestCase):
RendererB.format
)
resp = self.client.get('/' + param)
self.assertEqual(resp['Content-Type'], RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_B_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -201,7 +203,7 @@ class RendererEndToEndTests(TestCase):
"""If a 'format' keyword arg is specified, the renderer with the matching
format attribute should serialize the response."""
resp = self.client.get('/something.formatb')
self.assertEqual(resp['Content-Type'], RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_B_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -214,7 +216,7 @@ class RendererEndToEndTests(TestCase):
)
resp = self.client.get('/' + param,
HTTP_ACCEPT=RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_B_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -270,7 +272,7 @@ class UnicodeJSONRendererTests(TestCase):
obj = {'countries': ['United Kingdom', 'France', 'España']}
renderer = UnicodeJSONRenderer()
content = renderer.render(obj, 'application/json')
self.assertEqual(content, '{"countries": ["United Kingdom", "France", "España"]}')
self.assertEqual(content, '{"countries": ["United Kingdom", "France", "España"]}'.encode('utf-8'))
class JSONPRendererTests(TestCase):
......@@ -287,7 +289,7 @@ class JSONPRendererTests(TestCase):
resp = self.client.get('/jsonp/jsonrenderer',
HTTP_ACCEPT='application/javascript')
self.assertEqual(resp.status_code, status.HTTP_200_OK)
self.assertEqual(resp['Content-Type'], 'application/javascript')
self.assertEqual(resp['Content-Type'], 'application/javascript; charset=iso-8859-1')
self.assertEqual(resp.content,
('callback(%s);' % _flat_repr).encode('ascii'))
......@@ -298,7 +300,7 @@ class JSONPRendererTests(TestCase):
resp = self.client.get('/jsonp/nojsonrenderer',
HTTP_ACCEPT='application/javascript')
self.assertEqual(resp.status_code, status.HTTP_200_OK)
self.assertEqual(resp['Content-Type'], 'application/javascript')
self.assertEqual(resp['Content-Type'], 'application/javascript; charset=iso-8859-1')
self.assertEqual(resp.content,
('callback(%s);' % _flat_repr).encode('ascii'))
......@@ -310,7 +312,7 @@ class JSONPRendererTests(TestCase):
resp = self.client.get('/jsonp/nojsonrenderer?callback=' + callback_func,
HTTP_ACCEPT='application/javascript')
self.assertEqual(resp.status_code, status.HTTP_200_OK)
self.assertEqual(resp['Content-Type'], 'application/javascript')
self.assertEqual(resp['Content-Type'], 'application/javascript; charset=iso-8859-1')
self.assertEqual(resp.content,
('%s(%s);' % (callback_func, _flat_repr)).encode('ascii'))
......
rest_framework/tests/response.py
......@@ -101,7 +101,7 @@ class RendererIntegrationTests(TestCase):
def test_default_renderer_serializes_content(self):
"""If the Accept header is not set the default renderer should serialize the response."""
resp = self.client.get('/')
self.assertEqual(resp['Content-Type'], RendererA.media_type)
self.assertEqual(resp['Content-Type'], RendererA.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_A_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -109,13 +109,13 @@ class RendererIntegrationTests(TestCase):
"""No response must be included in HEAD requests."""
resp = self.client.head('/')
self.assertEqual(resp.status_code, DUMMYSTATUS)
self.assertEqual(resp['Content-Type'], RendererA.media_type)
self.assertEqual(resp['Content-Type'], RendererA.media_type + '; charset=utf-8')
self.assertEqual(resp.content, six.b(''))
def test_default_renderer_serializes_content_on_accept_any(self):
"""If the Accept header is set to */* the default renderer should serialize the response."""
resp = self.client.get('/', HTTP_ACCEPT='*/*')
self.assertEqual(resp['Content-Type'], RendererA.media_type)
self.assertEqual(resp['Content-Type'], RendererA.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_A_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -123,7 +123,7 @@ class RendererIntegrationTests(TestCase):
"""If the Accept header is set the specified renderer should serialize the response.
(In this case we check that works for the default renderer)"""
resp = self.client.get('/', HTTP_ACCEPT=RendererA.media_type)
self.assertEqual(resp['Content-Type'], RendererA.media_type)
self.assertEqual(resp['Content-Type'], RendererA.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_A_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -131,7 +131,7 @@ class RendererIntegrationTests(TestCase):
"""If the Accept header is set the specified renderer should serialize the response.
(In this case we check that works for a non-default renderer)"""
resp = self.client.get('/', HTTP_ACCEPT=RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_B_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -142,7 +142,7 @@ class RendererIntegrationTests(TestCase):
RendererB.media_type
)
resp = self.client.get('/' + param)
self.assertEqual(resp['Content-Type'], RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_B_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -150,7 +150,7 @@ class RendererIntegrationTests(TestCase):
"""If a 'format' query is specified, the renderer with the matching
format attribute should serialize the response."""
resp = self.client.get('/?format=%s' % RendererB.format)
self.assertEqual(resp['Content-Type'], RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_B_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -158,7 +158,7 @@ class RendererIntegrationTests(TestCase):
"""If a 'format' keyword arg is specified, the renderer with the matching
format attribute should serialize the response."""
resp = self.client.get('/something.formatb')
self.assertEqual(resp['Content-Type'], RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_B_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -167,7 +167,7 @@ class RendererIntegrationTests(TestCase):
the renderer with the matching format attribute should serialize the response."""
resp = self.client.get('/?format=%s' % RendererB.format,
HTTP_ACCEPT=RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type)
self.assertEqual(resp['Content-Type'], RendererB.media_type + '; charset=utf-8')
self.assertEqual(resp.content, RENDERER_B_SERIALIZER(DUMMYCONTENT))
self.assertEqual(resp.status_code, DUMMYSTATUS)
......@@ -204,7 +204,8 @@ class Issue807Testts(TestCase):
"""
headers = {"HTTP_ACCEPT": RendererA.media_type}
resp = self.client.get('/', **headers)
self.assertEqual(RendererA.media_type, resp['Content-Type'])
expected = "{0}; charset={1}".format(RendererA.media_type, 'utf-8')
self.assertEqual(expected, resp['Content-Type'])
def test_if_there_is_charset_specified_on_renderer_it_gets_appended(self):
"""
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment