django-mediasync 2.0: Havana Nights

by

It’s been almost a year since the last release of mediasync, but the new features we’ve worked on are worth the wait! If you use mediasync, please indicate that you do so on our Django Packages profile.

Source on GitHub: https://github.com/sunlightlabs/django-mediasync

Package on PyPI: http://pypi.python.org/pypi/django-mediasync/

Install with pip or easy_install:

pip install django-mediasync
easy_install django-mediasync

What is this media syncing you speak of?

For those of you new to the project, mediasync is a Django app that manages static media in both development and production. Imagine a project where you have to make updates to existing media, but all references are hardcoded to some absolute path in production. Do you update the production media and risk breaking the site or do you temporarily point to local media and hope you don’t forget to revert the change?

With mediasync you don’t need to worry about any of that. Paths to media are automatically generated: local in debug, remote in production, and manually overridden when needed. Modify your media in your local development environment then use mediasync to push the change to the remote production server. Reduce stress and add years to your life!

A whirlwind tour of new features

Pluggable backends

Don’t use S3? Well you can still use mediasync because we’ve added a pluggable backend system! S3 support still comes standard, but now you can sync to anything, as long as you write a client for it. The clients are very easy to implement: extend BaseClient, implement two required methods, and start syncing. A basic pseudo-FTP client:

import ftp
from mediasync.backends import BaseClient

class Client(BaseClient):

    def open(self):
        self.conn = ftp.connect(host="ftp.mysite.com", user=..., pass=...)

    def remote_media_url(self, with_ssl=False):
        return "ftp://ftp.mysite.com"

    def put(self, filedata, content_type, remote_path, force=False):
        self.conn.put(remote_path, filedata)
        return True

    def close(self):
        self.conn.close()
        self.conn = None

Official Cloud Files support

To celebrate pluggable backends, Rich Leland contributed a Rackspace Cloud Files backend. This is a fairly basic first version so expect more improvements soon. Features planned for a future release include:

  • conditional syncing
  • content compression with gzip
  • HTTP expiration and caching headers

Pluggable processors

If you need to do any special processing on your media files, custom file processors allow you to manipulate a file before it is synced remotely. CSS and JS minifiers are enabled by default, but can be removed, replaced, or added to at will. This feature provides the hooks that are needed to do all sorts of cool stuff like SASS and LESS processing, image compression, or anything else you can imagine. How easy is it to write a processor to make all CSS files uppercase?

def big_letters(filedata, content_type, remote_path, is_active):
    if is_active and content_type == 'text/css':
        return filedata.upper()

Back to gzip

Previous version of mediasync used deflate compression that, while supported on all major desktop browsers, had major issues on the Android mobile browser. And by major issues I mean media was unable to be rendered. This release moves from deflate to gzip to provide better browser support. gzip causes some problems with conditional syncing, but it was worth it to work around those in order to have things work on mobile clients.

Smart client compression detection

As pointed out by Steve Souders and Simon Willison, you can’t just force compressed content on every client. We’re guilty of this in previous versions of mediasync, but no more! When syncing content that can be compressed, mediasync pushes two versions of the file to the remote server: one original, one compressed. Template tags inspect the Accept-Encoding HTTP header of the request to determine if the client supports gzip and render paths to the appropriate version of the remote files.

Simplified settings

Much like database configuration, mediasync settings are now stored in a single MEDIASYNC dict in settings.py. A basic S3 configuration:

MEDIASYNC = {
    'BACKEND': 'mediasync.backends.s3',
    'AWS_KEY': 'my-secret-aws-key',
    'AWS_SECRET': 'my-secret-aws-secret',
    'AWS_BUCKET': 'my-media-bucket',
    'AWS_PREFIX': '4.0',
}

In addition, all settings are accessible and modifiable at runtime with the new msettings object.

from mediasync import backends
from mediasync.conf import msettings

# a client with AWS_PREFIX specified in settings.py
client = backends.client()

msettings['AWS_PREFIX'] = '5.0'

# a client with AWS_PREFIX of '5.0'
client = backends.client()

Tests!

I’ll be the first to admit that I’m terrible at writing tests. It’s my downfall as a developer. For this release though, I sucked it up and wrote a bunch of tests to keep all the new changes from breaking stuff. I still have a way to go to get everything tested, but I’m very comfortable with what is covered so far.

What’s next?

Some features that are being considered for future releases include:

  • additional support in the S3 backend for Amazon’s CloudFront CDN
  • an improved Rackspace Cloud Files backend
  • HTML5 manifest generation
  • a JS processor using Google’s Closure Compiler Service API
  • promoting YUI Compressor CSS/JS optimizer to official status
  • a pngcrush file processor for compressing PNG images

If you have any feature requests, please let us know!

Much thanks to…

This release was brought to you by the following people who were kind enough to lend their talents:

Double special thanks to Rob Hudson both for his code and feedback on the direction of the project.