If you're used to working with Django, you've probably done so before on a single host with its own file system. For example, media files will probably be stored in a directory in the root of the project directory.

This gives you, and Python, direct access to those files.

No persistent local file storage

The Divio Cloud architecture works a little differently. The same site may be running as several different instances, on several different hosts (this is one way we're able to scale so hugely - we can simply create new instances to handle increasing demand).

Although each of those instances will indeed have its own local file storage, it will be independent of each of the others, and it won't persist - once that instance ceases to exist, so will the files.

This means a project can't upload files, save them to disk, and then expect to find them again. They might be there, if you happen to land on the same instance when asking for them, but they might equally well not be.

Amazon S3 storage

Instead, we use Amazon S3 for file storage. It's an independent persistent storage system, that all instances of a particular site will equal and transparent access to.

Use Django's storage API

For most Django applications, this won't be an issue, nor will you need to learn about S3. 

Django is able to use multiple storage backends, all addressed through a common API. This is the safe and correct way to handle files in Django, so that applications can abstract from details of the storage implementation, and simply not need to know about it.

As long as the application uses Django's storage API, rather than attempting to manipulate Python File objects directly, it doesn't need to do anything differently.

Similarly, an application should not rely on knowing or manipulating a File object's file path.

Direct access to S3

The best way to manage files is from your own desktop, using the Divio app or the Divio CLI. Occasionally however you may require direct access to the S3 filesystem, for example if you have very large (1GB or more) of files that you wish to manipulate.

In that case we are able to provide you with this access (on request). Note that any operations are undertaken entirely at your own risk.

Caveats

Django Filer

If you are using Django Filer, please use the Django Filer Addon from the Marketplace, rather than installing the Filer into your project via its requirements. This is because Filer currently requires its THUMBNAIL_DEFAULT_STORAGE to be set manually. See the Django Filer Addon code to see how we do this.

Did this answer your question?