# `Waffle.Definition.Storage`
[🔗](https://github.com/elixir-waffle/waffle/blob/v2.0.0/lib/waffle/definition/storage.ex#L1)

Uploader configuration.

Add `use Waffle.Definition` inside your module to use it as uploader.

## Storage directory

By default, the storage directory is `uploads`. But, it can be customized
in two ways.

### By setting up configuration

Customize storage directory via configuration option `:storage_dir`.

    config :waffle,
      storage_dir: "my/dir"

### By overriding the relevant functions in definition modules

Every definition module has a default `storage_dir/2` which is overridable.

For example, a common pattern for user avatars is to store each user's
uploaded images in a separate subdirectory based on primary key:

    def storage_dir(version, {file, scope}) do
      "uploads/users/avatars/#{scope.id}"
    end

> **Note**: If you are "attaching" a file to a record on creation (eg, while inserting the record at the same time), then you cannot use the model's `id` as a path component.  You must either (1) use a different storage path format, such as UUIDs, or (2) attach and update the model after an id has been given. [Read more about how to integrate it with Ecto](https://hexdocs.pm/waffle_ecto/filepath-with-id.html#content)

> **Note**: The storage directory is used for both local filestorage (as the relative or absolute directory), and S3 storage, as the path name (not including the bucket).

## Asynchronous File Uploading

If you specify multiple versions in your definition module, each
version is processed and stored concurrently as independent Tasks.
To prevent an overconsumption of system resources, each Task is
given a specified timeout to wait, after which the process will
fail.  By default, the timeout is `15_000` milliseconds.

If you wish to change the time allocated to version transformation
and storage, you can add a configuration option:

    config :waffle,
      :version_timeout, 15_000 # milliseconds

To disable asynchronous processing, add `@async false` to your
definition module.

## Storage of files

Waffle currently supports:

  * `Waffle.Storage.Local`
  * `Waffle.Storage.S3`

Override the `__storage` function in your definition module if you
want to use a different type of storage for a particular uploader.

### Community adapters

* **Rackspace** - [arc_rackspace](https://github.com/lokalebasen/arc_rackspace)
* **Manta** - [arc_manta](https://github.com/onyxrev/arc_manta)
* **OVH** - [arc_ovh](https://github.com/stephenmoloney/arc_ovh)
* **Google Cloud Storage** - [waffle_gcs](https://github.com/elixir-waffle/waffle_gcs)
* **Microsoft Azure Storage** - [arc_azure](https://github.com/phil-a/arc_azure)
* **Aliyun OSS Storage** - [waffle_aliyun_oss](https://github.com/ug0/waffle_aliyun_oss)

## File Validation

While storing files on S3 eliminates some malicious attack vectors,
it is strongly encouraged to validate the extensions of uploaded
files as well.

Waffle delegates validation to a `validate/1` function with a tuple
of the file and scope.  Validation will be considered successful if
the function returns `true` or `:ok`. A customized error message can
be returned in the form of `{:error, message}`. Any other return value
will return `{:error, :invalid_file}`.

### Validating with external library (e.g. magic_bytes):

Extension-based validation can be trivially bypassed by renaming a file.
Because ImageMagick (and tools built on it) historically delegate format
detection to GhostScript via file extension. Validating by magic bytes,
reading the actual file header - eliminates that vector.

The [`magic_bytes`](https://hex.pm/packages/magic_bytes) package provides
exactly this. `%Waffle.File{}` exposes `path` (local, remote, and binary
uploads) and `stream` (set for streaming uploads), which map directly to
`MagicBytes.from_path/1` and `MagicBytes.from_stream/1`:

    defmodule Avatar do
      use Waffle.Definition

      @allowed_types ~w(image/jpeg image/png image/gif image/webp)

      def validate({%{path: path}, _}) when not is_nil(path) do
        case MagicBytes.from_path(path) do
          {:ok, mime} when mime in @allowed_types -> :ok
          {:ok, _mime} -> {:error, "invalid file type"}
          {:error, :unknown} -> {:error, "invalid file type"}
          {:error, _} -> {:error, "could not read file"}
        end
      end

      def validate({%{stream: stream}, _}) when not is_nil(stream) do
        case MagicBytes.from_stream(stream) do
          {:ok, mime} when mime in @allowed_types -> :ok
          {:ok, _mime} -> {:error, "invalid file type"}
          {:error, :unknown} -> {:error, "invalid file type"}
          {:error, _} -> {:error, "could not read file"}
        end
      end
    end

### Validating by file extension

Extension-based validation is simpler but less secure (see above). It may
be appropriate when you control the upload source or as an additional layer:

    defmodule Avatar do
      use Waffle.Definition
      @extension_whitelist ~w(.jpg .jpeg .gif .png)

      def validate({file, _}) do
        file_extension = file.file_name |> Path.extname() |> String.downcase()

        case Enum.member?(@extension_whitelist, file_extension) do
          true -> :ok
          false -> {:error, "invalid file type"}
        end
      end
    end

## Fetching remote files

When a remote URL is passed to an uploader, Waffle downloads it to a temporary
file before validation, transformation, and storage.

See `Waffle.HTTPClient.Req` for HTTP client setup and Req-specific options.
See `Waffle.HTTPClient.Request` for timeouts, redirects, retries, and response
body limits.

Remote-file HTTP client configuration does not affect requests made by
`Waffle.Storage.S3`, which are handled separately by ExAws.

### Filenames from Content-Disposition

When a remote response provides a filename through `Content-Disposition`,
Waffle uses it instead of the URL's basename. Both `filename` and RFC 6266
`filename*` parameters are supported, with `filename*` taking precedence.

Filenames containing path separators or control characters are rejected. In
that case, Waffle falls back to the filename derived from the URL.

### Passing custom request headers

Waffle does not add custom headers when downloading remote files. Override
`remote_file_headers/1` in your definition module to provide them. For example:

    defmodule Avatar do
      use Waffle.Definition

      def remote_file_headers(%URI{host: "elixir-lang.org"}) do
        credentials = Application.get_env(:my_app, :avatar_credentials)
        token = Base.encode64(credentials[:username] <> ":" <> credentials[:password])

        [{"Authorization", "Basic #{token}"}]
      end
    end

This authenticates requests only to the specified domain. Requests to other
domains use no custom headers.

## Temporary Directory

When processing files, Waffle creates temporary files for operations like downloading
remote files or applying transformations. By default, these files are stored in the
system's temporary directory (as returned by `System.tmp_dir/0`).

You can customize this location by setting the `:tmp_dir` configuration option:

    config :waffle,
      tmp_dir: "/path/to/custom/tmp"

Waffle may fail to remove temporary files if the process using them crashes.

---

*Consult [api-reference.md](api-reference.md) for complete listing*