Uploading Files

At the core of Pinata is file uploads through our File API. It’s fast and easy to use, whether you’re using the SDK or the API. Something unique about the Files API is that all files are given a unique Content Identifier or CID. These are not like normal IDs in that they are content addressable hashes based on the contents of the file. You can upload the same file over and over and it will always return the same CID. This has multiple benefits including:

• Deduplication: Since a duplicate file would have the same CID, it is not included upon upload and reduces the amount of files in your account. This keeps things organized and reduces your total costs.
• Versioning: Tracking the history of a file requires proprietary solutions built on top of the data storage provider and does not expose that history and provenance in a meaningful way to everyone who needs to access it.

Let’s look at the multiple ways you can upload files!

​

How to Upload Files

Uploading files with Pinata is simple, whether you want to use the SDK or the API. Key things to know:

• Uploads are done through multipart/form-data requests
• The SDK and API accept File objects per the Web API Standard for Files
• You can add additional info to your upload such as a custom name for the file, keyvalue metadata, and a target group destination for organization

Here is a simple example of how you might upload a file in Typescript

This will return the following response

• id: The ID of the file used for getting info, updating, or deleting
• name: The name of the file or the provided name in the addMetadata method
• cid: A cryptographic hash based on the contents of the file
• size: The size of the file in bytes
• number_of_files: The number of files in a reference
• mime_type: The mime type of the uploaded file
• user_id: User ID of the API key that upladed the file
• group_id: The group the file was uploaded to if applicable

​

Metadata

When uploading a file you can add additional metadata using the addMetadata method after the selected upload method. This can include an optional name override or keyvalue pairs that can be used to searching the file later on

​

Groups

Pinata offers File Groups to organize your content, as well as a method to set files as public or private. You can upload a file to a group by using the group method.

​

Client Side Uploads

There are situations where you may need to upload a file client side instead of server side. A great example is in Next.js where there is a 4MB file size restriction for files passed through Next’s API routes. To solve this you can create a signed upload URL on the server and then pass it to the client for it to be consumed. This way your admin API key stays safe behind a server. Creating signed upload URLs can be done with either the Files SDK or the API, and you can designate how long the URL is valid for or if there is other infromation you want to include such as metadata or a group ID.

Setting up a server side API endpoint might look something like this:

Then back on the client side code, you can upload using the signed URL instead of the regular upload endpoint.

​

Upload Progress

If you happen to use the API as well as local files you can also track the progress of the upload using a library like got. Better support for upload progress will come in later versions of the SDK!

import fs from "fs";
import FormData from "form-data";
import got from "got";

async function upload() {
  const url = `https://uploads.pinata.cloud/v3/files`;
  try {
    let data = new FormData();
    data.append(`file`, fs.createReadStream("path/to/file"));
    const response = await got(url, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.PINATA_JWT}`,
      },
      body: data,
    }).on("uploadProgress", (progress) => {
      console.log(progress);
    });
    console.log(JSON.parse(response.body));
  } catch (error) {
    console.log(error);
  }
}

If your file is larger than 100MB then a better approach is to follow the Resumable Upload Guide

​

Common File Recipes

Below are some common recipes for uploading a file.

​

Blob

Usually you can pass a Blob directly into the request but to help guarantee success we recommend passing it into a File object.

​

JSON

Pinata makes it easy to upload JSON objects using the json method.

​

Local Files

If you need to upload files from a local file source you can use fs to feed a file into a blob, then turn that blob into a File.

​

URL

To upload a file from an external URL you can stream the contents into an arrayBuffer, which then gets passed into a new Blob that can then be uploaded to Pinata. This has been abstracted in the SDK using the url method.

​

base64

To upload a file in base64 simply turn the contents into a buffer that is passed into a Blob. Alternatively you can use the SDK for this as well using the base64 method.

​

Resumable Uploads

The Files API upload endpoint https://uploads.pinata.cloud/v3/files is fully TUS compatible, so it can support larger files with the ability to resume uploads. Any file upload larger than 100MB needs to be uploaded through the TUS method. The Files SDK handles this automatically when you use pinata.upload.file() by checking the file size before uploading.

const pinata = new PinataSDK({
  pinataJwt: process.env.PINATA_JWT!,
  pinataGateway: "example-gateway.mypinata.cloud",
});

const upload = await pinata.upload.file(massiveFile);

If you want to take advantage of resumable uploads then we would recommend using one of the TUS clients and taking note of the following:

• Upload chunk size must be smaller than 50MB
• Instead of using the form data fields for group_id or keyvalues these can be passed directly into the upload metadata (see example below)
• Headers must include the Authorization with your Pinata JWT

Here is an example of an upload to Pinata using the tus-js-client

import * as tus from "tus-js-client";

async function resumeUpload(file) {
  try {
    const upload = new tus.Upload(file, {
      endpoint: "https://uploads.pinata.cloud/v3/files",
      chunkSize: 50 * 1024 * 1024, // 50MiB chunk size
      retryDelays: [0, 3000, 5000, 10000, 20000],
      onUploadUrlAvailable: async function () {
        if (upload.url) {
          console.log("Upload URL is available! URL: ", upload.url);
        }
      },
      metadata: {
        filename: "candyroad-demo.mp4", // name
        filetype: "video/mp4",
        group_id: "0192868e-6144-7685-9fc5-af68a1e48f29", // group ID
        keyvalues: JSON.stringifiy({ env: "prod" }), // keyvalues
      },
      headers: { Authorization: `Bearer ${process.env.PINATA_JWT}` }, // auth header
      uploadSize: fileStats.size,
      onError: function (error) {
        console.log("Failed because: " + error);
      },
      onProgress: function (bytesUploaded, bytesTotal) {
        const percentage = ((bytesUploaded / bytesTotal) * 100).toFixed(2);
        console.log(percentage + "%");
      },
      onSuccess: function () {
        console.log("Upload completed!");
      },
    });

    upload.start();
  } catch (error) {
    console.log(error);
  }
}

​

Web App

You can also use the Pinata App to upload files. It’s as simple as clicking the “Upload” button in the top right corner of the files page. Select your file, give it a name, then upload. Once it’s complete you’ll see it listed in the files page.

Start uploading by signing up for a free account!