Object Storage

Endpoint: https://s3.nfyio.com

This is the S3-compatible surface. All bucket and object operations — PutObject, GetObject, DeleteObject, ListObjectsV2, multipart upload, presigned URLs — are reached here, not on the backend API. Authenticate with an S3 access key + secret you minted at Object Storage → Access Keys in the dashboard. The wire protocol is AWS S3 with SigV4 signing, so any off-the-shelf S3 client works.

Bucket provisioning (creating the bucket record, choosing the storage type, enabling embeddings) is done from the dashboard or via the backend API. The S3 endpoint is for putting data into and out of buckets that already exist.

Region

Set the AWS SDK region to anything — the value is ignored on the wire. The platform default is us-east-1. Pick a constant region per integration so SigV4 signing is stable.

Path-style addressing

Use path-style URLs (https://s3.nfyio.com/<bucket>/<key>). Virtual-hosted-style (https://<bucket>.s3.nfyio.com/<key>) is also supported but less common. AWS SDKs default to virtual-hosted; flip the forcePathStyle / path_style_access flag when you initialize the client (examples below).

AWS CLI

One-time setup

aws configure --profile nfyio
# AWS Access Key ID:     AKIA…           ← from dashboard
# AWS Secret Access Key: …               ← from dashboard
# Default region name:   us-east-1
# Default output format: json

Wrap the endpoint flag in a shell alias so you don’t repeat it:

alias nfys3='aws --profile nfyio --endpoint-url https://s3.nfyio.com'

Common operations

# List buckets the key can see
nfys3 s3 ls

# List objects in a bucket
nfys3 s3 ls s3://my-bucket/
nfys3 s3 ls s3://my-bucket/reports/ --recursive

# Upload a single file
nfys3 s3 cp ./report.pdf s3://my-bucket/reports/q1.pdf

# Upload a directory recursively
nfys3 s3 cp ./build s3://my-bucket/static/ --recursive

# Sync a local folder (only changed files)
nfys3 s3 sync ./build s3://my-bucket/static --delete

# Download
nfys3 s3 cp s3://my-bucket/reports/q1.pdf ./q1.pdf

# Stream-download to stdout
nfys3 s3 cp s3://my-bucket/logs/today.log -

# Delete
nfys3 s3 rm s3://my-bucket/reports/q1.pdf
nfys3 s3 rm s3://my-bucket/old-prefix/ --recursive

# Move / rename (copy + delete)
nfys3 s3 mv s3://my-bucket/old/a.txt s3://my-bucket/new/a.txt

Lower-level `s3api` for full control

# Head an object (size, content-type, etag)
nfys3 s3api head-object --bucket my-bucket --key reports/q1.pdf

# List with pagination
nfys3 s3api list-objects-v2 --bucket my-bucket --prefix reports/ --max-keys 100

# Put with metadata + content-type
nfys3 s3api put-object \
  --bucket my-bucket --key reports/q1.pdf \
  --body ./q1.pdf \
  --content-type application/pdf \
  --metadata "author=ada,quarter=q1"

# Generate a 1-hour download URL
nfys3 s3 presign s3://my-bucket/reports/q1.pdf --expires-in 3600

Python — boto3

import boto3
from botocore.config import Config

s3 = boto3.client(
    "s3",
    endpoint_url="https://s3.nfyio.com",
    aws_access_key_id="AKIA…",
    aws_secret_access_key="…",
    region_name="us-east-1",
    config=Config(s3={"addressing_style": "path"}),
)

# Upload
s3.upload_file("./report.pdf", "my-bucket", "reports/q1.pdf",
               ExtraArgs={"ContentType": "application/pdf"})

# Download
s3.download_file("my-bucket", "reports/q1.pdf", "./q1.pdf")

# Stream to memory
obj = s3.get_object(Bucket="my-bucket", Key="reports/q1.pdf")
data = obj["Body"].read()

# List with pagination
paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-bucket", Prefix="reports/"):
    for item in page.get("Contents", []):
        print(item["Key"], item["Size"])

# Delete
s3.delete_object(Bucket="my-bucket", Key="reports/q1.pdf")

# Presigned download URL (1 hour)
url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "my-bucket", "Key": "reports/q1.pdf"},
    ExpiresIn=3600,
)

Node.js — `@aws-sdk/client-s3`

import {
  S3Client,
  PutObjectCommand,
  GetObjectCommand,
  DeleteObjectCommand,
  ListObjectsV2Command,
} from '@aws-sdk/client-s3'
import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
import { readFileSync } from 'node:fs'

const s3 = new S3Client({
  endpoint: 'https://s3.nfyio.com',
  region: 'us-east-1',
  credentials: {
    accessKeyId: process.env.NFYIO_S3_KEY,
    secretAccessKey: process.env.NFYIO_S3_SECRET,
  },
  forcePathStyle: true,
})

// Upload
await s3.send(new PutObjectCommand({
  Bucket: 'my-bucket',
  Key: 'reports/q1.pdf',
  Body: readFileSync('./report.pdf'),
  ContentType: 'application/pdf',
}))

// Download (Body is a stream)
const out = await s3.send(new GetObjectCommand({ Bucket: 'my-bucket', Key: 'reports/q1.pdf' }))
const buf = Buffer.concat(await out.Body.toArray())

// List
const list = await s3.send(new ListObjectsV2Command({
  Bucket: 'my-bucket',
  Prefix: 'reports/',
  MaxKeys: 100,
}))

// Delete
await s3.send(new DeleteObjectCommand({ Bucket: 'my-bucket', Key: 'reports/q1.pdf' }))

// Presigned URL (1 hour)
const url = await getSignedUrl(s3,
  new GetObjectCommand({ Bucket: 'my-bucket', Key: 'reports/q1.pdf' }),
  { expiresIn: 3600 })

Go — `aws-sdk-go-v2`

import (
    "context"
    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/credentials"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

cfg, _ := config.LoadDefaultConfig(context.TODO(),
    config.WithRegion("us-east-1"),
    config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider("AKIA…", "…", "")),
)
client := s3.NewFromConfig(cfg, func(o *s3.Options) {
    o.BaseEndpoint = aws.String("https://s3.nfyio.com")
    o.UsePathStyle = true
})

// PutObject, GetObject, DeleteObject, ListObjectsV2 all work as in vanilla AWS S3.

Multipart upload

Files larger than ~100 MB should use multipart. The high-level helpers in every SDK do this automatically.

AWS CLI

# Anything above the multipart_threshold (8 MB by default) is multipart
nfys3 s3 cp ./big.bin s3://my-bucket/datasets/big.bin

boto3 (manual)

mpu = s3.create_multipart_upload(Bucket="my-bucket", Key="datasets/big.bin")
parts = []
with open("big.bin", "rb") as f:
    part_no = 1
    while chunk := f.read(8 * 1024 * 1024):  # 8 MiB
        resp = s3.upload_part(
            Bucket="my-bucket", Key="datasets/big.bin",
            UploadId=mpu["UploadId"], PartNumber=part_no, Body=chunk,
        )
        parts.append({"ETag": resp["ETag"], "PartNumber": part_no})
        part_no += 1

s3.complete_multipart_upload(
    Bucket="my-bucket", Key="datasets/big.bin",
    UploadId=mpu["UploadId"],
    MultipartUpload={"Parts": parts},
)

If anything goes wrong before complete_multipart_upload, call s3.abort_multipart_upload(...) so the partial parts are cleaned up.

Node.js (high-level helper)

import { Upload } from '@aws-sdk/lib-storage'
import { createReadStream } from 'node:fs'

const upload = new Upload({
  client: s3,
  params: {
    Bucket: 'my-bucket',
    Key: 'datasets/big.bin',
    Body: createReadStream('./big.bin'),
  },
  partSize: 8 * 1024 * 1024,
  queueSize: 4,
})
upload.on('httpUploadProgress', p => console.log(p.loaded, '/', p.total))
await upload.done()

Presigned URLs

Presigned URLs let an unauthenticated browser or third-party service fetch (or upload) a single object for a limited time. Generate the URL on your backend with the access key, hand it to the client, never expose the key itself.

Download URL

url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "my-bucket", "Key": "reports/q1.pdf"},
    ExpiresIn=900,  # 15 minutes
)
# return url to the browser; an <a href={url}> works directly

Upload URL (browser → S3 direct)

url = s3.generate_presigned_url(
    "put_object",
    Params={"Bucket": "my-bucket", "Key": f"uploads/{uuid4()}", "ContentType": "image/png"},
    ExpiresIn=300,
)

The browser then PUTs the file body straight to that URL with the same Content-Type header. The platform never proxies the bytes through your backend.

Object metadata

Every object can carry user-defined metadata via the x-amz-meta-* headers. The platform indexes a few special keys that change behaviour:

Header	Effect
`x-amz-meta-author` (or any custom key)	Stored verbatim, returned on `HeadObject` / `GetObject`
`Content-Type`	Stored and returned. Setting it correctly matters for inline display via presigned URLs.
`Cache-Control`, `Content-Disposition`, `Content-Encoding`	Honored on `GetObject`

For Pro / Agentic-RAG buckets, the embedding pipeline picks up new objects automatically — no special metadata required. To skip indexing for a particular upload, set x-amz-meta-nfyio-skip-index: true.

Versioning

Versioning is toggled per bucket via the backend API:

curl -X PUT https://app.nfyio.com/api/object-storage/buckets/my-bucket/versioning \
  -H "Authorization: Bearer $NFYIO_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"enabled": true}'

Once enabled, the standard S3 versionId semantics work:

nfys3 s3api list-object-versions --bucket my-bucket --prefix reports/
nfys3 s3api get-object --bucket my-bucket --key reports/q1.pdf --version-id <id> ./q1-old.pdf
nfys3 s3api delete-object --bucket my-bucket --key reports/q1.pdf --version-id <id>

Bucket policies and ACL

Public-read on buckets is not supported on the platform — every read goes through either an S3 access key or a presigned URL. This is by design, to keep the audit trail and quota accounting accurate.

For per-bucket sharing, mint an S3 access key scoped to that bucket with read permission and hand it to the consumer.

Errors

S3 errors follow the standard AWS XML error format:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>NoSuchKey</Code>
  <Message>The specified key does not exist.</Message>
  <Key>reports/q1.pdf</Key>
  <RequestId>…</RequestId>
</Error>

Code	HTTP	Meaning
`InvalidAccessKeyId`	403	Access key not recognised
`SignatureDoesNotMatch`	403	Secret key wrong, or clock skew > 15 min
`AccessDenied`	403	Key is valid but lacks permission for this bucket / operation
`NoSuchBucket`	404	Bucket does not exist or is in another workspace
`NoSuchKey`	404	Object does not exist
`BucketAlreadyOwnedByYou`	409	Returned by `CreateBucket`-style flows; prefer the dashboard for provisioning
`EntityTooLarge`	400	Single-shot upload exceeded the part size limit — switch to multipart
`SlowDown`	503	Throttled — retry with exponential backoff

Limits

Single-shot upload: up to 5 GB. Use multipart for anything larger.
Multipart part size: 5 MB – 5 GB. SDK default of 8 MB is fine.
Multipart parts: up to 10 000 per upload.
Object size (multipart total): up to 5 TB.
Listing page size: up to 1 000 keys per ListObjectsV2 call. Paginate with ContinuationToken.

Migrating from AWS S3

Swap the endpoint and region — keep the SDK calls identical:

- const s3 = new S3Client({ region: 'us-east-1' })
+ const s3 = new S3Client({
+   endpoint: 'https://s3.nfyio.com',
+   region: 'us-east-1',
+   forcePathStyle: true,
+ })

If your existing code uses presigned URLs, no changes needed — the URLs work the same way with our endpoint baked in.

Object Storage

Region

Path-style addressing

AWS CLI

One-time setup

Common operations

Lower-level s3api for full control

Python — boto3

Node.js — @aws-sdk/client-s3

Go — aws-sdk-go-v2

Multipart upload

AWS CLI

boto3 (manual)

Node.js (high-level helper)

Presigned URLs

Download URL

Upload URL (browser → S3 direct)

Object metadata

Versioning

Bucket policies and ACL

Errors

Limits

Migrating from AWS S3

Lower-level `s3api` for full control

Node.js — `@aws-sdk/client-s3`

Go — `aws-sdk-go-v2`