Serverless File Upload Architecture on AWS

Instead of allowing users to upload directly to your production S3 bucket, use a controlled flow.

Architecture Overview

• User → Pre-Signed URL → Incoming S3 Bucket → Lambda Scan
  • If Clean → Move to Clean Bucket
  • If Infected → Move to Quarantine Bucket → Notify via SNS

This approach is called the Incoming / Clean / Quarantine Pattern.

Step 1: Upload Files Securely Using Pre-Signed URLs

Users should never upload files through your backend server. Instead, generate a pre-signed URL that allows temporary direct upload to an isolated S3 bucket.

Pre-signed URLs in Amazon S3 provide time-limited access with controlled permissions. This results in reduced server load and no backend file handling.

Example – Generate Pre-Signed URL (Node.js)

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"
import { getSignedUrl } from "@aws-sdk/s3-request-presigner"

const s3 = new S3Client({ region: "us-east-1" })

export async function generateUploadUrl(userId, fileName) {
  const command = new PutObjectCommand({
    Bucket: "incoming-upload-bucket",
    Key: `uploads/${userId}/${fileName}`,
    ContentType: "application/pdf"
  })

  return await getSignedUrl(s3, command, { expiresIn: 600 })
}

‍
Important security measures:

• Enable Block Public Access
• Enforce file size limits
• Validate MIME type
• Restrict upload to incoming bucket only

Never allow uploads directly into your clean or production bucket.

Step 2: Scan Files in Amazon S3 Using AWS Lambda

Once a file lands in your incoming bucket in Amazon S3, it can automatically trigger a Lambda function through an event notification. This keeps the workflow fully serverless and event-driven.

The Lambda function typically downloads the file into temporary storage, performs a malware scan, evaluates the result, and then moves the file to the appropriate destination. If the file is safe, it is transferred to a clean bucket. If it is infected, it is moved into quarantine. Finally, the function publishes a notification so the system and administrators know what happened.

Because everything is triggered by events, there are no background servers running continuously. The system reacts only when uploads occur, which keeps it efficient and cost-effective.

How to Scan Files for Viruses in S3

There are three common approaches to S3 virus scanning.

Option 1: ClamAV in AWS Lambda (Most Popular Serverless Method)

ClamAV is an open-source antivirus engine. You can package it as a Lambda layer and run scans inside the function.

Basic example:

import { execSync } from "child_process"

function scanFile(filePath) {
  const result = execSync(`clamscan ${filePath}`).toString()
  return result.includes("OK")
}

‍
This approach is commonly used by SaaS startups and applications that deal with moderate file sizes (typically under 100MB). It works well in fully serverless architectures and is a cost-effective option because it does not rely on external APIs.

One advantage of this method is that everything runs inside your AWS environment, which keeps the workflow event-driven and reduces infrastructure overhead. However, there are some limitations to consider. Lambda cold starts can introduce slight latency, virus definitions must be updated periodically, and very large files may exceed Lambda limits.

Option 2: Container-Based Malware Scanning (Enterprise Scale)

For larger files or high-volume workloads, scanning can be offloaded to container-based services such as Amazon ECS or Amazon EKS.

This approach removes Lambda execution limits and allows the system to process much larger files and higher traffic volumes. It is often used in enterprise environments, particularly in financial or healthcare systems where files may exceed 500MB and compliance requirements are strict.

Option 3: Third-Party Malware Scanning API

Some enterprises choose to integrate with external security scanning providers through APIs. In this setup, the Lambda function sends the uploaded file or its contents to a third-party scanning service and waits for the response.

Example:

import axios from "axios"

async function scanWithAPI(fileBuffer) {
  const response = await axios.post("https://scanner-api.com/scan", fileBuffer)
  return response.data.status === "clean"
}

‍
This model is commonly used in FinTech, HealthTech, and enterprise SaaS applications where strong malware detection and managed security updates are required. The trade-off is cost, as third-party services typically charge per scan or per volume processed.

Step 3: Clean and Quarantine Buckets

After the scanning process completes, the system decides where the file should go next.

If the file is clean, it is copied to the clean bucket, tagged with metadata such as scan-status=clean, and removed from the incoming bucket. If the file is infected, it is moved to the quarantine bucket and tagged with scan-status=infected. A security notification is then triggered so administrators are aware of the event.

Only the clean bucket should be connected to your production application. This separation enforces a zero-trust storage model where unverified files never interact with production systems.

Step 4: Notify Users or Admins Using SNS

Once the scanning process finishes, the system publishes a message using Amazon SNS to notify relevant systems or users about the result.

Example:

await sns.publish({
  TopicArn: process.env.TOPIC_ARN,
  Message: JSON.stringify({
    file: key,
    status: isClean ? "CLEAN" : "INFECTED"
  })
})

‍
SNS can send email alerts, trigger additional Lambda functions, notify Slack channels, or push events into monitoring dashboards. This improves observability and creates an audit trail for security and compliance purposes.

Additional Security Best Practices for File Uploads

Malware scanning alone is not sufficient to fully secure a file upload pipeline. Additional validation layers should be implemented to ensure files are legitimate and safe to process.

These protections include verifying file extensions, validating MIME types, checking file signatures using magic bytes, enforcing strict file size limits, and applying strong IAM policies to control access. S3 object tagging, lifecycle rules, and versioning should also be enabled to support auditing and automated cleanup.

Example – Magic byte validation for PDF:

const isPDF = buffer.slice(0, 4).toString() === "%PDF"

Never trust file extensions alone.

How to Scale This Serverless File Upload Architecture

If upload traffic grows significantly, it can be helpful to introduce buffering into the event flow.

A common scalable architecture is: S3 → EventBridge → SQS → Lambda

This pattern allows retry handling, dead-letter queues, controlled concurrency, and backpressure protection. Instead of overwhelming Lambda during traffic spikes, events are queued and processed gradually, making the system more resilient and production-ready.

Cost Optimization Tips

To reduce AWS costs, several best practices can be applied. Lifecycle rules can automatically delete old quarantined files, preventing unnecessary storage expenses. Lambda memory should be optimized according to the workload, and unnecessary data transfers should be avoided.

Compressing files before scanning can reduce storage and transfer costs, and S3 Intelligent-Tiering may help optimize storage costs for files that are accessed infrequently. Since serverless infrastructure scales automatically with usage, you only pay for the resources actually consumed.

Final Architecture Recommendation

For most SaaS builders:

Amazon S3 + AWS Lambda + ClamAV + Amazon SNS

For high-scale enterprise systems:

S3 + SQS + ECS + Security API integration

Start simple. Scale when needed.

Conclusion

Building a secure file upload system on AWS is critical for modern SaaS applications.

By combining Amazon S3 for storage, AWS Lambda for malware scanning, and Amazon SNS for notification, you can create a fully serverless, scalable, and secure file upload pipeline.

This architecture ensures:

• Only clean files enter production
• Infected files are isolated
• Users are notified
• Compliance risk is reduced
• Infrastructure remains serverless

Secure file upload design is responsible engineering.

References

Amazon S3 Documentation

https://docs.aws.amazon.com/s3/

AWS Lambda Documentation

https://docs.aws.amazon.com/lambda/

Amazon SNS Documentation

https://docs.aws.amazon.com/sns/

AWS Blog – Scan S3 Objects for Malware

https://aws.amazon.com/blogs/security/how-to-scan-amazon-s3-objects-for-malware/

ClamAV Official Site

https://www.clamav.net/

‍