Migrate DynamoDB Data to Astra DB with a Serverless Data Loader

Migrating data from one database to another is rarely simple. It often involves:

• Standing up infrastructure
• Designing database schema for the target system based on the existing schema
• Leveraging multiple (often clunky) tools with complex UX and syntax to extract and load the data, and performing data conversions, either with custom logic or third-party tools.
• Cleaning up infrastructure

The DataStax Astra DB Data Loader enables you to load historical data from DynamoDB into Astra DB with just a few clicks. There is nothing to download, nothing to set up / clean up, and nothing to code. 

With our recent announcement of a 50% reduction in serverless WRU/write operations, migrating data from DynamoDB to AstraDB—the only serverless implementation built on  Apache Cassandra—has never been easier or more cost effective.

Price isn’t the only incentive for write-intensive use cases like logging, time-series, and IoT sensor - even write intensive web apps and web services. Cassandra’s write performance is unmatched. Your apps have the wings to go global: it’s a new cost structure for operating in new geos/regions, reducing latency, and data sovereignty.  Astra DB’s consumption-based cost model combines with autoscaling to remove cost as barrier for multi-region deployments. Astra DB’s costs for serverless reads, storage, and transfer are already competitive, so why not seize the opportunity for both your read and write-intensive use cases? 

For developers, Astra DB offers serverless DBaaS as familiar APIs. Use our SDKs to wire your applications to Astra’s autogenerated API endpoints (REST, GraphQL, gRPC and schemaless JSON). Of course, you’re welcome to use traditional binary language drivers. No matter what your workload dictates, regardless of how you deploy, it’s easier than ever to connect your: 

• Web apps and Jamstack apps
• Microservices and FaaS (functions)
• Embedded / sensor systems
• Desktop client/server applications

Best of all, with Astra DB you can take your Cassandra skills with you to Amazon Web Services (AWS), Microsoft Azure, Google Cloud, Kubernetes, and self-managed clouds. At Cassandra levels of reliability and scale and Astra DB price performance, there’s no need to re-architect or rewrite as you scale: learn it once, take it with you anywhere. 

Let’s examine how this all works.

Taking the sting out of a complex migration task

Our goal is to make your migration effort as frictionless as possible.By adding a DynamoDB source to the Astra Data Loader, we provide a serverless loader that can be kicked off from the Astra DB console. Just provide the Astra DB Data Loader with access to the S3 bucket containing your DynamoDB export. Then, it will:

• Infer a schema from the data 
• Enable the user to tweak the inferred schema in the Astra DB console
• Create the schema in Astra DB
• Asynchronously load everything in a background serverless job

As part of the flow, you’ll provide Astra DB with permissions to read from your S3 bucket by configuring a bucket policy in the AWS UI or console. The permissions can be removed once the data load is completed.

Want to see it in action?  Watch this video as we show you how it’s done on our DataStax Developers Youtube Channel.  Don’t forget to subscribe and ring the alert bell for more awesome content and in - depth, hands on workshops.

Security and privacy

The exact IAM permissions needed for the loader include ListBucket, GetBucketLocation, and GetObject; they can be applied to a bucket policy as follows:

{ "Statement": 
  [ 
  { 
    "Action": [ "s3:ListBucket", "s3:GetBucketLocation" ],
    "Principal": { "AWS": "arn:aws:iam::445559476293:role/astra-loader" },
    "Effect": "Allow", 
    "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME" 
  }, 
  { 
    "Effect": "Allow",
    "Principal": { "AWS": "arn:aws:iam::445559476293:role/astra-loader" }, 
    "Action": [ "s3:GetObject" ], 
    "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/*" 
  } 
  ] 
}

Astra DB loads data from the bucket and the permissions can be removed once the loading job is complete. No cloud resources to clean up–everything spins to zero once it’s done.  

Give it a try!

Getting started with Astra DB is free up to 80GB storage and 20 million read/write operations a month, with no credit card is required. That’s more than enough to have a go at migrating your DyanmoDB data. If you have a DynamoDB table and are curious about what the data and access patterns might look like in Astra DB, give the Data Loader a try. It takes only a few clicks. 

Got a question or issue? The chat console in Astra DB console is staffed by developers, not salespeople, so if you get stuck, we’re just a message away.  Let us know how it goes!

Learn more

Astra DB Data Loader Docs 

DataStax Astra DB product page

Register for an Astra DB account

WRU / Write operations price reduction announcement

Key-Value Benchmark with Java 16 zGC

Astra DB Document API

Astra DB guardrails

Here, There, and Everywhere: Why DataStax Built a Serverless, Multi-Region DBaaS 

Apache Cassandra Documentation

Connect with us 

Join our Discord: DataStax Developers

DataStax Community Platform

DataStax Academy

DataStax Certifications

DataStax Workshops