Python Boto3 AWS SDK — Core Concepts

What Boto3 is

Boto3 is the official AWS SDK for Python. It provides Python bindings for every AWS service — over 300 of them. Whether you want to manage S3 buckets, launch EC2 instances, query DynamoDB tables, or send SQS messages, Boto3 is the standard way to do it from Python.

AWS generates Boto3’s service definitions from JSON models, which means new AWS features are available in Boto3 shortly after launch. You are not waiting for someone to manually write a wrapper.

Two interfaces: Client and Resource

Boto3 offers two ways to interact with AWS services:

Client (low-level)

The client provides a 1:1 mapping to the AWS API. Every API operation has a corresponding method. Responses are raw dictionaries.

import boto3

s3_client = boto3.client("s3")
response = s3_client.list_buckets()
for bucket in response["Buckets"]:
    print(bucket["Name"], bucket["CreationDate"])

Resource (high-level)

The resource interface wraps API calls in Python objects with attributes and methods. It feels more Pythonic.

s3 = boto3.resource("s3")
for bucket in s3.buckets.all():
    print(bucket.name, bucket.creation_date)

The resource interface is more readable but covers fewer services. For services without resource support, you use the client. In practice, many developers default to the client interface for consistency.

Authentication and sessions

Boto3 looks for credentials in this order:

1. Explicit parameters passed to boto3.client() or Session()
2. Environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
3. Shared credentials file: ~/.aws/credentials
4. AWS config file: ~/.aws/config
5. IAM role (on EC2 instances, Lambda, ECS)

The recommended approach depends on where your code runs:

• Local development — use aws configure to set up ~/.aws/credentials
• Production on AWS — use IAM roles attached to your compute resource (no keys needed)
• CI/CD pipelines — use environment variables or OIDC federation

Sessions

A session holds configuration state (region, credentials, profile):

session = boto3.Session(
    profile_name="production",
    region_name="us-east-1",
)
s3 = session.client("s3")

This is useful when you need to work with multiple AWS accounts or regions in the same script.

Working with S3

S3 is the most commonly used AWS service with Boto3:

s3 = boto3.client("s3")

# Upload a file
s3.upload_file("report.pdf", "my-bucket", "reports/2026/report.pdf")

# Download a file
s3.download_file("my-bucket", "data/input.csv", "/tmp/input.csv")

# Generate a presigned URL (temporary access link)
url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "my-bucket", "Key": "reports/2026/report.pdf"},
    ExpiresIn=3600,
)

Pagination

Many AWS APIs return results in pages. Listing 10,000 S3 objects does not happen in one call. Boto3’s paginators handle this:

s3 = boto3.client("s3")
paginator = s3.get_paginator("list_objects_v2")

for page in paginator.paginate(Bucket="my-bucket", Prefix="logs/"):
    for obj in page.get("Contents", []):
        print(obj["Key"], obj["Size"])

Without paginators, you would need to manually track continuation tokens — a common source of bugs where scripts miss objects beyond the first page.

Waiters

Some AWS operations are asynchronous. Creating an EC2 instance returns immediately, but the instance takes time to start. Waiters poll until a resource reaches a desired state:

ec2 = boto3.client("ec2")
ec2.start_instances(InstanceIds=["i-1234567890abcdef0"])

waiter = ec2.get_waiter("instance_running")
waiter.wait(InstanceIds=["i-1234567890abcdef0"])
print("Instance is running")

Common misconception

A frequent mistake is assuming Boto3 operations are free. Every API call costs money (usually fractions of a cent), but tight loops making thousands of calls can add up. Listing all objects in a million-object S3 bucket at $0.005 per 1,000 requests costs $5 just for the list operation. Be intentional about API call volume.

When to use Boto3

• Automating AWS resource management (create, update, delete)
• Building data pipelines that read from S3 or DynamoDB
• Integrating AWS services into Python applications
• Writing Lambda functions (Boto3 is pre-installed in Lambda)
• Scripts that need to work across multiple AWS accounts or regions

The one thing to remember: Boto3 is Python’s standard gateway to all 300+ AWS services — master sessions, clients, paginators, and waiters, and you can automate anything in the AWS cloud.