AWS Interview Prep

A public subnet has a route to an Internet Gateway (IGW) in its route table and instances can have public IPs. A private subnetroutes outbound traffic through a NAT Gateway (for updates, API calls etc.) but has no inbound path from the internet.

The key distinction is the route table, not the subnet itself. Any subnet becomes "public" once you add a 0.0.0.0/0 → IGW route.

In practice, Security Groups are your primary control. NACLs are a secondary defense layer, typically left at defaults unless you need explicit deny rules (e.g., blocking a specific IP range).

I'd design a 3-tier architecture across at least 2 AZs:

VPC: 10.0.0.0/16

Public Subnets (2 AZs):
  ├── ALB
  ├── NAT Gateways (one per AZ for HA)
  └── Bastion host (if needed, prefer SSM)

Private App Subnets (2 AZs):
  ├── EC2 ASG / ECS / EKS worker nodes
  └── No direct internet access

Private Data Subnets (2 AZs):
  ├── RDS Multi-AZ
  └── ElastiCache

Security Group Chain:
  ALB SG → App SG → DB SG (reference by SG ID, not CIDR)

Key decisions: NAT GW per AZ avoids cross-AZ dependency. Security groups reference each other (not CIDRs) so they stay valid if IPs change. VPC Flow Logs enabled for audit.

The ASG marks the instance as unhealthy and begins the replacement process: it terminates the unhealthy instance and launches a new one from the Launch Template. The new instance is registered with the Target Group and must pass ALB health checks before receiving traffic.

There are two health check types: EC2 (system-level — host unreachable) and ELB (application-level — HTTP check fails). You should use ELB health checks for production ASGs.

The choice depends on several factors:

My decision framework: Lambda for event-driven, sub-15min workloads. ECS Fargate if the team doesn't know K8s and wants containers. EKS if K8s skills exist and you need portability. EC2 only for long-running stateful workloads or GPU.

User Data runs a script on first boot — great for small config changes and injecting variables. AMI baking (e.g., with Packer) pre-installs all software into a golden image.

Use User Data for: env-specific config, pulling secrets, dynamic values. Use baked AMIs for: consistent, fast boot (no downloads), production deployments where ASG needs to scale quickly.

Ideal approach: baked AMI + minimal User Data. The AMI has all software installed; User Data only sets environment-specific config.

Multi-AZ creates a synchronous standby replica in another Availability Zone. The standby receives every write in real-time but is not accessible for reads.

During failover (hardware failure, AZ outage, maintenance): AWS automatically promotes the standby to primary. The DNS CNAME endpoint stays the same, so your application reconnects automatically. Failover typically takes 60-120 seconds.

Step 1: Diagnose

• CloudWatch: CPU, FreeableMemory, ReadIOPS, WriteIOPS, DatabaseConnections
• Performance Insights: find top SQL queries by wait time
• pg_stat_statements for query-level metrics

Step 2: Quick wins

• Add missing indexes (from slow query analysis)
• Connection pooling with PgBouncer or RDS Proxy
• Enable query caching / optimize N+1 queries

Step 3: Scale

• Vertical: Resize instance class (e.g., db.r6g.xlarge)
• Read Replicas for read-heavy workloads
• ElastiCache for frequently-read data

Step 4: Architecture

• Consider Aurora PostgreSQL (5x throughput, auto-scaling storage)
• Shard if write-bottlenecked
• CQRS pattern: separate read and write paths

Blue/Green: Run two identical environments. Deploy to green (idle), test, then route traffic from blue to green. Instant rollback = switch back to blue. Costs 2x resources during deployment.

Rolling: Update instances one (or a batch) at a time within the same environment. Lower cost but slower rollback — you must re-deploy the old version.

Use blue/green for critical services where instant rollback is essential. Use rolling for cost-sensitive or stateful services. AWS CodeDeploy supports both strategies.

Developer Push
     │
     ▼
CodePipeline (per service)
  ├── Source:   GitHub (CodeStar Connection)
  ├── Build:    CodeBuild
  │   ├── Unit tests
  │   ├── SAST scan (Semgrep/SonarQube)
  │   ├── Docker build → ECR
  │   └── Artifact: Helm chart / task definition
  ├── Deploy Staging:
  │   ├── ECS/EKS deployment
  │   ├── Integration tests
  │   └── Manual approval gate
  └── Deploy Production:
      ├── Blue/green or canary
      ├── CloudWatch alarms → auto-rollback
      └── Post-deploy smoke tests

Key design decisions:

• One pipeline per service — independent deployment cadence
• Immutable artifacts — same Docker image from staging to prod
• Auto-rollback — tied to CloudWatch alarms (5xx rate, latency)
• Infrastructure changes in a separate pipeline with Terraform
• Secrets via AWS Secrets Manager, never in code or env vars

Terraform state (terraform.tfstate) is a JSON file that maps your HCL config to real AWS resources. It tracks resource IDs, dependency order, and metadata so Terraform knows what exists and what needs to change.

Remote state (e.g., S3 + DynamoDB locking) is important because:

• Multiple team members can collaborate without conflicting
• State often contains secrets — S3 encryption protects it
• DynamoDB locking prevents concurrent apply operations
• CI/CD pipelines need a shared source of truth

Prevention:

• All changes through IaC pipelines — no manual console changes
• Use terraform plan in scheduled CI jobs to detect drift
• AWS Config rules to detect non-compliant resources

Detection:

# Scheduled drift detection
terraform plan -detailed-exitcode
# Exit 0 = no changes, Exit 2 = drift detected

# Refresh state to match reality
terraform refresh  # (or terraform apply -refresh-only)

Resolution:

• If manual change was needed: terraform import the resource
• If manual change was wrong: terraform apply to override
• If state is corrupted: terraform state rm + re-import

Least privilege means granting only the minimum permissionsrequired for a task. In practice:

• Never use * for actions or resources in production policies
• Scope permissions to specific resources using ARNs
• Use conditions (e.g., aws:SourceIp, aws:RequestedRegion)
• Prefer IAM Roles over access keys (no credentials to rotate)
• Use IAM Access Analyzer to find unused permissions

{
  "Effect": "Allow",
  "Action": [
    "s3:GetObject",
    "s3:PutObject"
  ],
  "Resource": "arn:aws:s3:::my-app-bucket/uploads/*",
  "Condition": {
    "StringEquals": {
      "aws:RequestedRegion": "us-east-1"
    }
  }
}

A layered approach:

• AWS Secrets Manager for database credentials, API keys — supports auto-rotation
• SSM Parameter Store (SecureString) for configuration values — free, KMS-encrypted
• At deployment: Inject secrets as env vars via ECS task definitions or K8s ExternalSecrets
• At build: Never in Dockerfiles or buildspec. Use CodeBuild env var references to Secrets Manager
• Rotation: Automated rotation for RDS credentials. Lambda-based rotation for custom secrets
• Audit: CloudTrail logs every Secrets Manager API call

Anti-patterns I'd flag: .env files in repos, hardcoded credentials in User Data, sharing access keys across services, and not rotating secrets.

Tier the alerts by severity:

First 5 minutes (Triage):

• CloudWatch dashboard: ALB 5xx spike, healthy host count
• If 0 healthy hosts → instance-level issue. If hosts up but 503 → app level
• Check recent deployments in CodePipeline (most common cause)

Diagnose:

• ALB target group: which instances are unhealthy?
• SSH/SSM to an unhealthy instance → check app logs, port 3000, disk/memory
• If app won't start → check /health endpoint locally, review recent code changes
• If DB timeout → check RDS metrics, security group, connection count

Mitigate:

• If deployment caused it → rollback immediately (don't debug in prod)
• If capacity issue → manually scale ASG, consider instance type bump
• If database → check connection pooling, kill long-running queries

After resolution:

• Blameless post-mortem within 48 hours
• Document: timeline, root cause, action items
• Improve: add missing alerts, update runbooks, add integration tests

Phase 1: Strangler Fig Pattern

Don't rewrite everything at once. Put an ALB/API Gateway in front and incrementally route specific endpoints to new services while the monolith handles the rest.

Phase 2: Identify Seams

• Domain boundaries (DDD bounded contexts)
• Start with a low-risk, well-defined service (e.g., notifications, search)
• Each service gets its own database (avoid shared DB anti-pattern)

Phase 3: Infrastructure

• ECS Fargate or EKS for container orchestration
• EventBridge or SQS for async communication between services
• Service mesh (App Mesh) for observability across services
• CloudMap for service discovery

Phase 4: Operationalize

• Each service has its own CI/CD pipeline
• Centralized logging (CloudWatch Logs / OpenSearch)
• Distributed tracing (X-Ray)

Work backward from the requirements:

• CDN layer: CloudFront for static content — offload 70-80% of requests
• Compute: ECS/EKS + Fargate with HPA, or Lambda for truly stateless operations
• Caching: ElastiCache Redis in front of the database — sub-ms reads
• Database: Aurora with read replicas, or DynamoDB for simple key-value (single-digit ms)
• Async: Decouple non-critical paths with SQS (write to queue, process later)

Latency budget:

Total:   < 100ms
  ├── Network (CloudFront edge):  ~5ms
  ├── ALB + TLS:                  ~5ms
  ├── App logic:                 ~20ms
  ├── Cache hit (Redis):          ~1ms
  └── DB (if cache miss):        ~30ms
  Margin:                        ~39ms

Key: cache everything possible, scale horizontally, keep services stateless, use read replicas, and measure with X-Ray to find bottlenecks.

SQS (Simple Queue Service) is a message queue — one producer sends, one consumer processes. Messages are pulled and deleted after processing. Use for: decoupling, work queues, buffering.

SNS (Simple Notification Service) is pub/sub — one publisher, many subscribers (Lambda, SQS, email, HTTP). Messages are pushed immediately. Use for: fan-out, notifications, event broadcasting.

Common pattern: SNS → multiple SQS queues (fan-out). An order event triggers SNS, which fans out to an email queue, an inventory queue, and an analytics queue.