Generate production-ready Terraform configurations for cloud infrastructure including VPC, compute, storage, databases, and networking.
As part of the "Terraform Infrastructure Builder" workflow, this step focuses on research_design_requirements. The goal is to generate comprehensive, detailed, and production-ready design specifications for cloud infrastructure that will subsequently be translated into Terraform configurations.
This document outlines the detailed design specifications for a robust, scalable, secure, and cost-optimized cloud infrastructure, intended for production environments. The primary objective is to define the architectural components, their interconnections, and operational considerations that will guide the generation of Terraform configurations.
It's important to clarify the interpretation of certain terms in the context of Infrastructure-as-Code (IaC):
For this deliverable, we will assume AWS as the target cloud provider due to its widespread adoption and comprehensive service offerings, but the principles are largely cloud-agnostic.
* Virtual Private Cloud (VPC) and Networking
* Compute Resources (EC2, Auto Scaling, Load Balancing)
* Database Services (Relational, NoSQL)
* Storage Solutions (Object, Block, File)
* Security, Identity, and Access Management (IAM)
* Monitoring, Logging, and Alerting
* Deployment Pipeline Considerations
* Dedicated VPC: A single, isolated VPC per environment (e.g., prod-vpc).
* CIDR Block: Appropriately sized CIDR block (e.g., /16 or /18) to allow for future expansion.
* Multi-AZ Deployment: At least three Availability Zones (AZs) for high availability.
* Subnetting Strategy:
* Public Subnets: For resources requiring direct internet access (e.g., Load Balancers, NAT Gateways). One per AZ.
* Private Application Subnets: For application servers, backend services. One per AZ.
* Private Database Subnets: For database instances. One per AZ.
* Private Management Subnets (Optional): For jump boxes, monitoring agents.
* Internet Gateway (IGW): Attached to the VPC for outbound internet access from public subnets.
* NAT Gateway (NAT GW): Deployed in public subnets (one per AZ for redundancy) to allow private instances outbound internet access.
* Route Tables:
* Public Route Table: Routes to IGW for public subnets.
* Private Route Table: Routes to NAT GW for private subnets.
* Database Route Table: Routes to NAT GW for private database subnets.
* DNS Resolution: Enable DNS Hostnames and DNS Resolution within the VPC.
* Instance Types: Selected based on application workload requirements (CPU, memory, network performance). Utilize burstable (T-series) for low-traffic, cost-sensitive, or development workloads; general-purpose (M-series) or compute-optimized (C-series) for production.
* AMIs: Use hardened, up-to-date AMIs (e.g., Amazon Linux 2, Ubuntu LTS) or custom golden AMIs.
* Auto Scaling Groups (ASG):
* Deployed across multiple private application subnets for high availability and scalability.
* Min/Max/Desired capacities configured based on expected load.
* Scaling Policies: Target Tracking (e.g., CPU utilization, ALB request count per target) and Scheduled Scaling.
* Launch Templates: Used for defining instance configurations (AMI, instance type, EBS, user data, security groups, IAM role).
* Load Balancers:
* Application Load Balancer (ALB): For HTTP/HTTPS traffic, deployed in public subnets. Configured with listener rules, target groups, and health checks. Integrated with AWS WAF for web application firewall capabilities.
* Network Load Balancer (NLB) (Optional): For high-performance TCP/UDP traffic or specific needs.
* ECS/EKS Cluster: If containerized applications are in scope, an ECS/EKS cluster will be provisioned within the private application subnets.
* Fargate/EC2: Choice between Fargate (serverless) or EC2 (managed instances) for compute, based on control and cost requirements.
* Engine: PostgreSQL or MySQL (or Aurora for higher performance/scalability) preferred.
* Deployment: Multi-AZ deployment for high availability with automatic failover.
* Instance Type: Chosen based on performance requirements (IOPS, memory).
* Storage: Provisioned IOPS (PIOPS) or General Purpose SSD (gp2/gp3) based on workload. Encryption at rest enabled.
* Backup & Recovery: Automated backups, retention period configured. Point-in-time recovery enabled.
* Read Replicas (Optional): For read-heavy workloads or cross-region disaster recovery.
* Connectivity: Accessible only from private application subnets via security groups.
* Tables: Provisioned with appropriate Read/Write Capacity Units (RCUs/WCUs) or On-Demand mode.
* Global Tables (Optional): For multi-region replication.
* Backup: Point-in-time recovery enabled, or on-demand backups.
* Encryption: Encryption at rest enabled.
* Buckets: Created for static assets, backups, logs, data lakes.
* Versioning: Enabled for critical data.
* Lifecycle Policies: Configured for cost optimization (e.g., transition to Glacier, expiration).
* Encryption: Server-side encryption (SSE-S3, SSE-KMS) enabled by default.
* Access Control: Least privilege IAM policies and bucket policies.
* Volumes: Provisioned for EC2 instances. General Purpose SSD (gp2/gp3) or Provisioned IOPS SSD (io1/io2) based on performance needs.
* Encryption: Encryption at rest enabled.
* Snapshots: Automated snapshots for backup.
* EFS: For shared file systems across multiple EC2 instances or containers. Encryption at rest and in transit.
* FSx for Lustre/Windows File Server: For specific high-performance computing or Windows-based workloads.
* Least Privilege Principle: All IAM roles and policies will grant only the minimum necessary permissions.
* IAM Roles for EC2: Instances will use IAM roles instead of access keys for AWS service interaction.
* MFA: Enforced for all root and administrative IAM users.
* Access Keys: Regularly rotated, ideally avoided for programmatic access unless absolutely necessary.
* Granular Rules: Defined for each component (e.g., ALB SG, App SG, DB SG) allowing only necessary inbound/outbound traffic.
* Stateful: Traffic permitted in one direction is automatically allowed back.
* Stateless: Used as an additional layer of security at the subnet level, primarily for stricter inbound/outbound control if needed (less common than SGs for primary control).
* CMKs: Used for encrypting EBS volumes, S3 buckets, RDS databases, and other services.
* Centralized Key Management: All encryption keys managed centrally.
* Associated with ALBs for protection against common web exploits (OWASP Top 10).
* For secure storage and retrieval of database credentials, API keys, and other sensitive configuration data.
* Metrics: Collect standard and custom metrics for EC2, RDS, ALB, etc.
* Alarms: Configure alarms for critical thresholds (e.g., high CPU, low disk space, error rates) to trigger notifications (SNS).
* Dashboards: Create centralized dashboards for operational visibility.
* Log Groups: Centralized collection of application logs, VPC Flow Logs, Load Balancer Access Logs.
* Log Retention: Configured based on compliance and operational needs.
* Audit Logging: Enabled for all management events and data events (optional for S3, Lambda) to track API calls and user activity.
* S3 Storage: Logs stored in an S3 bucket with appropriate lifecycle policies.
* Compliance: Enabled for continuous monitoring of resource configurations and compliance with desired state.
While this step focuses on infrastructure generation, the generated Terraform will be designed to integrate seamlessly into a CI/CD pipeline:
vpc, ec2, rds, s3) for reusability and clarity.As a professional AI assistant within PantheraHive, I am generating detailed design specifications for the "Terraform Infrastructure Builder" workflow. This output is comprehensive, actionable, and designed to provide a clear vision for the user interface and experience.
This document outlines the design specifications, wireframe descriptions, color palettes, and user experience (UX) recommendations for the Terraform Infrastructure Builder. The goal is to create an intuitive, powerful, and professional tool for generating production-ready Terraform configurations.
This section covers the core functionality, workflow, input mechanisms, output, and advanced features of the Terraform Infrastructure Builder.
This document details the finalized design assets for your cloud infrastructure, leveraging Terraform for robust, scalable, and maintainable deployment. While "design assets" typically refer to user interface elements, in the context of Infrastructure as Code (IaC) and Terraform, we interpret these as comprehensive architectural specifications, visual representations of the infrastructure, and best practices for managing and interacting with the deployed environment.
This deliverable encompasses the detailed specifications, architectural representations, and operational recommendations for your production-ready cloud infrastructure, built entirely with Terraform. The goal is to provide a holistic view of the infrastructure design, ensuring clarity, maintainability, and alignment with modern cloud best practices.
Key Deliverables in this Section:
These specifications form the backbone of your Terraform configurations, outlining every component with precision.
10.0.0.0/16 (Example: Customizable per environment/region) * CIDR Blocks: 10.0.1.0/24 (AZ-1), 10.0.2.0/24 (AZ-2), 10.0.3.0/24 (AZ-3)
* Purpose: Load balancers, NAT Gateways, public-facing services (if applicable).
* Associated with: Internet Gateway, Public Route Table.
* CIDR Blocks: 10.0.11.0/24 (AZ-1), 10.0.12.0/24 (AZ-2), 10.0.13.0/24 (AZ-3)
* Purpose: Application servers, container orchestrators.
* Associated with: NAT Gateway (for outbound internet access), Private Route Table.
* CIDR Blocks: 10.0.21.0/24 (AZ-1), 10.0.22.0/24 (AZ-2), 10.0.23.0/24 (AZ-3)
* Purpose: Database instances (RDS, DocumentDB, etc.).
* Associated with: Private Route Table (no direct internet access).
* Public Route Table: Default route to IGW.
* Private Route Table (App): Default route to NAT Gateway.
* Private Route Table (DB): No default route to internet, only internal VPC routes.
* Instance Type: t3.medium (scalable, e.g., m5.large for production)
* AMI: Latest Amazon Linux 2 or Ubuntu LTS.
* Auto Scaling Group:
* Min/Max/Desired: 2/6/2 (example, based on load)
* Scaling Policies: CPU utilization-based (e.g., scale out > 70%, scale in < 30%).
* Launch Template: Defines instance configuration, user data for bootstrapping.
* Placement: Across multiple private application subnets for high availability.
* Type: application
* Scheme: internet-facing (for public apps) or internal (for internal apps).
* Listener Ports: HTTP:80 (redirect to HTTPS), HTTPS:443.
* Target Groups: Registered with Auto Scaling Groups for dynamic scaling.
* Security Group: Allows inbound traffic on 80/443 from 0.0.0.0/0 (for public ALB) or specific VPC CIDRs (for internal ALB).
* Purpose: Static content hosting, logging, backups, data lake.
* Configuration:
* Versioning: Enabled for critical data.
* Encryption: SSE-S3 by default, SSE-KMS for sensitive data.
* Public Access Block: Enabled by default (no public access).
* Lifecycle Policies: Define transitions to Glacier, deletion rules.
* Bucket Policy: Restrict access to specific IAM roles/users.
* Attached to EC2: Root volumes for OS, additional volumes for data (if not ephemeral).
* Type: gp3 (cost-effective general purpose, scalable IOPS/throughput).
* Encryption: Enabled by default using AWS KMS.
* Snapshots: Automated backups configured via AWS Backup or custom scripts.
PostgreSQL14.x)db.t3.small (scalable, e.g., db.m5.large for production).gp3 (scalable, e.g., 100GB initial, auto-scaling enabled).7 days (scalable up to 35 days).5432 from application security groups only. * Inbound: HTTP:80, HTTPS:443 from 0.0.0.0/0 (public ALB) or specific CIDRs/SGs (internal ALB).
* Outbound: All traffic.
* Inbound: HTTP:80, HTTPS:443 from ALB Security Group.
* Inbound: SSH:22 from specific bastion host/VPN IP (for management).
* Outbound: 5432 to Database Security Group.
* Outbound: 443 to S3/External APIs (via NAT Gateway).
* Inbound: PostgreSQL:5432 from Application Security Group.
* Outbound: None (or specific ports for replication/monitoring if required).
* Inbound: SSH:22 from specific trusted IP ranges.
* Outbound: SSH:22 to Application Security Group (for management).
* Principle of Least Privilege: Roles attached to EC2 instances granting only necessary permissions (e.g., S3 read/write, CloudWatch metrics, SSM access).
These "wireframes" describe the visual architecture diagrams that will accompany your Terraform deployment, providing a clear map of the infrastructure. We recommend using standard cloud provider iconography (e.g., AWS Architecture Icons) for clarity.
* Cloud Provider Region boundary.
* VPC boundary.
* Public vs. Private Subnets.
* Internet Gateway, NAT Gateways.
* Load Balancer (ALB).
* Auto Scaling Group (representing application servers).
* Database (RDS Multi-AZ).
* S3 (for storage/static assets).
* Arrows indicating primary data flow (e.g., User -> ALB -> App -> DB).
* VPC with all subnets clearly labeled (Public, Private App, Private DB) and their CIDR ranges.
* Internet Gateway, NAT Gateways, Route Tables.
* Security Groups as conceptual boundaries around components (ALB, App, DB).
* Inbound/outbound traffic flows between components and to/from the internet.
* Availability Zones clearly demarcated.
* ALB with listeners and target groups.
* Auto Scaling Group with multiple EC2 instances spread across AZs.
* Connection from application instances to the database.
* Connection to S3 for static assets or logging.
* Monitoring (CloudWatch) and Logging (CloudWatch Logs) integrations.
* RDS Multi-AZ deployment with primary and standby instances.
* Replication between primary and standby.
* Database subnets and their isolation.
* Security Group protecting the database.
* Connections from application servers to the database.
To ensure clarity and consistency across all architectural diagrams, we recommend the following visual guidelines:
These recommendations focus on improving the "user experience" for engineers and operations teams interacting with the Terraform codebase and the deployed infrastructure.
* Break down infrastructure into reusable, logical modules (e.g., vpc, ec2-app-tier, rds-postgres, s3-bucket).
* Each module should have a clear purpose, inputs, outputs, and README.md.
* This promotes reusability, reduces redundancy, and simplifies maintenance.
* Separate root modules for each environment (e./environments/dev, ./environments/stage, ./environments/prod).
Use terraform.tfvars or -dev.tfvars, *-prod.tfvars for environment-specific variable overrides.
* Implement consistent, descriptive naming conventions for all resources (e.g., `project-env-resource-purpose