Question

How to not destroy my opensearch domain when performing 'terraform destroy'?

I have created many different AWS resources with terraform such as ECS, VPC, EC2, opensearch... I'm in a debugging phase and I need to frequently create/destroy dozens of resources besides opensearch (I don't want to destroy opensearch because it's very slow to create/delete)

I used "terraform state rm " to remove opensearch from state management, but it associated a security group:

resource "aws_security_group" "opensearch" {
  name   = "${terraform.workspace}-opensearch"
  vpc_id = local.vpc_id
}

I used "terraform state rm " again to remove this sg from state, but after that when I run terraform apply, terraform still seems to be trying to create this sg:

Error: creating Security Group (default-opensearch): InvalidGroup.Duplicate: The security group 'default-opensearch' already exists for VPC 'xxxxx'
 2  63  2
1 Jan 1970

Solution

 3

I use three or four separate deployments with their own code directories and their own state files, categorized by how often I might want/need to create and destroy the objects and the relative cost of recreating them.

  1. setup - things that need set up before everything else: e.g. SES, VPC, secrets
  2. database: databases and their ancillary resources. Database might take ten minutes to create but could take hours/days to reload.
  3. storage: s3 buckets, EFS. Ephemeral-ish, can rapidly recreate but would be annoying and often unnecessary.
  4. application: everything else.

If I need the results of one apply as input to the next, I'll either use a resource's data block, or a terraform_remote_state data block.

2024-07-02
Len Jaffe

Solution

 0

I faced an issue where Terraform was trying to create a security group that already existed. Here's how I resolved the problem:

1. Verify the Existing Security Group:

Firstly, I checked if the security group already existed in my AWS environment. I used the AWS Management Console, but you can also use the AWS CLI with the command:

aws ec2 describe-security-groups --group-names default-opensearch

2. Delete the Security Group Manually:

Since the security group existed, I deleted it manually. This can be done using the AWS Management Console or the AWS CLI:

aws ec2 delete-security-group --group-name default-opensearch

3. Update Terraform Configuration:

I then updated my Terraform configuration to prevent it from trying to create the same security group again. Here’s the updated code:

resource "aws_security_group" "opensearch" {
  name   = "${terraform.workspace}-opensearch-${random_string.suffix.result}"
  vpc_id = local.vpc_id
}

resource "random_string" "suffix" {
  length  = 8
  special = false
}

This configuration adds a random suffix to the security group name, avoiding any naming conflicts in the future.

Re-import the Resource if Necessary:

If I wanted to keep the existing security group but manage it with Terraform again, I would re-import it into the state using:

terraform import aws_security_group.opensearch <security_group_id>

Apply Terraform Changes:

Finally, I ran terraform apply again to create any new resources. This time, Terraform did not attempt to recreate the existing security group, and everything worked as expected.

2024-07-02
tushar shah