EKS Blueprint

EKS上可以装很多三方软件来定制化它的功能，用户需要管理多个EKS集群，上面运行着相同的网络策略、权限配置、外部AWS资源。很多用户想有一种解决方案，可以用Terraform或CDK来统一创建EKS集群并配置好这些环境。

EKS Blueprints除了描述控制平面和数据平面的配置外，还可以在里面声明add-on和IaC配置。下面是一个创建在三个AZ的EKS集群，它里面安装好了各种add-on和不同团队的工作负载：

img

使用EKS Blueprint，不仅可以创建EKS，还可以把各种add-on创建好，并且为这些add-on使用IRSA配置好IAM policy、IAM Role和service account

在https://github.com/aws-ia/terraform-aws-eks-blueprints/tree/main/examples 里有很多案例，如结合Karpenter、Loki、Appmesh等外部组件

EKS Blueprints示例

下面是一个Terraform示例，它创建一个托管节点组，配置好vpc-cni, coredns, kube-proxy, aws-load-balancer-controller, metrics server 和 cluster-autoscaler ，可见想要安装add-on，只要把它的值配置成true就可以：

module "eks_blueprints" {
  source = "github.com/aws-ia/terraform-aws-eks-blueprints?ref=v4.0.2"

  # EKS Cluster VPC and Subnet mandatory config
  vpc_id             = <vpc_id>
  private_subnet_ids = <private_subnet_ids>

  # EKS CLUSTER VERSION
  cluster_version = "1.21"

  # EKS MANAGED NODE GROUPS
  managed_node_groups = {
    mg_5 = {
      node_group_name = "managed-ondemand"
      instance_types  = ["m5.large"]
      min_size        = "2"
    }
  }
}

# Add-ons
module "kubernetes_addons" {
  source = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons?ref=v4.0.2"

  eks_cluster_id = module.eks_blueprints.eks_cluster_id

  # EKS Add-ons
  enable_amazon_eks_vpc_cni            = true
  enable_amazon_eks_coredns            = true
  enable_amazon_eks_kube_proxy         = true
  enable_amazon_eks_aws_ebs_csi_driver = true

  # Self-managed Add-ons
  enable_aws_for_fluentbit            = true
  enable_aws_load_balancer_controller = true
  enable_aws_efs_csi_driver           = true
  enable_cluster_autoscaler           = true
  enable_metrics_server               = true
}

如果使用CDK，则代码如下：

const app = new cdk.App();

const stackId = "<stack_id>";

// By default will provision in a new VPC
blueprints.EksBlueprint.builder()
    .region('us-west-2')
    .version(eks.KubernetesVersion.V1_21)
    .addOns(
        new blueprints.addons.VpcCniAddOn(),
        new blueprints.addons.CoreDnsAddOn(),
        new blueprints.addons.KubeProxyAddOn(),
        
        // Self-managed Add-ons
        new blueprints.addons.AwsForFluentBitAddOn(),
        new blueprints.addons.AwsLoadBalancerControllerAddOn(),
        new blueprints.addons.ClusterAutoScalerAddOn(),
        new blueprints.addons.EfsCsiDriverAddOn(),
        new blueprints.addons.MetricsServerAddOn()
    )
    .build(app, stackId);

每个add-on都是开源的Helm库，EKS Blueprints为每个add-on配置好了IRSA

Worker节点配置示例

module "eks_blueprints" {
  ...
 # Managed Node Groups
  managed_node_groups = {
    mg_5 = {
      node_group_name = "managed-ondemand"
      instance_types  = ["m5.large"]
      min_size        = "2"
      max_size        = "5"
    }
  }

  # Fargate Profiles
  fargate_profiles = {
    default = {
      fargate_profile_name = "default"
      fargate_profile_namespaces = [{
        namespace = "default"
      }]
      additional_tags = { ExtraTag = "Fargate" }
    }
  }
}

Multi-team clusters

如果想限制某个团队的权限访问，底层是使用原生的Role、Role binding来管理namespace权限：

module “eks-blueprints” {
  …
  application_teams = {
    team-blue = {
      "labels" = {
        "appName" = "blue-team-app",
      }
      "quota" = {
        "requests.cpu"    = "2000m",
        "requests.memory" = "4Gi",
        "limits.cpu"      = "2000m",
        "limits.memory"   = "8Gi",
      }
      users = ["arn:aws:iam::<aws-account-id>:user/team-blue-user"]
    }
  }

  platform_teams = {
    platform_admin = {
      users = ["arn:aws:iam::<aws-account-id>:user/platform-user"]
    }
  }
}

EKS Blueprints目前支持用Terraform或CDK来实现，github地址如下：

EKS Blueprints for Terraform 、 EKS Blueprints for CDK