首页/资讯中心/详情

Go-Kratos 项目 SonarQube、Jenkins 与 K8s CI/CD 实践

Go-Kratos 项目 SonarQube、Jenkins 与 K8s CI/CD 实践
📅 发布时间：2026/6/18 19:45:50

Go-Kratos 项目 SonarQube、Jenkins 与 K8s CI/CD 实践

环境规划

服务器名称 操作系统 服务名称 IP 功能描述
devops ubuntu-22.04.5 GitLab 10.1.1.252:30080 代码仓库
devops ubuntu-22.04.5 Harbor 10.1.1.252:18090 镜像仓库
devops ubuntu-22.04.5 Jenkins 10.1.1.252:8080 CI/CD
devops ubuntu-22.04.5 SonarQube 10.1.1.252:9000 代码质量检测
master ubuntu-22.04.5 K8S-master 10.1.1.2 K8S控制节点
node1 ubuntu-22.04.5 k8s-node1 10.1.1.3 k8s工作节点1
node2 ubuntu-22.04.5 k8s-node2 10.1.1.4 k8s工作节点2

默认上述服务都已经搭建完毕

CI/CD解决方案架构图

image-20230713162047388

Harbor配置

Docker配置

由于需要通过docker构建镜像并且推送到镜像仓库中, 需要建立docker与镜像仓库建立认证关系

# 添加认证地址
vim /etc/docker/daemon.json
{
"insecure-registries": ["10.1.1.252:18090"]
}# 添加认证地址之后,重启docker
systemctl daemon-reload&& systemctl restart docker# 进行登录认证
docker login  -u admin -p Harbor12345 http://10.1.1.252:18090

image-20250826170606142

K8S配置

由于需要从镜像仓库拉取镜像创建Pod, 需要建立K8S与镜像仓库认证关系

# 创建命名空间
kubectl create ns kube-devops# 创建secret
kubectl create secret docker-registry harbor-secret --docker-server=10.1.1.252:18090 --docker-username=admin --docker-password=Harbor12345 -n kube-devops# 查看创建的secret
kubectl get secret -n kube-devops

image-20250826171925404

jenkins配置

镜像构建

由于后期需要用到go,node和sonar-scanner,重新制作jenkins

FROM jenkins/jenkins:latest
USER root
RUN sed -i 's@deb.debian.org@repo.huaweicloud.com@g' /etc/apt/sources.list.d/debian.sourcesRUN apt-get update -y && apt-get install -y iputils-ping curl ca-certificates vim wget unzip \&& curl -fsSL https://deb.nodesource.com/setup_18.x | bash - \&& apt-get install -y nodejs \&& node -v && npm -v \&& rm -rf /var/lib/apt/lists/*# Copy local Go and SonarScanner archives into the image
COPY go1.24.0.linux-arm64.tar.gz /tmp/go.tar.gz
COPY sonar-scanner-cli-7.2.0.5079-linux-aarch64.zip /tmp/sonar-scanner.zip# Install Go
RUN tar -C /usr/local -xzf /tmp/go.tar.gz \&& rm -f /tmp/go.tar.gz# Install SonarScanner
RUN unzip -q /tmp/sonar-scanner.zip -d /opt \&& rm -f /tmp/sonar-scanner.zip \&& mv /opt/sonar-scanner-* /usr/local/sonar-scanner# Environment variables for Go and SonarScanner
ENV GOROOT=/usr/local/go
ENV GOPATH=/var/jenkins_home/go
ENV SONAR_SCANNER_HOME=/usr/local/sonar-scanner
ENV PATH=$SONAR_SCANNER_HOME/bin:$GOROOT/bin:$GOPATH/bin:/usr/local/bin:$PATH
ENV GOPROXY=https://goproxy.cn,direct
ENV GO111MODULE=on

插件下载

  1. Build Authorization Token Root
  2. Gitlab
  3. SonarQube Scanner
  4. Node and Label parameter
  5. Kubernetes
  6. docker
  7. Config File Provider
  8. Git Paramete
  9. Blue Ocean
  10. Git
  11. Pipeline Stage View

插件配置

SonarQube Scanner

  1. 登录sonarqube之后点击我的账号-->安全---->生成令牌

    ![image-20250826191424730](/Users/sr/Library/Application Support/typora-user-images/image-20250826191424730.png)

  2. 在jenkins中,创建认证凭证

    image-20250826191530207

  3. 点击全局凭证, 并且添加凭证

image-20250826191707581

image-20250826191847209

  1. 进入jenkins系统配置,搜索sonarqube server

![image-20250826190608915](/Users/sr/Library/Application Support/typora-user-images/image-20250826190608915.png)

  1. 点击Add SonarQube添加凭证

image-20250826192258696

gitlab

  1. 点击全局凭证, 并且添加凭证

image-20250826202156573

kubernetes

  1. 点击cloud---new cloud

image-20250826202630646

  1. 选择kubernetes类型

image-20250826202701126

  1. 查看K8S地址
kubectl cluster-info

image-20250826211836558

  1. 配置域名解析

    由于我这边对外提供的是域名, 需要再jenkins服务器中填写域名解析

    vim /etc/hosts
10.1.1.2  lb.kubesphere.local

  2. 配置认证凭证

    配置rbac权限

    #创建serviceaccounts
kubectl create sa jenkins
#对jenkins做cluster-admin绑定
kubectl create clusterrolebinding jenkins --clusterrole cluster-admin --serviceaccount=default:jenkins

    获取token, kubernetes-plugin与k8s连接时，并不是直接使用serviceaccount，而是通过token。因此我们需要获取serviceaccount:jenkins对应的token。

    # 查看sa命名空间
kubectl get sa -n default# 查看secret
kubectl describe sa jenkins -n default# 获取token
kubectl describe secrets jenkins-token-82fv9 -n default

    image-20250826222929045

​ 配置凭证

image-20250826223036211

  1. 获取K8S证书key

    yq e '.clusters[0].cluster.certificate-authority-data' /root/.kube/config | base64 -d

    image-20250826223212562

  2. 配置K8S连接

image-20250826223321511

  1. 点击测试

image-20250826223351534

Managed files

  1. 点击创建新的配置文件

![image-20250826232901293](/Users/sr/Library/Application Support/typora-user-images/image-20250826232901293.png)

  1. content中填写config文件值
cat /root/.kube/config

![image-20250826232931980](/Users/sr/Library/Application Support/typora-user-images/image-20250826232931980.png)

CICD配置

jenkins

创建项目

image-20250826224845810

  1. 点击高级按钮

image-20250826225619390

  1. 生成token

    image-20250826225654566

  2. 进入gitlab中选择对应的项目,配置webhook

image-20250826230929952

点击测试选择推送事件,出现200则说明配置成功了

image-20250826231107102

流水线配置

  1. 在流水线中选择SCM并且连接gitlab中项目仓库, 认证选择连接gitlab的认证, 脚本路径选择Jenkinsfile

image-20250826231936848

jenkinsfile编写

pipeline {agent {node {label 'master' // 在标记为master的Jenkins节点上执行}}// 定义全局环境变量environment {// Docker镜像仓库地址DOCKER_HARBOR_REGISTRY = '10.1.1.252:18090'// Harbor私有仓库的凭证ID（在Jenkins中配置的用户名密码凭证）DOCKER_HARBOR_CREDENTIAL_ID = 'harbor-user-password'// Git代码仓库地址GIT_REPO_URL = 'http://10.1.1.252:30080/root/guliedu.git'// Git仓库的凭证ID（在Jenkins中配置的用户名密码凭证）GIT_CREDENTIAL_ID = 'gitlab-username-password'// SonarQube凭证SONARQUBE_CREDENTIAL_ID = 'sonarqube-token'// SonarQubeURLSONAR_HOST_URL = 'http://10.1.1.252:9000'// Kubernetes集群配置文件的凭证IDKUBECONFIG_CREDENTIAL_ID = '93e074ab-55e4-45c8-b745-29131928f583'// Docker Hub命名空间（用于推送镜像）DOCKERHUB_NAMESPACE = 'guliedu'// 应用程序名称PROJECT_NAME = 'kratos-guliedu-backend'// 全局应用环境变量IS_DEV_MODE = "${BRANCH_NAME == 'main' ? 'false' : 'true'}"  // 根据分支动态设置开发模式CONSUL_HOST = '10.1.1.254:8500'         // Consul服务地址FLUENTD_HOST = '10.1.1.254:24224'       // Fluentd日志收集地址}// 定义流水线参数，用户可以在触发构建时指定parameters {// 分支名称参数，使用下拉选择，默认值为devchoice(name: 'BRANCH_NAME',choices: ['dev', 'main', 'test'],description: '请选择要发布的分支名称')// 手动指定要构建的服务（下拉选择）。选择 auto 表示自动探测改动服务choice(name: 'TARGET_SERVICES',choices: ['auto', 'edu', 'file', 'learning', 'trade', 'user', 'video'],description: '请选择要构建的服务（auto = 自动探测改动服务）')// 标签名称参数，用于版本标记，必须以v开头string(name: 'TAG_NAME', defaultValue: 'snapshot', description: '标签名称，必须以 v 开头，例如：v1、v1.0.0')}// 填写流水线步骤stages {// 克隆代码stage('Clone Code') {steps {checkout([$class: 'GitSCM',branches: [[name: "${BRANCH_NAME}"]],extensions: [],userRemoteConfigs: [[credentialsId: "${GIT_CREDENTIAL_ID}",url: "${GIT_REPO_URL}"]]])}}// 单元测试stage('Unit Test') {steps {// 显示当前环境变量（用于调试）sh 'echo "=== Environment Variables ==="'sh 'echo "IS_DEV_MODE: $IS_DEV_MODE"'sh 'echo "CONSUL_HOST: $CONSUL_HOST"'sh 'echo "FLUENTD_HOST: $FLUENTD_HOST"'sh 'echo "BRANCH_NAME: $BRANCH_NAME"'sh 'echo "========================="'// 运行单元测试sh 'go test ./...'}}// 代码质量分析stage('Sonarqube Analysis') {steps {// 设置SonarQube环境变量script {// SonarQube服务器地址env.SONAR_HOST_URL = "${SONAR_HOST_URL}"// 项目唯一标识符，格式：项目名-分支名env.SONAR_PROJECT_KEY = "${PROJECT_NAME}-${BRANCH_NAME}"// 项目显示名称env.SONAR_PROJECT_NAME = "${PROJECT_NAME}-${BRANCH_NAME}"// 项目版本号，使用Jenkins构建号env.SONAR_PROJECT_VERSION = "${BUILD_NUMBER}"// 源代码根目录env.SONAR_SOURCES = '.'// 排除的文件和目录（不进行代码质量分析）// **/third_party/** - 第三方库文件// **/doc/** - 文档目录// **/api/** - API定义文件（protobuf生成的接口定义）// **/deploy/** - 部署配置文件// **/httpclient/** - HTTP客户端测试文件// **/*_test.go - Go测试文件// **/*_pb.go - protobuf生成的Go文件// **/*_validate.go - protobuf验证文件env.SONAR_EXCLUSIONS = '**/third_party/**,**/doc/**,**/api/**,**/deploy/**,**/httpclient/**,**/*_test.go,**/*_pb.go,**/*_validate.go,**/*_gen.go,**/*.js,**/*.jsx,**/*.ts,**/*.tsx,**/package*.json,**/webpack*.js,**/vite*.config.*'// 排除覆盖率统计的文件和目录// **/*_test.go - 测试文件不计算覆盖率// **/third_party/** - 第三方库不计算覆盖率env.SONAR_COVERAGE_EXCLUSIONS = '**/*_test.go,**/*_pb.go,**/*_validate.go,**/third_party/**,**/api/**'// 禁用 JS/TS 分析，避免 Node 依赖env.SONAR_JS_EXCLUSIONS = '**/*'env.SONAR_TS_EXCLUSIONS = '**/*'}// 打印SonarQube配置信息sh 'echo "=== SonarQube Configuration ==="'sh 'echo "SONAR_HOST_URL: $SONAR_HOST_URL"'sh 'echo "SONAR_PROJECT_KEY: $SONAR_PROJECT_KEY"'sh 'echo "SONAR_PROJECT_NAME: $SONAR_PROJECT_NAME"'sh 'echo "SONAR_PROJECT_VERSION: $SONAR_PROJECT_VERSION"'sh 'echo "SONAR_SOURCES: $SONAR_SOURCES"'sh 'echo "==============================="'withCredentials([string(credentialsId: "${SONARQUBE_CREDENTIAL_ID}", variable: 'SONAR_TOKEN')]) {withSonarQubeEnv('sonarqube') {sh '''sonar-scanner \-Dsonar.host.url=$SONAR_HOST_URL \-Dsonar.token=$SONAR_TOKEN \-Dsonar.projectKey=$SONAR_PROJECT_KEY \-Dsonar.projectName=$SONAR_PROJECT_NAME \-Dsonar.projectVersion=$SONAR_PROJECT_VERSION \-Dsonar.sources=$SONAR_SOURCES \-Dsonar.exclusions=$SONAR_EXCLUSIONS \-Dsonar.go.coverage.reportPaths=coverage.out \-Dsonar.sourceEncoding=UTF-8 \-Dsonar.javascript.exclusions=$SONAR_JS_EXCLUSIONS \-Dsonar.typescript.exclusions=$SONAR_TS_EXCLUSIONS \'''}timeout(unit: 'MINUTES', activity: true, time: 5) {waitForQualityGate 'true'}}}}// 构建并且推送镜像stage('Build & Push Image') {steps {withCredentials([usernamePassword(credentialsId: "${DOCKER_HARBOR_CREDENTIAL_ID}", passwordVariable: 'DOCKER_PASSWORD', usernameVariable: 'DOCKER_USERNAME')]) {script {// 获取用户选择的服务def selected = params.TARGET_SERVICES?.trim()echo "TARGET_SERVICES selected: ${selected}"def services = []// 根据用户选择决定构建策略if (selected == 'auto') {// 自动检测模式：检测代码变更的服务// 1) 优先用上次成功提交区间def range = env.GIT_PREVIOUS_SUCCESSFUL_COMMIT ? "${env.GIT_PREVIOUS_SUCCESSFUL_COMMIT}..HEAD" : ""def changed = ""if (range) {changed = sh(returnStdout: true, script: "git diff --name-only ${range}").trim()echo "Changed by range ${range}:\n${changed}"}// 2) 退回 merge-baseif (!changed) {sh 'git fetch --no-tags --all || true'def base = sh(returnStdout: true, script: 'git merge-base HEAD origin/$BRANCH_NAME').trim()echo "merge-base: ${base}"changed = sh(returnStdout: true, script: "git diff --name-only ${base} HEAD").trim()echo "Changed by merge-base:\n${changed}"}// 3) 最后兜底：上一提交到当前if (!changed) {changed = sh(returnStdout: true, script: "git diff --name-only HEAD^..HEAD || true").trim()echo "Changed by HEAD^..HEAD:\n${changed}"}// 从变更文件中提取服务名称if (changed) {services = changed.split('\n').findAll { it.startsWith('app/') && it.split('/').size() > 1 }.collect { it.split('/')[1] }.unique()}echo "Auto-detected services: ${services}"} else {// 手动选择模式：构建指定服务services = [selected]echo "Manual selection: ${services}"}// 如果没有需要构建的服务，则跳过构建步骤if (!services || services.isEmpty()) {echo 'No service selected or detected under app/. Skip build.'currentBuild.result = 'SUCCESS'return}// 登录 Docker Registrysh 'echo "$DOCKER_PASSWORD" | docker login $DOCKER_HARBOR_REGISTRY -u "$DOCKER_USERNAME" --password-stdin'// 逐个服务构建并推送for (svc in services) {// 根据分支确定镜像标签def tag = (env.BRANCH_NAME == 'main') ? 'latest' : "snapshot-${env.BRANCH_NAME}-${svc}-${env.BUILD_NUMBER}"sh """echo Building service: ${svc} with tag: ${tag}docker build -f app/${svc}/service/Dockerfile \\--build-arg APP_RELATIVE_PATH=${svc}/service \\--build-arg APP_VERSION=${BUILD_NUMBER} \\--build-arg IS_DEV_MODE=${IS_DEV_MODE} \\--build-arg CONSUL_HOST=${CONSUL_HOST} \\--build-arg FLUENTD_HOST=${FLUENTD_HOST} \\-t ${DOCKER_HARBOR_REGISTRY}/${DOCKERHUB_NAMESPACE}/${svc}:${tag} .docker push ${DOCKER_HARBOR_REGISTRY}/${DOCKERHUB_NAMESPACE}/${svc}:${tag}"""}}}}}// K8S 部署stage("K8S Deploy"){stages {sh "echo k8s deploy steps"}}}
}