欢迎来到尧图网

客户服务 关于我们

您的位置:首页 > 汽车 > 时评 > k8s部署jenkins集群,启动slave pod里除jnlp以外的容器失败

k8s部署jenkins集群,启动slave pod里除jnlp以外的容器失败

2024/10/26 7:28:21 来源:https://blog.csdn.net/zhuganlai168/article/details/142786455  浏览:    关键词:k8s部署jenkins集群,启动slave pod里除jnlp以外的容器失败

一、错误详情

java.net.ProtocolException: Expected HTTP 101 response but was '500 Internal Server Error'at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:229)at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:196)
Also:   java.lang.Throwable: waiting hereat io.fabric8.kubernetes.client.utils.Utils.waitUntilReady(Utils.java:169)at io.fabric8.kubernetes.client.utils.Utils.waitUntilReadyOrFail(Utils.java:180)at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.exec(PodOperationsImpl.java:332)at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.exec(PodOperationsImpl.java:85)at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:425)at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:328)at hudson.Launcher$ProcStarter.start(Launcher.java:507)at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:176)at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:132)at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:324)at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:319)at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:193)at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)at jdk.internal.reflect.GeneratedMethodAccessor913.invoke(Unknown Source)at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)at java.base/java.lang.reflect.Method.invoke(Unknown Source)at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:163)at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:158)at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)at jdk.internal.reflect.GeneratedMethodAccessor250.invoke(Unknown Source)at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)at java.base/java.lang.reflect.Method.invoke(Unknown Source)at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)at com.cloudbees.groovy.cps.Next.step(Next.java:83)at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:403)at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:97)at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:315)at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:279)at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)at java.base/java.util.concurrent.FutureTask.run(Unknown Source)at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)at java.base/java.util.concurrent.FutureTask.run(Unknown Source)at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)at java.base/java.lang.Thread.run(Unknown Source)
Caused: io.fabric8.kubernetes.client.http.WebSocketHandshakeExceptionat io.fabric8.kubernetes.client.okhttp.OkHttpWebSocketImpl$BuilderImpl$1.onFailure(OkHttpWebSocketImpl.java:66)at okhttp3.internal.ws.RealWebSocket.failWebSocket(RealWebSocket.java:571)at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:198)at okhttp3.RealCall$AsyncCall.execute(RealCall.java:203)at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
Caused: io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:103)at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:97)at io.fabric8.kubernetes.client.utils.Utils.waitUntilReady(Utils.java:170)at io.fabric8.kubernetes.client.utils.Utils.waitUntilReadyOrFail(Utils.java:180)at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.exec(PodOperationsImpl.java:332)at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.exec(PodOperationsImpl.java:85)at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:425)at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:328)at hudson.Launcher$ProcStarter.start(Launcher.java:507)at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:176)at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:132)at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:324)at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:319)at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:193)at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)at jdk.internal.reflect.GeneratedMethodAccessor913.invoke(Unknown Source)at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)at java.base/java.lang.reflect.Method.invoke(Unknown Source)at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:163)at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:158)at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)at WorkflowScript.run(WorkflowScript:225)at ___cps.transform___(Native Method)at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)at jdk.internal.reflect.GeneratedMethodAccessor250.invoke(Unknown Source)at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)at java.base/java.lang.reflect.Method.invoke(Unknown Source)at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)at com.cloudbees.groovy.cps.Next.step(Next.java:83)at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:403)at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:97)at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:315)at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:279)at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)at java.base/java.util.concurrent.FutureTask.run(Unknown Source)at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)at java.base/java.util.concurrent.FutureTask.run(Unknown Source)at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)at java.base/java.lang.Thread.run(Unknown Source)
Finished: FAILURE

在这里插入图片描述

其对应的源码见kubernetes-client

在这里插入图片描述

二、jenkins集群

在这里插入图片描述
slave pod 下的容器包括jnlp和maven,查看均运行正常。

三、错误原因

使用k8s部署jenkins集群,包括master节点,slave节点是弹性扩容缩容。
在创建slave pod后,第一步是成功拉取代码,这一步没什么问题。
第二步,需要调用slave pod中的maven容器,这一步报错。

slave pod除了有jnlp容器外,还有maven等容器。当然,不同语言的pod模板,包含的容器列表不一。

在排查该错误的时候,有考虑过是否因为jenkins插件升级导致不兼容。

然而,我们把jenkins恢复到之前的备份版本后,仍旧报错。

期间尝试通过查看jenkins日志,也并没有看到更详细的错误。

最后在查看k8s的kube日志,是k8s集群底层报错导致。

四、总结

如果你也遇到这个报错,除了查看jenkins插件的源码外,查看具体组件的日志寻找错误原因。

这点非常重要,也提醒我们在编写程序的时候,必要地打印关键日志,帮助运维软件的同事分析问题。

版权声明:

本网仅为发布的内容提供存储空间,不对发表、转载的内容提供任何形式的保证。凡本网注明“来源:XXX网络”的作品,均转载自其它媒体,著作权归作者所有,商业转载请联系作者获得授权,非商业转载请注明出处。

我们尊重并感谢每一位作者,均已注明文章来源和作者。如因作品内容、版权或其它问题,请及时与我们联系,联系邮箱:809451989@qq.com,投稿邮箱:809451989@qq.com