502 BAD GATEWAY-k8s的cgroup限制了apache的可用内存
2024-08-23 03:43:19
1.release的组件逻辑图
2.表象:按F12,总是报502 BAD GATEWAY
3.nginx日志
[error] #: * upstream prematurely closed connection while reading response header from upstream, client: 83.234.185.35, server: tqy.hangzhouali.net.cn
4.又nginx日志假设是后端apache的问题,由于以前服务是正常的,所以假设nginx配置也没问题,apache的规则也没问题
5.去看apache日志
Resource temporarily unavailable: AH03104: apr_thread_create: unable to create worker thread
httpd apache centos Cannot allocate memory: AH00481: fork: Unable to fork new process
看样子apache不能fork新的线程
6.去k8s的主机看系统日志
[ +0.000001] kmem: usage 111176kB, limit 9007199254740988kB, failcnt
[ +0.000001] Memory cgroup stats for /kubepods/burstable/podbab37dc4-09ed-11ea-a19e-82943d98d7a4: cache:0KB rss:0KB rss_huge:0KB mapped_file:0KB swap:0KB inactive_anon:0KB active_anon:0KB inactive_file:0KB active
[ +0.000024] Memory cgroup stats for /kubepods/burstable/podbab37dc4-09ed-11ea-a19e-82943d98d7a4/b229faca56cfc16d36fcd6bc7d12ac01508830c3ecc01f1ba1e01d15d6db5946: cache:0KB rss:296KB rss_huge:0KB mapped_file:0KB
[ +0.000009] Memory cgroup stats for /kubepods/burstable/podbab37dc4-09ed-11ea-a19e-82943d98d7a4/d9a5f7981ad793e525179a2b7ed1858a29e3a8e1852fbb6f02ac130774db56f8: cache:328264KB rss:75332KB rss_huge:4096KB mapped
[ +0.000007] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
[ +0.000061] [ ] - pause
[ +0.000002] [ ] httpd
[ +0.000002] [ ] rotatelogs
[ +0.000002] [ ] rotatelogs
[ +0.000007] [] rotatelogs [ +0.000001] Memory cgroup out of memory: Kill process 3433 (httpd) score 1618 or sacrifice child
[ +0.001943] Killed process (httpd) total-vm:1603408kB, anon-rss:6256kB, file-rss:980kB, shmem-rss:12kB
[ +1.721253] cbr0: port (vethbeabb887) entered disabled state
[ +0.003202] device vethbeabb887 left promiscuous mode
7.得出结论:apache的容器的内存溢出,被cgroup限制了,应该是在腾讯云哪里设置内存的地方设置错了
一看,原来才给了512M.改为2048M后问题不再出现
最新文章
- JavaScript函数的4种调用方法详解
- ElasticSearch-5.0.0安装中文分词插件IK
- 探索ASP.NET MVC框架之路由系统
- linuxmint 17安装scim输入法
- Linux从零单排(二):setuptools、pip、anaconda2的环境配置
- jQuery Flickerplate 幻灯片
- How to push your code in git
- 【转】IOS --- OC与Swift混编
- C#调用IOS推送
- DataGridView自动行号
- 转: Nodejs概述
- Spring之ContextLoaderListener的作用
- BMP文件格式详解
- JavaScript高级程序设计---学习笔记(五)
- Python函数声明以及与其他编程语言数据类型的比较
- Android Toast 工具类
- nginx 1.4.3能直接升到1.8.1吗
- Light OJ 1007
- ES 使用小结
- Redis无法保存ef复杂对象