压测:celey backend为rabbitmq pk redis
使用celery的backend异步获取结果,本文使用rabbitmq 和 redis分别作为backend,代码对比如下
from celery import Celery, platforms
import time
import os
from datetime import datetime app = Celery('proj',
broker='amqp://admin:admin@ip:5672', //rabbitmq
backend='amqp://admin:admin@ip:5672', //redis
//backend='redis://ip:10013/0',
include=['tasks']
)
app.conf.update(
CELERYD_PREFETCH_MULTIPLIER=,
CELERY_ACKS_LAT = True,
) @app.task
def fun_1(n):
return {"task_id": "0a14c9ac91de419880c4499a8f57418e",
"status": ,
"desc": str(n) } @app.task
def chk_total_callback(tasks_rets):
end_time = datetime.now()
print "end_time", end_time
return if __name__ == "__main__":
app.start()
发送消息
from tasks import *
from celery import group, chain
from datetime import datetime
import time #job_group = group([fun_1.s(i) for i in range()])
beg_time = datetime.now()
for j in range():
job_group = group([fun_1.s(i) for i in range()])
chain(job_group, chk_total_callback.s())()
#time.sleep(0.25) print "beg_time:", beg_time
压测场景一:
注:1000*20:一次发送1000次,分片20
力度 |
backend |
耗时 |
内存峰值 |
均值 |
1000*20 |
rabbitmq |
18:40:36-18:41:21 |
0.504G |
45ms |
redis |
18:48:37-18:51:21 |
11.84M |
82ms |
|
2000*20 |
rabbitmq |
21:57:06 -21:58:41 |
1.008G |
47.5ms |
redis |
18:58:52-19:04:10 |
20.26M |
159ms |
|
3000*20 |
rabbitmq |
21:51:40-21:54:03 |
1.512G |
47.6ms |
redis |
19:43:31-19:51:41 |
28.18M |
163ms |
|
4000*20 |
rabbitmq |
20:59:12-21:02:28 |
2.268G |
49ms |
redis |
20:00:00-20:10:52 |
37.10M |
163ms |
压测场景二:
500*20: 500次,每次间隔1s,分片20
力度 |
backend |
耗时 |
内存峰值 |
耗时 |
500*20 |
rabbitmq |
22:34:28-22:43:01 |
0.504G |
513s |
redis |
22:49:23-22:57:56 |
7.63M |
513s |
压测场景三:
8000 * 0.25 = 2000s, 任务数:8000 8000 * 20 + 8000 = 8000 * 21=168000 * 2=336000
力度 |
backend |
耗时 |
内存峰值 |
耗时 |
8000 * 0.25 = 2000s *20 |
rabbitmq |
10:51:38-11:28:06 |
1.26G |
2188s |
redis |
11:56:17-12:33:00 |
65.96M |
2203s |
|
16000 * 0.25 = 4000s *20 |
rabbitmq |
18:26:39-19:39:37 |
4.28G |
73*60=4380 |
redis |
20:13:57-22:49:33 |
130.69M |
9360s |
最新文章
- 自己写的一个Pager分页组件,WebForm,Mvc都适用
- 【原】iOS学习之Quartz2D(1)
- 【翻译六】java-连接和实例
- asp.net mvc route 中新发现的小技巧
- js替换字符指定字符方法
- VsFTP+本地文件认证+SSl实现虚拟用户配置
- JavaScript中七种函数调用方式及对应 this 的含义
- SQL server 分离数据库与删除数据库的区别
- 简洁的PHP操作SQLite类
- Android开发之漫漫长途 IX——彻底掌握Binder
- Now trying to drop the old temporary tablespace, the session hangs.
- javascript 常用方法 解析URL,补充前导字符
- FluentDataflow - Fluent Style TPL Dataflow
- Django MTV 开发模式 + 数据库配置
- WebService的几种验证方式
- Delphi疑难问题整理!!!
- css控制div上浮下落
- pandas数组和numpy数组在使用索引数组过滤数组时的区别
- Ubuntu16.04安装Mininet
- centos6下通用二进制格式安装MySQL过程