python使用multiprocessing进行多进程编程（1）

multiprocessing模块实现了对多进程编程的封装，让我们可以非常方便的使用多进程进行编程。它的使用方法非常类似threading模块。

1.创建一个进程

import multiprocessing

def worker():

    """worker function"""

    print 'Worker'

    return

if __name__ == '__main__':

    jobs = []

    for i in range(5):

        p = multiprocessing.Process(target=worker)

        jobs.append(p)

        p.start()

输出结果：

$ python multiprocessing_simple.py

Worker

Worker

Worker

Worker

Worker

2.传递参数给子进程

import multiprocessing

def worker(num):

    """thread worker function"""

    print 'Worker:', num

    return

if __name__ == '__main__':

    jobs = []

    for i in range(5):

        p = multiprocessing.Process(target=worker, args=(i,))

        jobs.append(p)

        p.start()

运行结果：

$ python multiprocessing_simpleargs.py

Worker: 0

Worker: 1

Worker: 2

Worker: 3

Worker: 4

注意：所传递的参数必需是可以序列化的

3.后台进程

默认情况下，主进程会等待所有子进程运行完毕后退出。如果需要让子进程独立在后台运行，可以如下操作。

import multiprocessing

import time

import sys

def daemon():

    p = multiprocessing.current_process()

    print 'Starting:', p.name, p.pid

    sys.stdout.flush()

    time.sleep(2)

    print 'Exiting :', p.name, p.pid

    sys.stdout.flush()

def non_daemon():

    p = multiprocessing.current_process()

    print 'Starting:', p.name, p.pid

    sys.stdout.flush()

    print 'Exiting :', p.name, p.pid

    sys.stdout.flush()

if __name__ == '__main__':

    d = multiprocessing.Process(name='daemon', target=daemon)

    d.daemon = True

    n = multiprocessing.Process(name='non-daemon', target=non_daemon)

    n.daemon = False

    d.start()

    time.sleep(1)

    n.start()

运行结果：

$ python multiprocessing_daemon.py

Starting: daemon 13866

Starting: non-daemon 13867

Exiting : non-daemon 13867

可以看到，daemon并没有exiting输出。

4.等待进程

import multiprocessing

import time

import sys

def daemon():

    print 'Starting:', multiprocessing.current_process().name

    time.sleep(2)

    print 'Exiting :', multiprocessing.current_process().name

def non_daemon():

    print 'Starting:', multiprocessing.current_process().name

    print 'Exiting :', multiprocessing.current_process().name

if __name__ == '__main__':

    d = multiprocessing.Process(name='daemon', target=daemon)

    d.daemon = True

    n = multiprocessing.Process(name='non-daemon', target=non_daemon)

    n.daemon = False

    d.start()

    time.sleep(1)

    n.start()

    d.join()

    n.join()

join方法会等待对应的进程执行完毕，然后进行下一步操作。

运行结果：

$ python multiprocessing_daemon_join.py

Starting: non-daemon

Exiting : non-daemon

Starting: daemon

Exiting : daemon

join方法还可以指定timeout参数，等待一段时间，如果子进程还没有执行完毕，那么就会返回。

import multiprocessing

import time

import sys

def daemon():

    print 'Starting:', multiprocessing.current_process().name

    time.sleep(2)

    print 'Exiting :', multiprocessing.current_process().name

def non_daemon():

    print 'Starting:', multiprocessing.current_process().name

    print 'Exiting :', multiprocessing.current_process().name

if __name__ == '__main__':

    d = multiprocessing.Process(name='daemon', target=daemon)

    d.daemon = True

    n = multiprocessing.Process(name='non-daemon', target=non_daemon)

    n.daemon = False

    d.start()

    n.start()

    d.join(1)

    print 'd.is_alive()', d.is_alive()

    n.join()

5.关闭子进程

import multiprocessing

import time

def slow_worker():

    print 'Starting worker'

    time.sleep(0.1)

    print 'Finished worker'

if __name__ == '__main__':

    p = multiprocessing.Process(target=slow_worker)

    print 'BEFORE:', p, p.is_alive()

    p.start()

    print 'DURING:', p, p.is_alive()

    p.terminate()

    print 'TERMINATED:', p, p.is_alive()

    p.join()

    print 'JOINED:', p, p.is_alive()

输出：

$ python multiprocessing_terminate.py

BEFORE: <Process(Process-1, initial)> False

DURING: <Process(Process-1, started)> True

TERMINATED: <Process(Process-1, started)> True

JOINED: <Process(Process-1, stopped[SIGTERM])> False

6.进程返回状态

0，没有错误产生

>0，进程产生了一个错误

<0，进程被杀死

import multiprocessing

import sys

import time

def exit_error():

    sys.exit(1)

def exit_ok():

    return

def return_value():

    return 1

def raises():

    raise RuntimeError('There was an error!')

def terminated():

    time.sleep(3)

if __name__ == '__main__':

    jobs = []

    for f in [exit_error, exit_ok, return_value, raises, terminated]:

        print 'Starting process for', f.func_name

        j = multiprocessing.Process(target=f, name=f.func_name)

        jobs.append(j)

        j.start()

    jobs[-1].terminate()

    for j in jobs:

        j.join()

        print '%s.exitcode = %s' % (j.name, j.exitcode)

输出：

$ python multiprocessing_exitcode.py

Starting process for exit_error

Starting process for exit_ok

Starting process for return_value

Starting process for raises

Starting process for terminated

Process raises:

Traceback (most recent call last):

  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python

2.7/multiprocessing/process.py", line 258, in _bootstrap

    self.run()

  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python

2.7/multiprocessing/process.py", line 114, in run

    self._target(*self._args, **self._kwargs)

  File "multiprocessing_exitcode.py", line 24, in raises

    raise RuntimeError('There was an error!')

RuntimeError: There was an error!

exit_error.exitcode = 1

exit_ok.exitcode = 0

return_value.exitcode = 0

raises.exitcode = 1

terminated.exitcode = -15

巴特西

python使用multiprocessing进行多进程编程（1）

1.创建一个进程

2.传递参数给子进程

3.后台进程

4.等待进程

5.关闭子进程

6.进程返回状态

最新文章

热门文章