机器学习模型常用Docker部署，而如何对Docker部署的模型进行管理呢？工业界的解决方案是使用Kubernetes来管理、编排容器。Kubernetes的理论知识不是本文讨论的重点，这里不再赘述，有关Kubernetes的优点读者可自行Google。笔者整理的Kubernetes入门系列重点是如何实操，前三节介绍了Kubernets的安装、Dashboard的安装，以及如何在Kubernetes中部署一个无状态的应用，本节将讨论如何在Kubernetes中部署一个可对外服务的Tensorflow机器学习模型，作为Kubernetes入门系列的结尾。

希望Kubernetes入门系列能对K8S初学者提供一些参考，对文中描述有不同观点，或者对工业级部署与应用机器学习算法模型有什么建议，欢迎大家在评论区讨论与交流~~~

1. Docker中运行TensorFolw Serving

运行half_plus_two模型 ^[1]

# Download the TensorFlow Serving Docker image and repo

docker pull tensorflow/serving

mkdir /data0/modules

cd /data0/modules

git clone https://github.com/tensorflow/serving

# Location of demo models

TESTDATA="/data0/modules/serving/tensorflow_serving/servables/tensorflow/testdata/"

# Start TensorFlow Serving container and open the REST API port

docker run -dit --rm -p 8501:8501 \

-v /data0/modules/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_two_cpu:/models/half_plus_two \

-e MODEL_NAME=half_plus_two  tensorflow/serving 

# Query the model using the predict API

curl -d '{"instances": [1.0, 2.0, 5.0]}' \

    -X POST http://localhost:8501/v1/models/half_plus_two:predict

# Returns => { "predictions": [2.5, 3.0, 4.5] }

2. 构建TensorFolw模型的Docker镜像

后台运行serving容器

docker run -d --rm --name serving_base tensorflow/serving

docker cp /data0/modules/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_two_cpu serving_base:/models/half_plus_two

生成关于模型的镜像

docker commit --change "ENV MODEL_NAME half_plus_two" serving_base ljh/half_plus_two

停止serving容器

docker kill serving_base

docker rm serving_base

启动服务

docker run -dit --rm -p 8501:8501 \

-e MODEL_NAME=half_plus_two  ljh/half_plus_two

查询模型

curl -d '{"instances": [1.0, 2.0, 5.0]}'    -X POST http://localhost:8501/v1/models/half_plus_two:predict

# Returns => { "predictions": [2.5, 3.0, 4.5] }

3. Kubernetes部署TensorFolw模型

创建关于模型的Deployment

yaml文件

cat deployment.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  name: halfplustwo-deployment

spec:

  selector:

    matchLabels:

      app: halfplustwo

  replicas: 1

  template:

    metadata:

      labels:

        app: halfplustwo

    spec:

      containers:

        - name: halfplustwo

          image: ljh/half_plus_two:latest

          imagePullPolicy: IfNotPresent

          ports:

            - containerPort: 8501

              name: restapi

            - containerPort: 8500

              name: grpc

创建一个Deployment:

kubectl apply -f deployment.yaml

展示Deployment相关信息:

kubectl get deployment -o wide

kubectl describe deployment halfplustwo-deployment

列出deployment创建的pods:

kubectl get pods -l app=halfplustwo

展示某一个pod信息

kubectl describe pod <pod-name>

使用service暴露你的应用

yaml文件

cat service.yaml

apiVersion: v1

kind: Service

metadata:

  labels:

    run: halfplustwo-service

  name: halfplustwo-service

spec:

  ports:

    - port: 8501

      targetPort: 8501

      name: restapi

    - port: 8500

      targetPort: 8500

      name: grpc

  selector:

    app: halfplustwo

  type: LoadBalancer

启动service

kubectl create -f service.yaml

or

kubectl apply -f service.yaml

查看service

kubectl get service

#output:

NAME                  TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                         AGE

halfplustwo-service   LoadBalancer   10.96.181.116   <pending>     8501:30771/TCP,8500:31542/TCP   4s

kubernetes            ClusterIP      10.96.0.1       <none>        443/TCP                         8d

nginx                 NodePort       10.96.153.10    <none>        80:30088/TCP                    29h

测试

curl -d '{"instances": [1.0, 2.0, 5.0]}'    -X POST http://localhost:8501/v1/models/half_plus_two:predict

{"predictions": [2.5, 3.0, 4.5]}

删除deployment和service

kubectl delete -f deployment.yaml

kubectl delete -f service.yaml

4. 参考资料

[1] https://www.tensorflow.org/tfx/serving/docker    TensorFlow Serving 与 Docker

[2] https://www.tensorflow.org/tfx/serving/serving_kubernetes?hl=zh_cn   将TensorFlow Serving与 Kubernetes结合使用

[3] https://towardsdatascience.com/scaling-machine-learning-models-using-tensorflow-serving-kubernetes-ed00d448c917  Scaling Machine Learning models using Tensorflow Serving & Kubernetes

[4] http://www.tuwee.cn/2019/03/03/Kubernetes+Tenserflow-serving%E6%90%AD%E5%BB%BA%E5%8F%AF%E5%AF%B9%E5%A4%96%E6%9C%8D%E5%8A%A1%E7%9A%84%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E5%BA%94%E7%94%A8/ Kubernetes+Tenserflow-serving搭建可对外服务的机器学习应用

巴特西

Kubernetes入门(四)——如何在Kubernetes中部署一个可对外服务的Tensorflow机器学习模型