Explorar o código

优化dockerfile和定时任务

hexiaoshi hai 1 ano
pai
achega
7012dce760
Modificáronse 3 ficheiros con 151 adicións e 2 borrados
  1. 6 1
      Dockerfile
  2. 1 1
      crontab
  3. 144 0
      烟草模型部署文档.md

+ 6 - 1
Dockerfile

@@ -8,7 +8,12 @@ COPY . /app/
 
 
 RUN mv /app/crontab /etc/cron.d/crontab && chmod 0644 /etc/cron.d/crontab \ 
-        && /usr/bin/crontab /etc/cron.d/crontab && pip install -r requirements.txt
+        && /usr/bin/crontab /etc/cron.d/crontab \ 
+        && pip install --upgrade pip setuptools -i https://mirrors.aliyun.com/pypi/simple  \ 
+        && pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple
+
+RUN find . | grep -E "(__pycache__|Dockerfile|\.pyc|\.pyo$)" | xargs rm -rf && python3 -m compileall -b . \ 
+        &&  find . -name "*.py" |xargs rm -rf
 
 
 CMD ["cron", "-f"]

+ 1 - 1
crontab

@@ -1,3 +1,3 @@
 # START CRON JOB
-1 2 * * * python /export/python/BrandCultivation/app.py 
+1 2 * * * python /app/app.pyc
 # END CRON JOB

+ 144 - 0
烟草模型部署文档.md

@@ -0,0 +1,144 @@
+# 烟草推荐模型部署文档
+
+## 1、配置文件说明:
+
+- ### database_config.yaml  这个是数据配置文件
+    
+
+```
+mysql:
+  host: 'rm-t4n6rz18y4t5x47y70o.mysql.singapore.rds.aliyuncs.com'
+  port: 3036
+  db: 'brand_cultivation'
+  user: 'xxxxx'
+  passwd: 'xxxxx'
+
+redis:
+  host: 'r-t4nb4n9i8je7u6ogk1pd.redis.singapore.rds.aliyuncs.com'
+  port: 5000
+  db: 10
+  passwd: 'xxxxx'
+```
+
+- ### crontab 定时任务配置文件
+    
+
+```
+# START CRON JOB
+1 2 * * * python /app/app.pyc
+# END CRON JOB
+```
+
+ 
+
+## 2、模型启动配置说明:
+
+### app.py
+
+```
+    parser.add_argument("--run_all", action='store_true')
+    parser.add_argument("--run_hot", action='store_true')
+    parser.add_argument("--run_itemcf", action='store_true')
+    parser.add_argument("--run_itemcf_inference", action='store_true'
+```
+
+### 总共有4种启动模式分别是:
+
+1\. 启动热度召回和协同过滤  
+        2. 启动热度召回  
+        3. 启动协同过滤  
+        4. 启动系统过滤推理
+
+## 3、模型docker运行配置说明:
+
+### docker镜像是:registry.cn-hangzhou.aliyuncs.com/hexiaoshi/brandcultivation:0.0.1
+
+```yaml
+docker run --name BrandCultivation -d -v /export/brandcultivation/crontab:/etc/cron.d/crontab -v /export/brandcultivation/database_config.yaml:/app/config/database_config.yaml  registry.cn-hangzhou.aliyuncs.com/hexiaoshi/brandcultivation:0.0.1
+```
+
+## 4、模型kubernetes运行配置说明
+
+yaml文件如下:
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: brandcultivation
+  namespace: default
+  labels:
+    app: brandcultivation
+spec:
+  selector:
+    matchLabels:
+      app: brandcultivation
+  replicas: 1
+  strategy:
+    rollingUpdate:
+      maxSurge: 25%
+      maxUnavailable: 25%
+    type: RollingUpdate
+  template:
+    metadata:
+      labels:
+        app: brandcultivation
+    spec:
+      containers:
+      - name: brandcultivation
+        image: registry.cn-hangzhou.aliyuncs.com/hexiaoshi/brandcultivation:0.0.1
+        imagePullPolicy: IfNotPresent
+        resources:
+          requests:
+            cpu: 4000m
+            memory: 4096Mi
+            ephemeral-storage: 20Gi             
+          limits:
+            cpu: 4000m
+            memory: 4096Mi
+            ephemeral-storage: 20Gi            
+        ports:
+        - containerPort:  80
+          name: brandcultivation
+        volumeMounts:
+        - name: localtime
+          mountPath: /etc/localtime
+        - name: config
+          mountPath: /app/config/database_config.yaml
+          subPath: database_config.yaml
+        - name: config
+          mountPath: /etc/cron.d/crontab
+          subPath: crontab          
+        - name: localtime
+          hostPath:
+            path: /usr/share/zoneinfo/Asia/Shanghai
+        - name: config
+          configMap:
+            name: brandcultivation
+      restartPolicy: Always
+---
+kind: ConfigMap
+apiVersion: v1
+metadata:
+  name: brandcultivation
+  namespace: default
+data:
+  database_config.yaml: |
+    mysql:
+      host: 'rm-t4n6rz18y4t5x47y70o.mysql.singapore.rds.aliyuncs.com'
+      port: 3036
+      db: 'brand_cultivation'
+      user: 'BrandCultivation'
+      passwd: '8BfWBc18NBXl#CMd'
+
+    redis:
+      host: 'r-t4nb4n9i8je7u6ogk1pd.redis.singapore.rds.aliyuncs.com'
+      port: 5000
+      db: 10
+      passwd: 'gHmNkVBd88sZybj'
+  crontab: |
+    # START CRON JOB
+    1 2 * * * python /app/app.pyc
+    # END CRON JOB
+
+```