在Python中使用protobuf2.6.1 string format utf-8 and unicode error
2024-09-06 17:01:00
版本信息:
protobuf: v2.6.1
python: 2.7
关于在Python中使用protobuf时 string格式字段的编码问题
在python中编码格式多采用utf-8格式。而protobuf
官网中这样说到:
如果不做处理,在message 中定义了一个string类型的字段后,出现错误如下:
ERROR: ValueError: '\xe5\x94\x90\xe6\x9e\x9c' has type bytes, but isn't in 7-bit ASCII encoding. Non-ASCII strings must be converted to unicode objects before being added.
解决办法有两种。如下:
1) 一劳永逸的方法-修改源码
a. 文件../google/protobuf/internal/decoder.py
def StringDecoder(field_number, is_repeated, is_packed, key, new_default):
"""Returns a decoder for a string field.""" local_DecodeVarint = _DecodeVarint
local_unicode = unicode def _ConvertToUnicode(byte_str):
try:
#return local_unicode(byte_str, 'utf-8') # 注释掉 不转码
return byte_str
except UnicodeDecodeError, e:
# add more information to the error message and re-raise it.
e.reason = '%s in field: %s' % (e, key.full_name)
raise
b. 文件../google/protobuf/internal/type_checkers.py
class UnicodeValueChecker(object): """Checker used for string fields. Always returns a unicode value, even if the input is of type str.
""" def CheckValue(self, proposed_value):
if not isinstance(proposed_value, (bytes, unicode)):
message = ('%.1024r has type %s, but expected one of: %s' %
(proposed_value, type(proposed_value), (bytes, unicode)))
raise TypeError(message) # If the value is of type 'bytes' make sure that it is in 7-bit ASCII
# encoding.
# if isinstance(proposed_value, bytes):
# try:
# proposed_value = proposed_value.decode('ascii')
# except UnicodeDecodeError:
# raise ValueError('%.1024r has type bytes, but isn\'t in 7-bit ASCII '
# 'encoding. Non-ASCII strings must be converted to '
# 'unicode objects before being added.' %
# (proposed_value))
return proposed_value
2) 很烦的方法-手动转码
在message中赋值时 都带上 decode("utf-8")
最新文章
- SVG入门
- 机顶盒上gridview+ScrollView的使用。
- phpAdmin安装
- C#的变迁史 - C# 4.0 之多线程篇
- (译)你应该知道的jQuery小技巧
- oracle返回多个参数
- Android PopupWindow使用之地区、学校选择二级联动
- self.view 的不当操作造成死循环
- ZOJ 1095 Humble Numbers
- LOAD TABLE tbl_name FROM MASTER语法 SQL
- POJ_3045_Cow_Acrobats_(贪心)
- (转)PHP函数之error_reporting(E_ALL ^ E_NOTICE)详细说明
- Asp.Net--主题/皮肤文件
- nginx负载 发向代理配置文件参考
- 加载xib文件的两种方式
- ORA-00245问题总结
- java 虚拟机的类加载机制
- 软工实践项目需求分析(团队)修改版get√-黄紫仪
- acm 2043
- 2018.12.17 bzoj3667: Rabin-Miller算法(Pollard-rho)
热门文章
- CF191C Fools and Roads - 树剖解法
- Applied Spatiotemporal Data Mining应用时空数据挖掘
- scrapy项目2:爬取智联招聘的金融类高端岗位(spider类)
- [SPOJ20174]DIVCNT3 - Counting Divisors (cube):Min_25筛
- linux服务器在线测速
- WebView:是应用程序打开web网页的UI控件前台
- a标签点击,页面自动刷新
- c# SQLite 判断表、字段是否存在的方法,新增、删除、重命名列
- FLUME安装&;环境(一):netcat类型配置
- input函数以及while处理列表和字典