============================================================================

原创作品，同意转载。

转载时请务必以超链接形式标明原始出处、以及本声明。

请注明转自：http://yunjianfei.iteye.com/blog/

============================================================================

近期在用tornado写一个基于Rest的WebService服务端，仅仅提供后端服务，其它webserver应用通过URL，Rest的方式来訪问。

我们在开发web应用的时候。难免会想到ORM的一些框架。比方java ee中经常使用的hibernate, ibatis以及python中的SQLAlchemy之类。

使用ORM会在一定程度上加快我们的开发效率。

一个简易ORM框架主要实现例如以下几个功能就足够了：

1.插入： 类对象映射为数据库记录

2.查询：数据库记录映射为类对象

3.改动、删除：能够通过自己写sql语句来搞定。

python中有类，同一时候也有dict字典类型。假设将字典再包装为类。则显得过渡包装了，反倒非常不灵活，因此，提炼一下，python的ORM框架仅仅须要实现例如以下几点就足够：

1.插入： python的dict映射为数据库记录

2.查询：数据库记录映射为python的dict以及list等

3.改动、删除：能够通过自己写sql语句来搞定。

经过一些測试，技术选型，终于确定了使用tornadb。很轻量级，查询数据库返回的对象直接映射为python的数据类型dict或者list之类。能够用类似java中“对象.属性”的方式来訪问数据。

这简直是太爽了~首先，看一个小样例。

import types

import time

class Row(dict):

    """A dict that allows for object-like property access syntax."""

    def __getattr__(self, name):

        try:

            return self[name]

        except KeyError:

            raise AttributeError(name)

dic = Row()

dic.name = 'hello'

dic.num = '12334'

print type(dic)

print "dic.name: " + dic.name

print "dic.num: " + dic.num

输出结果为：

<class '__main__.Row'>

dic.name: hello

dic.num: 12334

通过这个样例。我们能够看到，python里面的dict类型。是能够变成类似java中“对象.属性”的方式来訪问的。

torndb就是通过这种方式，查询返回的数据能够通过“.列名”来直接訪问。

查询的时候直接返回dict或者list类型，那插入呢？假设能够像java一样，传入一个对象，通过ORM框架直接反射为sql操作。这样多方便啊~

还是dict，假设我们插入的时候。直接将插入的数据格式保存为dict，通过这个dict生成insert语句就能够了，经过查阅各种资料。我提炼出来了例如以下方法：（使用的时候直接将该方法放入torndb.py中就可以）

    def insert_by_dict(self, tablename, rowdict, replace=False):

        cursor = self._cursor()

        cursor.execute("describe %s" % tablename)

        allowed_keys = set(row[0] for row in cursor.fetchall())

        keys = allowed_keys.intersection(rowdict)

        if len(rowdict) > len(keys):

            unknown_keys = set(rowdict) - allowed_keys

            logging.error("skipping keys: %s", ", ".join(unknown_keys))

        columns = ", ".join(keys)

        values_template = ", ".join(["%s"] * len(keys))

        if replace:

            sql = "REPLACE INTO %s (%s) VALUES (%s)" % (

                tablename, columns, values_template)

        else:

            sql = "INSERT INTO %s (%s) VALUES (%s)" % (

                tablename, columns, values_template)

        values = tuple(rowdict[key] for key in keys)

        try:

            cursor.execute(sql, values)

            #self._execute(cursor, sql, values, None)

            return cursor.lastrowid

        finally:

            cursor.close()

这样，插入的时候我们就再也不用写繁琐的sql语句了，仅仅须要将我们要插入的对象使用dict封装。比方：

有个host表，里面有hostname，ip两个字段，则我们能够用例如以下几行代码，就能够插入到数据库：

    host = {}

    host['hostname'] = 'test1'

    host['ip'] = '10.22.10.90'

    ret = db.insert_by_dict("Host", host)

是不是非常方便呢？：）以下是我改动过后，完整的torndb源代码。欢迎大家多多下载使用。

#!/usr/bin/env python

#

# Copyright 2009 Facebook

#

# Licensed under the Apache License, Version 2.0 (the "License"); you may

# not use this file except in compliance with the License. You may obtain

# a copy of the License at

#

#     http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT

# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the

# License for the specific language governing permissions and limitations

# under the License.

"""A lightweight wrapper around MySQLdb.

Originally part of the Tornado framework.  The tornado.database module

is slated for removal in Tornado 3.0, and it is now available separately

as torndb.

"""

from __future__ import absolute_import, division, with_statement

import copy

import itertools

import logging

import os

import time

try:

    import MySQLdb.constants

    import MySQLdb.converters

    import MySQLdb.cursors

except ImportError:

    # If MySQLdb isn't available this module won't actually be useable,

    # but we want it to at least be importable on readthedocs.org,

    # which has limitations on third-party modules.

    if 'READTHEDOCS' in os.environ:

        MySQLdb = None

    else:

        raise

version = "0.2"

version_info = (0, 2, 0, 0)

class Connection(object):

    """A lightweight wrapper around MySQLdb DB-API connections.

    The main value we provide is wrapping rows in a dict/object so that

    columns can be accessed by name. Typical usage::

        db = torndb.Connection("localhost", "mydatabase")

        for article in db.query("SELECT * FROM articles"):

            print article.title

    Cursors are hidden by the implementation, but other than that, the methods

    are very similar to the DB-API.

    We explicitly set the timezone to UTC and assume the character encoding to

    UTF-8 (can be changed) on all connections to avoid time zone and encoding errors.

    The sql_mode parameter is set by default to "traditional", which "gives an error instead of a warning"

    (http://dev.mysql.com/doc/refman/5.0/en/server-sql-mode.html). However, it can be set to

    any other mode including blank (None) thereby explicitly clearing the SQL mode.

    """

    def __init__(self, host, database, user=None, password=None,

                 max_idle_time=7 * 3600, connect_timeout=0,

                 time_zone="+0:00", charset = "utf8", sql_mode="TRADITIONAL"):

        self.host = host

        self.database = database

        self.max_idle_time = float(max_idle_time)

        args = dict(conv=CONVERSIONS, use_unicode=True, charset=charset,

                    db=database, init_command=('SET time_zone = "%s"' % time_zone),

                    connect_timeout=connect_timeout, sql_mode=sql_mode)

        if user is not None:

            args["user"] = user

        if password is not None:

            args["passwd"] = password

        # We accept a path to a MySQL socket file or a host(:port) string

        if "/" in host:

            args["unix_socket"] = host

        else:

            self.socket = None

            pair = host.split(":")

            if len(pair) == 2:

                args["host"] = pair[0]

                args["port"] = int(pair[1])

            else:

                args["host"] = host

                args["port"] = 3306

        self._db = None

        self._db_args = args

        self._last_use_time = time.time()

        try:

            self.reconnect()

        except Exception:

            logging.error("Cannot connect to MySQL on %s", self.host,

                          exc_info=True)

    def __del__(self):

        self.close()

    def close(self):

        """Closes this database connection."""

        if getattr(self, "_db", None) is not None:

            self._db.close()

            self._db = None

    def reconnect(self):

        """Closes the existing database connection and re-opens it."""

        self.close()

        self._db = MySQLdb.connect(**self._db_args)

        self._db.autocommit(True)

    def initClientEncode(self):

        """mysql client encoding=utf8"""

        curs = self._cursor()

        curs.execute("SET NAMES utf8")

        return curs

    def iter(self, query, *parameters, **kwparameters):

        """Returns an iterator for the given query and parameters."""

        self._ensure_connected()

        cursor = MySQLdb.cursors.SSCursor(self._db)

        try:

            self._execute(cursor, query, parameters, kwparameters)

            column_names = [d[0] for d in cursor.description]

            for row in cursor:

                yield Row(zip(column_names, row))

        finally:

            cursor.close()

    def query(self, query, *parameters, **kwparameters):

        """Returns a row list for the given query and parameters."""

        cursor = self._cursor()

        try:

            self._execute(cursor, query, parameters, kwparameters)

            column_names = [d[0] for d in cursor.description]

            return [Row(itertools.izip(column_names, row)) for row in cursor]

        finally:

            cursor.close()

    def get(self, query, *parameters, **kwparameters):

        """Returns the (singular) row returned by the given query.

        If the query has no results, returns None.  If it has

        more than one result, raises an exception.

        """

        rows = self.query(query, *parameters, **kwparameters)

        if not rows:

            return None

        elif len(rows) > 1:

            raise Exception("Multiple rows returned for Database.get() query")

        else:

            return rows[0]

    # rowcount is a more reasonable default return value than lastrowid,

    # but for historical compatibility execute() must return lastrowid.

    def execute(self, query, *parameters, **kwparameters):

        """Executes the given query, returning the lastrowid from the query."""

        return self.execute_lastrowid(query, *parameters, **kwparameters)

    def execute_lastrowid(self, query, *parameters, **kwparameters):

        """Executes the given query, returning the lastrowid from the query."""

        cursor = self._cursor()

        try:

            self._execute(cursor, query, parameters, kwparameters)

            return cursor.lastrowid

        finally:

            cursor.close()

    def execute_rowcount(self, query, *parameters, **kwparameters):

        """Executes the given query, returning the rowcount from the query."""

        cursor = self._cursor()

        try:

            self._execute(cursor, query, parameters, kwparameters)

            return cursor.rowcount

        finally:

            cursor.close()

    def executemany(self, query, parameters):

        """Executes the given query against all the given param sequences.

        We return the lastrowid from the query.

        """

        return self.executemany_lastrowid(query, parameters)

    def executemany_lastrowid(self, query, parameters):

        """Executes the given query against all the given param sequences.

        We return the lastrowid from the query.

        """

        cursor = self._cursor()

        try:

            cursor.executemany(query, parameters)

            return cursor.lastrowid

        finally:

            cursor.close()

    def get_fields_str(self, tablename):

        cursor = self._cursor()

        cursor.execute("describe %s" % tablename)

        fields=[]

        for row in cursor.fetchall():

            fields.append(row[0])

        str = ", ".join(fields)

        cursor.close()

        return str

    def get_fields_prefix_str(self, tablename, prefix):

        cursor = self._cursor()

        cursor.execute("describe %s" % tablename)

        fields=[]

        for row in cursor.fetchall():

            fields.append(prefix+row[0])

        str = ", ".join(fields)

        cursor.close()

        return str

    def get_select_sql(self, tablename):

        str = self.get_fields_str(tablename)

        sql = "SELECT " + str + " FROM " + tablename + " "

        return sql

    def insert_by_dict(self, tablename, rowdict, replace=False):

        cursor = self._cursor()

        cursor.execute("describe %s" % tablename)

        allowed_keys = set(row[0] for row in cursor.fetchall())

        keys = allowed_keys.intersection(rowdict)

        if len(rowdict) > len(keys):

            unknown_keys = set(rowdict) - allowed_keys

            logging.error("skipping keys: %s", ", ".join(unknown_keys))

        columns = ", ".join(keys)

        values_template = ", ".join(["%s"] * len(keys))

        if replace:

            sql = "REPLACE INTO %s (%s) VALUES (%s)" % (

                tablename, columns, values_template)

        else:

            sql = "INSERT INTO %s (%s) VALUES (%s)" % (

                tablename, columns, values_template)

        values = tuple(rowdict[key] for key in keys)

        try:

            cursor.execute(sql, values)

            #self._execute(cursor, sql, values, None)

            return cursor.lastrowid

        finally:

            cursor.close()

    def transaction(self, query, *parameters, **kwparameters):

        self._db.begin()

        cursor = self._cursor()

        status = True

        try:

            for sql in query:

                cursor.execute(sql, kwparameters or parameters)

            self._db.commit()

        except OperationalError, e:

            self._db.rollback()

            status = False

            raise Exception(e.args[1], e.args[0])

        finally:

            cursor.close()

        return status

    def executemany_rowcount(self, query, parameters):

        """Executes the given query against all the given param sequences.

        We return the rowcount from the query.

        """

        cursor = self._cursor()

        try:

            cursor.executemany(query, parameters)

            return cursor.rowcount

        finally:

            cursor.close()

    update = execute_rowcount

    updatemany = executemany_rowcount

    insert = execute_lastrowid

    insertmany = executemany_lastrowid

    def _ensure_connected(self):

        # Mysql by default closes client connections that are idle for

        # 8 hours, but the client library does not report this fact until

        # you try to perform a query and it fails.  Protect against this

        # case by preemptively closing and reopening the connection

        # if it has been idle for too long (7 hours by default).

        if (self._db is None or

            (time.time() - self._last_use_time > self.max_idle_time)):

            self.reconnect()

        self._last_use_time = time.time()

    def _cursor(self):

        self._ensure_connected()

        return self._db.cursor()

    def _execute(self, cursor, query, parameters, kwparameters):

        try:

            return cursor.execute(query, kwparameters or parameters)

        except OperationalError:

            logging.error("Error connecting to MySQL on %s", self.host)

            self.close()

            raise

class Row(dict):

    """A dict that allows for object-like property access syntax."""

    def __getattr__(self, name):

        try:

            return self[name]

        except KeyError:

            raise AttributeError(name)

if MySQLdb is not None:

    # Fix the access conversions to properly recognize unicode/binary

    FIELD_TYPE = MySQLdb.constants.FIELD_TYPE

    FLAG = MySQLdb.constants.FLAG

    CONVERSIONS = copy.copy(MySQLdb.converters.conversions)

    field_types = [FIELD_TYPE.BLOB, FIELD_TYPE.STRING, FIELD_TYPE.VAR_STRING]

    if 'VARCHAR' in vars(FIELD_TYPE):

        field_types.append(FIELD_TYPE.VARCHAR)

    for field_type in field_types:

        CONVERSIONS[field_type] = [(FLAG.BINARY, str)] + CONVERSIONS[field_type]

    # Alias some common MySQL exceptions

    IntegrityError = MySQLdb.IntegrityError

    OperationalError = MySQLdb.OperationalError

外带一个小样例,完整版请參照我在github上公布的一个webservice框架：https://github.com/yunfeiflying/tornado-rest-web-service-framwork/

#!/usr/bin/env python2.7

#

# -*- coding:utf-8 -*-

#

#   Author  :   YunJianFei

#   E-mail  :   yunjianfei@126.com

#   Date    :   2014/02/25

#   Desc    :   Test db

#

""" Data Access Object

    This file impelements DBI for the table 'Host'

The Host table's create sql is : 

CREATE TABLE IF NOT EXISTS `test`.`Host` (

  `host_id` INT NOT NULL AUTO_INCREMENT,

  `host_type` INT NULL,

  `hostname` VARCHAR(45) NULL,

  `ip` VARCHAR(45) NULL,

  `create_time` VARCHAR(45) NULL,

  `cpu_count` INT NULL,

  `cpu_pcount` INT NULL,

  `memory` INT NULL,

  `os` VARCHAR(200) NULL,

  `comment` VARCHAR(200) NULL,

  PRIMARY KEY (`host_id`))

ENGINE = InnoDB;

"""

from util.dbconst import TableName, TableFields, TableSelectSql

import logging

class HostDao:

    def __init__(self, db):

        mysql_host = "192.168.10.11:3306"

        db_name = "test"

        db_user = "root"

        db_pass = ""

        self.db = torndb.Connection(

            host=mysql_host, database=db_name,

            user=db_user, password=db_pass

        )

    def insert_by_dict(self, host, replace=False):

        try:

            id = self.db.insert_by_dict("Host", host, replace)

            return id

        except Exception, ex:

            logging.error("Insert host failed! Exception: %s   Host: %s", str(ex), str(host))

            return None

    def if_exist(self, hostname, ip):

        ret = self.get_by_hostname(hostname)

        if ret != None:

            return True

        ret = self.get_by_ip(ip)

        if ret != None:

            return True

        return False

    def get_by_ip(self, ip):

        sql = TableSelectSql.HOST + " where ip='" + str(ip)+"'"

        return self.db.get(sql)

    def get_all(self):

        sql = TableSelectSql.HOST

        return self.db.query(sql)

    def get_by_hostname(self, hostname):

        sql = TableSelectSql.HOST + " where hostname='" + str(hostname)+"'"

        return self.db.get(sql)

    def get_by_id(self, host_id):

        sql = TableSelectSql.HOST + " where host_id=%s" % str(host_id)

        return self.db.get(sql)

    def get_id_by_hostname(self, hostname):

        sql = TableSelectSql.HOST + " where hostname='" + str(hostname)+"'"

        ret = self.db.get(sql)

        if ret != None:

            return ret.host_id

        return None

    def update_worker_num_by_hostname(self, hostname, worker_num):

        try:

            sql = "UPDATE Host SET worker_num=%s WHERE hostname='%s'" % (worker_num, str(hostname))

            ret = self.db.execute(sql)

            return ret

        except Exception, ex:

            logging.error("Update Host failed! Exception: %s   hostname: %s , worker_num: %s", str(ex), str(hostname), worker_num)

            return None

    def update_worker_num_by_id(self, host_id, worker_num):

        try:

            sql = "UPDATE Host SET worker_num=%s WHERE host_id=%s" % (worker_num, host_id)

            ret = self.db.execute(sql)

            return ret

        except Exception, ex:

            logging.error("Update Host failed! Exception: %s   host_id: %s , worker_num: %s", str(ex), host_id, worker_num)

            return None

    def del_by_hostname(self, hostname):

        try:

            sql = "DELETE FROM Host WHERE hostname='" + str(hostname) + "'"

            ret = self.db.execute(sql)

            return ret

        except Exception, ex:

            logging.error("Delete host failed! Exception: %s   hostname: %s", str(ex), str(hostname))

            return None

    def del_by_id(self, host_id):

        try:

            sql = "DELETE FROM Host WHERE host_id=" + str(host_id)

            ret = self.db.execute(sql)

            return ret

        except Exception, ex:

            logging.error("Delete host failed! Exception: %s   host_id: %s", str(ex), host_id)

            return None

最新文章

  1. mybatis实战教程(mybatis in action)之三：实现数据的增删改查
  2. lombok
  3. RabbitMQ官方中文入门教程(PHP版) 第二部分:工作队列（Work queues）
  4. Visual Studio示例代码浏览器
  5. 从P6 EPPM 8 R3 到P6 EPPM 16 R1 有哪些改变?
  6. CF Sea and Islands
  7. （原）使用vectot的.end()报错：iterators incompatible
  8. php基础知识笔记
  9. Advanced Data Structures
  10. Openjudge-计算概论（A）-大象喝水
  11. Android Doze模式启用和恢复
  12. Python之set
  13. WIN7下安装SVNserver端及client搭建协作环境
  14. helm 持久化部署ingres
  15. vue-cli按需加载，懒加载组件
  16. Python实现，从txt取出号码，在数据库里查询此号码是否存在
  17. 设计模式-行为型模式，python备忘录模式
  18. DevOps 解决方案 - 腾讯云
  19. Nginx初入
  20. ctex2.9.2输出中文

热门文章

  1. POJ 2418 简单trie树
  2. 6月10遇到的bug
  3. for 循环练习题（2）
  4. 【PostgreSQL-9.6.3】表操作语句
  5. AI：IPPR的数学表示-CNN结构/参数分析
  6. 相机标定：PNP基于单应面解决多点透视问题
  7. jupyter notebook主目录修改
  8. Linux基础：uniq命令总结
  9. html表单练习
  10. Java常量池详细说明