1，Tomcat启动报错例如以下：

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'memcachedClient' defined in 

file [/usr/local/apache-tomcat-6.0.37_6500/webapps/trade_service/WEB-INF/classes/META-INF/spring/springContext_memcached.xml]: 

Instantiation of bean failed; nested exception is org.springframework.beans.factory.BeanDefinitionStoreException: Factory method 

[public net.rubyeye.xmemcached.MemcachedClient net.rubyeye.xmemcached.XMemcachedClientBuilder.build() throws java.io.IOException] 

threw exception; nested exception is java.lang.OutOfMemoryError

        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod

(ConstructorResolver.java:581)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod

(AbstractAutowireCapableBeanFactory.java:1015)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance

(AbstractAutowireCapableBeanFactory.java:911)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean

(AbstractAutowireCapableBeanFactory.java:485)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean

(AbstractAutowireCapableBeanFactory.java:456)

        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)

        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton

(DefaultSingletonBeanRegistry.java:225)

        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)

        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)

        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference

(BeanDefinitionValueResolver.java:322)

        ... 44 more

Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Factory method [public 

net.rubyeye.xmemcached.MemcachedClient net.rubyeye.xmemcached.XMemcachedClientBuilder.build() throws java.io.IOException] threw 

exception; nested exception is java.lang.OutOfMemoryError

        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate

(SimpleInstantiationStrategy.java:169)

        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod

(ConstructorResolver.java:570)

        ... 53 more

Caused by: java.lang.OutOfMemoryError

,

2，看到创建memcache clent报错，立即去memcacheserver查看一下，memcache正常执行，内存也足够剩余。

[root@localhost ~]# tail -f /tmp/memcache_error.log

<287    0x00 0x00 0x00 0x00

<287    0x00 0x00 0x00 0x00

<287    0x00 0x00 0x00 0x00

>287 Writing bin response:

>287   0x81 0x0b 0x00 0x00

>287   0x00 0x00 0x00 0x00

>287   0x00 0x00 0x00 0x06

>287   0x00 0x00 0x00 0x00

>287   0x00 0x00 0x00 0x00

>287   0x00 0x00 0x00 0x00

...

[root@localhost ~]# free -m

             total       used       free     shared    buffers     cached

Mem:          5938       3368       2570          0        195        296

-/+ buffers/cache:       2876       3061

Swap:         7935          0       7935

[root@localhost ~]#

问题在哪里？再看看日志里面有误error或者warning信息：

[root@localhost ~]# cat  /tmp/memcache_error.log  |grep warning

[root@localhost ~]# cat  /tmp/memcache_error.log  |grep error

[root@localhost ~]#

一点错误信息也没有，推断问题不是出在memcache这一块。

3，去check tomcatserver

[root@localhost logs]# free -m

             total       used       free     shared    buffers     cached

Mem:         18022      16943       1078          0        104        675

-/+ buffers/cache:      16162       1859

Swap:        20031       8012      12019

[root@localhost logs]# 

仅仅剩余1个G的内存，去改动一下tomcat的内存配置：

[root@localhost bin]# vim catalina.sh

#JAVA_OPTS="-Xms2048m -Xmx2048m -Xss1024K -XX:PermSize=256m -XX:MaxPermSize=768m"

# add by wangqi on 20140614

 JAVA_OPTS="-Xms512m -Xmx512m -Xss128K -XX:PermSize=64m -XX:MaxPermSize=256m"

再重新启动tomcat，看到还是报一样的错误。

4，回到应用本身去查问题

跟开发者一起排查，看到是trade_service启动了过多的多余的事务，把tomcat的内存耗光了，所以关闭掉多余的事务，再启动tomcat，就不会再报错。

感慨：就跟大部分web应用卡，页面登录不上去，对应过慢非常多是出在效率低下的sql上面，tomcat启动不起来，内存溢出大部分也是在java代码上面，大家须要写代码或者sql须要慎重，且写且慎重。