import keras

from keras.datasets import mnist

from keras.models import Sequential

from keras.layers import Dense, Activation, Flatten

from keras.layers import Conv2D, MaxPooling2D, UpSampling2D

import matplotlib.pyplot as plt

from keras import backend as K

import numpy as np

# (x_train, y_train), (x_test, y_test) = mnist.load_data()

f = np.load("mnist.npz")

x_train, y_train = f['x_train'], f['y_train']

x_test, y_test = f['x_test'], f['y_test']

f.close()

x_train = x_train.reshape(x_train.shape[0], 28, 28, 1) #transform 2D 28x28 matrix to 3D (28x28x1) matrix

x_test = x_test.reshape(x_test.shape[0], 28, 28, 1)

x_train = x_train.astype('float32')

x_test = x_test.astype('float32')

x_train /= 255 #inputs have to be between [0, 1]

x_test /= 255

model = Sequential()

#1st convolution layer

model.add(Conv2D(16, (3, 3) #16 is number of filters and (3, 3) is the size of the filter.

    , padding='same', input_shape=(28,28,1)))

model.add(Activation('relu'))

model.add(MaxPooling2D(pool_size=(2,2), padding='same'))

#2nd convolution layer

model.add(Conv2D(2,(3, 3), padding='same')) # apply 2 filters sized of (3x3)

model.add(Activation('relu'))

model.add(MaxPooling2D(pool_size=(2,2), padding='same'))

#-------------------------

#3rd convolution layer

model.add(Conv2D(2,(3, 3), padding='same')) # apply 2 filters sized of (3x3)

model.add(Activation('relu'))

model.add(UpSampling2D((2, 2)))

#4rd convolution layer

model.add(Conv2D(16,(3, 3), padding='same'))

model.add(Activation('relu'))

model.add(UpSampling2D((2, 2)))

#-------------------------

model.add(Conv2D(1,(3, 3), padding='same'))

model.add(Activation('sigmoid'))

print(model.summary())

model.compile(optimizer='adadelta', loss='binary_crossentropy')

model.fit(x_train, x_train

    , epochs=3

    , validation_data=(x_test, x_test)

)

restored_imgs = model.predict(x_test)

for i in range(5):

    plt.imshow(x_test[i].reshape(28, 28))

    plt.gray()

    plt.show()

    plt.imshow(restored_imgs[i].reshape(28, 28))

    plt.gray()

    plt.show()

    print("----------------------------")

layers = len(model.layers)

for i in range(layers):

    print(i, ". ", model.layers[i].output.get_shape())

"""

0 .  (?, 28, 28, 16)

1 .  (?, 28, 28, 16)

2 .  (?, 14, 14, 16)

3 .  (?, 14, 14, 2)

4 .  (?, 14, 14, 2)

5 .  (?, 7, 7, 2)

6 .  (?, 7, 7, 2)

7 .  (?, 7, 7, 2)

8 .  (?, 14, 14, 2)

9 .  (?, 14, 14, 16)

10 .  (?, 14, 14, 16)

11 .  (?, 28, 28, 16)

12 .  (?, 28, 28, 1)

13 .  (?, 28, 28, 1)

"""

#layer[7] is activation_3 (Activation), it is compressed representation

get_3rd_layer_output = K.function([model.layers[0].input], [model.layers[7].output])

compressed = get_3rd_layer_output([x_test])[0]

#layer[7] is size of (None, 7, 7, 2). this means 2 different 7x7 sized matrixes. We will flatten these matrixes.

compressed = compressed.reshape(10000,7*7*2)

# clustering

from tensorflow.contrib.factorization.python.ops import clustering_ops

import tensorflow as tf

unsupervised_model = tf.contrib.learn.KMeansClustering(

    10 #num of clusters

    , distance_metric = clustering_ops.SQUARED_EUCLIDEAN_DISTANCE

    , initial_clusters=tf.contrib.learn.KMeansClustering.RANDOM_INIT

)

def train_input_fn():

    data = tf.constant(compressed, tf.float32)

    return (data, None)

print(compressed[:3])

unsupervised_model.fit(input_fn=train_input_fn, steps=1000)

clusters = unsupervised_model.predict(input_fn=train_input_fn)

index = 0

for i in clusters:

    current_cluster = i['cluster_idx']

    features = x_test[index]

    if index < 200 and current_cluster == 5:

        plt.imshow(x_test[index].reshape(28, 28))

        plt.gray()

        plt.show()

    index = index + 1

"""

"""

我摘录的代码。

原文：https://sefiks.com/2018/03/21/autoencoder-neural-networks-for-unsupervised-learning/

Previously, we’ve applied conventional autoencoder to handwritten digit database (MNIST). That approach was pretty. We can apply same model to non-image problems such as fraud or anomaly detection. If the problem were pixel based one, you might remember that convolutional neural networks are more successful than conventional ones. However, we tested it for labeled supervised learning problems. The question is that can I adapt convolutional neural networks to unlabeled images for clustering? Absolutely yes! these customized form of CNN are convolutional autoencoder.

Remember autoencoder post. Network design is symettric about centroid and number of nodes reduce from left to centroid, they increase from centroid to right. Centroid layer would be compressed representation. We will apply same procedure for CNN, too. We will additionally consume convolution, activation and pooling layer for convolutional autoencoder.

Convolutional autoencoder

We can call left to centroid side as convolution whereas centroid to right side as deconvolution. Deconvolution side is also known as unsampling or transpose convolution. We’ve mentioned how pooling operation works. It is a basic reduction operation. How can we apply its reverse operation? That might be a little confusing. I’ve found a excellent animation for unsampling. Input matrix size of 2×2 (blue one) will be deconvolved to a matrix size of 4×4 (cyan one). To do this duty, we can add imaginary elements (e.g. 0 values) to the base matrix and it is transformed to 6×6 sized matrix.

Unsampling

We will work on handwritten digit database again. We’ll design the structure of convolutional autoencoder as illustrated above.

You can summarize the constructed network structure.

This command dumps the following output. Base input is size of 28×28 at the beginnig, 2 first two layers are responsible for reduction, following 2 layers are in charged of restoration. Final layer restores same size of input as seen.

_____________
Layer (type) Output Shape Param #
========
conv2d_1 (Conv2D) (None, 28, 28, 16) 160
_____________
activation_1 (Activation) (None, 28, 28, 16) 0
_____________
max_pooling2d_1 (MaxPooling2 (None, 14, 14, 16) 0
_____________
conv2d_2 (Conv2D) (None, 14, 14, 2) 290
_____________
activation_2 (Activation) (None, 14, 14, 2) 0
_____________
max_pooling2d_2 (MaxPooling2 (None, 7, 7, 2) 0
_____________
conv2d_3 (Conv2D) (None, 7, 7, 2) 38
_____________
activation_3 (Activation) (None, 7, 7, 2) 0
_____________
up_sampling2d_1 (UpSampling2 (None, 14, 14, 2) 0
_____________
conv2d_4 (Conv2D) (None, 14, 14, 16) 304
_____________
activation_4 (Activation) (None, 14, 14, 16) 0
_____________
up_sampling2d_2 (UpSampling2 (None, 28, 28, 16) 0
_____________
conv2d_5 (Conv2D) (None, 28, 28, 1) 145
_____________
activation_5 (Activation) (None, 28, 28, 1) 0
========

Here, we can start training.

Loss values for both training set and test set are satisfactory.

loss: 0.0968 – val_loss: 0.0926

Let’s visualize some restorations.

Testing

Restorations seems really satisfactory. Images on the left side are
original images whereas images on the right side are restored from
compressed representation.

Some restorations of convolutional autoencoder

Notice that 5th layer named max_pooling2d_2 states the compressed
representation and it is size of (None, 7, 7, 2). This work reveals that
we can restore 28×28 pixel image from 7x7x2 sized matrix with a little
loss. In other words, compressed representation takes a 8 times less
space to original image.

Compressed Representations

You might wonder how to extract compressed representations.

Now, we can apply clustering to compressed representation. I would like to apply k-means clustering.

Training is over. Now, we can check clusters for all test set.

For example, 6th cluster consists of 46 items. Distribution for this
cluster is like that: 22 items are 4, 14 items are 9, 7 items are 7, and
1 item is 5. It seems mostly 4 and 9 digits are put in this cluster.

So, we’ve integrated both convolutional neural networks and
autoencoder ideas for information reduction from image based data. That
would be pre-processing step for clustering. In this way, we can apply
k-means clustering with 98 features instead of 784 features. This could
fasten labeling process for unlabeled data. Of course, with autoencoding comes great speed. Source code of this post is already pushed into GitHub.

最新文章

1. ZOJ Problem Set - 1251 Box of Bricks
2. spark-shell和scala错误
3. JRebel: ERROR Could not define reloadable class 'com.sun.proxy.$Proxy118': java.lang.OutOfMemoryError: PermGen space
4. [转] JSON for java入门总结
5. java 集合框架(TreeSet操作,自动对数据进行排序,重写CompareTo方法)
6. SQL外连接
7. django 3.post接口开发
8. Node.js进程内存使用查看方法及返回对象的含义
9. mysql拼接sql的语法concat()用法
10. CH4701 天使玩偶
11. 第九章 词典 (d1)散列：排解冲突(1)
12. 【转载】Free Lunch is Over(免费午餐已经结束了)
13. vue.js 中slot 用处大（转载）
14. linux快速进入全屏命令行模式
15. instancetype 与id
16. WINDBG解决cpu占高的问题
17. MFC程序如何修改icon图标
18. 【usaco】1.1
19. c#实现wifi连接器
20. 使用QT开发GoogleMap瓦片显示和下载工具（1）——QT开发环境准备

热门文章

1. 彻底弄懂JS事件委托的概念和作用
2. VC_窗口exe_printf信息
3. P678-vect2.cpp
4. HDU 4812 D Tree
5. Python + Selenium WebDriver Api 知识回顾
6. 第 8 章 容器网络 - 054 - 准备 macvlan 环境
7. 手机计算器1+1=2---Appium自动化
8. Word Ladder(双向BFS)
9. Arduino 开关控制小灯持续亮之具体思路
10. 输出图片格式BARTENDER