北屿

北屿小智障

0%

机器学习练习

Posted on Edited on In

本来想着放在笔记后面，后来发现好像问题有点多…决定新开一篇文章来写

hw1

https://www.kaggle.com/c/ml2019spring-hw1/overview
作业说明

我一开始做了一个非常naive的model，反正分开处理，Python用的也不是很熟练，就当练代码了
一开始没用AdaGrad，发现Learning Rate真的很难设置
如果数据量小，那么Learning Rate应该大一些，如果数据量大，那就得小一些了，这个都是相对的
如果数据本身小，那么初始值就没那么重要，如果本身大的话，那初始值就需要自己手动设置一下了
举个第一个的栗子，如果一开始初始值都设成0,12960条数据，一开始得到的Loss就有51361098.87099994，Learning Rate我设成了1e-8，得到的结果还越来越大…
当我把数据量调小一点，比如20条这个样子，还是有点用的…Learning Rate = 1e-6，迭代50次大概能得到一个不错的结果…
实测1e-9,1e-12时有点用了…
我忽然有个大胆的想法…如果前面变化大点后面变化小点是不是很科学，可以飞快接近结果
p.s. Python的输出调试真的不好用…不如c++的#define debug(x) cout<<#x<<"="<<x<<endl，可能我还没get到该技能
原来只需要预测PM2.5就行了，那我直接写全得了
一开始Loss=4231600.00，最后313187.27500353276

现在我不是很清楚要什么时候结束…所以先直接计次循环了
先在这保存一下代码…之后改成AdaGrad
果然这个naive的程序得到了private:9.66107 public:8.18926的高分，我觉得以后应该把训练的模块和输出模块分开来写，不然必须训练一次才能输出233
第一次训练出来的w和b

1
2
w = [0.02560433358938389, 0.012801930896077247, 0.004710981594648449, 0.005885507375027003, 0.009942499746330985, 0.026670961533262413, -0.0010603121145215564, 0.25976948883940576, 0.6202885065990642]
b = 0.01483973260431497
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
import numpy as np

# read data

ff = open('train.csv', 'r', True, 'big5hkscs');
data_init = ff.readlines()
total_line = len(data_init)-1
# total_line = 19
total_type = 8

data = [[] for i in range(total_type)]
# print(data)

for i in range(1, total_line+1): 
    # print(i, data_init[i], end=' ')
    for j in data_init[i].split(',')[3:]: 
        # print(j, end=' ')
        if(j.startswith("NR")):
            data[(i-1)%total_type].append(0.0)
        else:
            data[(i-1)%total_type].append(float(j))
    # print()
# print(data)

# for i in data:
#     print(len(i))

ff.close()

# init model without adagrad
# just consider one type
learningrate = 1e-11
w = [0 for i in range(9)]
b = 0
par = [0 for i in range(9)]

# data[0]=data[0][:20]
# print(data[0])
def f(a): # calc y
    y = b
    # print(len(a))
    for i in range(0, len(a)):
        y += w[i]*a[i]
    return y

def LF(): # loss function
    tot1, tot2 = 0, 0
    for i in range(9, len(data[0])-1):
        tot1 += data[0][i]-f(data[0][i-9: i])
        tot2 += (data[0][i]-f(data[0][i-9: i]))**2
    return tot1, tot2

def getpart(): # get partial w_i,b
    global w,b,learningrate
    parw = [0 for i in range(9)]
    parb = 0
    for i in range(9, len(data[0])-1):
        a = data[0][i-9: i]
        tmp = data[0][i]-f(a)
        # print('a = ', a, ' tmp = ', tmp, 'f(a) = ', f(a))
        parb += -2*(tmp)
        for j in range(len(parw)):
            parw[j] += -2*tmp*a[j]

    # print(parw, parb)
    for i in range(9):
        w[i]=w[i]-learningrate*parw[i]
    b = b-learningrate*parb

# main loop
for i in range(500):
    loss1, loss2 = LF()
    print(loss2)
    getpart()
    # print(w, b, learningrate, end=' -> ')

其实我觉得浮点数精度带来的误差还是很大的…
今天跑不完了，明早起来再跑吧233

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
import numpy as np

# read data

ff = open('train.csv', 'r', True, 'big5hkscs');
data_init = ff.readlines()
total_line = len(data_init)-1
# total_line = 19
total_type = 18

data = [[] for i in range(total_type)]
# print(data)

for i in range(1, total_line+1): 
    # print(i, data_init[i], end=' ')
    for j in data_init[i].split(',')[3:]: 
        # print(j, end=' ')
        if(j.startswith("NR")):
            data[(i-1)%total_type].append(0.0)
        else:
            data[(i-1)%total_type].append(float(j))
    # print()
# print(data)

# for i in data:
#     print(len(i))

ff.close()

# init model without adagrad
# just consider one type
learningrate = 1e-9
# w = [0 for i in range(9)]
# b = 0

w = [0.025824524413455778, 0.013532431831135698, 0.006330265404890848, 0.007117255928494381, 0.01912520613272634, 0.031712687701763935, -0.1291674302877186, 0.21741455874560975, 0.7772617955877607]
b = 0.02353711727271333

par = [0 for i in range(9)]

print(data[9])


def f(a): # calc y
    y = b
    # print(len(a))
    for i in range(0, len(a)):
        y += w[i]*a[i]
    return y

def LF(): # loss function
    tot1, tot2 = 0, 0
    for i in range(9, len(data[9])-1):
        tot1 += data[9][i]-f(data[9][i-9: i])
        tot2 += (data[9][i]-f(data[9][i-9: i]))**2
    return tot1, tot2

def getpart(): # get partial w_i,b
    global w,b,learningrate
    parw = [0 for i in range(9)]
    parb = 0
    for i in range(9, len(data[9])-1):
        a = data[9][i-9: i]
        tmp = data[9][i]-f(a)
        # print('a = ', a, ' tmp = ', tmp, 'f(a) = ', f(a))
        parb += -2*(tmp)
        for j in range(len(parw)):
            parw[j] += -2*tmp*a[j]

    # print(parw, parb)
    for i in range(9):
        w[i]=w[i]-learningrate*parw[i]
    b = b-learningrate*parb

# main loop
times = 1000
learningrate = 1e-9
loss2 = 0
while True:
    lst = loss2
    loss1, loss2 = LF()
    print(loss2)
    getpart()
    if abs(lst-loss2) < 0.1:
        break
    # print(w, b, learningrate, end=' -> ')

learningrate = 1e-10
while True:
    lst = loss2
    loss1, loss2 = LF()
    print(loss2)
    getpart()
    if abs(lst-loss2) < 0.1:
        break

learningrate = 1e-11
while True:
    lst = loss2
    loss1, loss2 = LF()
    print(loss2)
    getpart()
    if abs(lst-loss2) < 0.1:
        break


print(w, b)
print('----------------------')

# read test data & output result
ff = open('test.csv', 'r', True, 'big5hkscs')
fo = open('ans.csv', 'w', True, 'utf-8')

dd = ff.readlines()

print('id,value', file=fo)

for i in dd:
    line = i.split(',')
    # print(line)
    if(line[1].startswith("PM2.5")):
        a = []
        for j in line[2:]:
            a.append(float(j))
        print(line[0], ',', f(a), file=fo, sep='')

总共跑了大概30min，然后得到了一个结果226303.2859914291，得到的w和b在下面，得到的分数是private:7.29680 public:6.02679，也差不多是我这种乱搞做法的比较好的结果了吧

1
2
w = [-0.01196089473848148, -0.0002154942523566853, 0.15614885717614674, -0.18703390611436793, -0.028484991059330843, 0.46031602604586386, -0.5230520726349329, 0.013331313987351568, 1.091108574289711]
b = 0.15907341031251657

我发现那个助教做Demo的时候也是计次…
AdaGrad 有进步但是不太大private:7.24162 public:5.93032
我发现Learning Rate的初始值还是很重要的，不然第一次偏差太大，后面跑回来就比较慢了

我又在这基础上Train了一下，private:7.24797 public:5.92997，分数反而下降了…
不解挠头，我好像不知道怎么变得更好了…
我觉得吧…可能是多加些参数了，把其他的条件考虑进去再加上二次项

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
import numpy as np

# read data

ff = open('train.csv', 'r', True, 'big5hkscs');
data_init = ff.readlines()
total_line = len(data_init)-1
# total_line = 19
total_type = 18

data = [[] for i in range(total_type)]
# print(data)

for i in range(1, total_line+1): 
    # print(i, data_init[i], end=' ')
    for j in data_init[i].split(',')[3:]: 
        # print(j, end=' ')
        if(j.startswith("NR")):
            data[(i-1)%total_type].append(0.0)
        else:
            data[(i-1)%total_type].append(float(j))
    # print()
# print(data)

# for i in data:
#     print(len(i))

ff.close()

# init model without adagrad
# just consider one type

learningrate = 0.02
# w = [0 for i in range(9)]
# b = 0

w = [-0.027453064157575136, -0.023350934972838352, 0.20364257875107225, -0.22111429241618002, -0.054197655604759414, 0.511059787427728, -0.5551156971312344, 0.0029147111942378363, 1.0896194827444017]
b = 1.516876275197267

par = [0 for i in range(9)]

g = [0 for i in range(9)]
gb = 0

print(data[9])


def f(a): # calc y
    y = b
    # print(len(a))
    for i in range(0, len(a)):
        y += w[i]*a[i]
    return y

def LF(): # loss function
    tot1, tot2 = 0, 0
    for i in range(9, len(data[9])-1):
        tot1 += data[9][i]-f(data[9][i-9: i])
        tot2 += (data[9][i]-f(data[9][i-9: i]))**2
    return tot1, tot2

def getpart(): # get partial w_i,b
    global w, b, learningrate, g, gb
    parw = [0 for i in range(9)]
    parb = 0
    for i in range(9, len(data[9])-1):
        a = data[9][i-9: i]
        tmp = data[9][i]-f(a)
        # print('a = ', a, ' tmp = ', tmp, 'f(a) = ', f(a))
        parb += -2*(tmp)
        for j in range(len(parw)):
            parw[j] += -2*tmp*a[j]

    for i in range(9):
        g[i] += parw[i]**2
    gb += parb**2

    # print(parw, parb)
    for i in range(9):
        w[i]=w[i]-learningrate/np.sqrt(g[i])*parw[i]
    b = b-learningrate/np.sqrt(gb)*parb

# main loop


times = 10000
for i in range(times): 
    loss1, loss2 = LF()
    print(loss2)
    getpart()


print(w, b)
print('----------------------')

# read test data & output result
ff = open('test.csv', 'r', True, 'big5hkscs')
fo = open('ans.csv', 'w', True, 'utf-8')

dd = ff.readlines()

print('id,value', file=fo)

for i in dd:
    line = i.split(',')
    # print(line)
    if(line[1].startswith("PM2.5")):
        a = []
        for j in line[2:]:
            a.append(float(j))
        print(line[0], ',', f(a), file=fo, sep='')

哇哦 果然，考虑了二次项，就Training了一次，从0开始，虽然得到的loss有249940.5819514999，但是得到的结果就比之前好得多private:7.19917 public:6.21872
惊了第二次private:6.64656 public:6.05368，直接过了strong baseline，进前30了…
第三次private:6.48045 public:5.98424，嗯，这个作业就先这样吧…继续看看然后搞后面的了

1
2
3
4
5
6
7
8
9
10
11
# train once
w = [0.038419991693316446, 0.007631560179476181, 0.017121148559214105, 0.008416625938076962, 0.03344269223900898, 0.07485213150840617, 0.02749672696408752, 0.22824357598542835, 0.5376247829222051, -0.0009578224442509027, -5.528884121464584e-05, 0.0015892367969355542, -0.002221411021284927, -0.0005837884778601767, 0.004221860266845362, -0.006797969291691342, -0.00225338003274975, 0.006461549785024542] 
b = 0.7707164178028746

# train twice
w = [0.04550231793751899, -0.002754055810508499, 0.01938652727209127, -0.012606727374062035, 0.026644811794796486, 0.08529566547520964, -0.07289173105545857, 0.20945837091642078, 0.6670259504885876, -0.0010213018236748075, 5.8566384278349695e-05, 0.0015977330997915425, -0.0020381249207644136, -0.0005208828351068215, 0.00420797586899282, -0.005646668562707524, -0.0020696412509102606, 0.004923568539656863]
b = 0.9254806340743505

# train third
w = [0.04570395973486438, -0.011294077186582839, 0.025386967127299973, -0.029212382678632333, 0.028545066193682748, 0.11101330569370138, -0.14814515409168333, 0.17550209989307616, 0.7622350658920563, -0.0010221349938858725, 0.00015306217718345381, 0.0015709039506438167, -0.0019038640450172112, -0.0005728314281040582, 0.00400234194537975, -0.004792016150808898, -0.001711617920489784, 0.0038067167936126386]
b = 1.0625348284097278
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
import numpy as np

# read data

ff = open('train.csv', 'r', True, 'big5hkscs');
data_init = ff.readlines()
total_line = len(data_init)-1
# total_line = 19
total_type = 18

data = [[] for i in range(total_type)]
# print(data)

for i in range(1, total_line+1): 
    # print(i, data_init[i], end=' ')
    for j in data_init[i].split(',')[3:]: 
        # print(j, end=' ')
        if(j.startswith("NR")):
            data[(i-1)%total_type].append(0.0)
        else:
            data[(i-1)%total_type].append(float(j))
    # print()
# print(data)

# for i in data:
#     print(len(i))

ff.close()

# init model without adagrad
# just consider one type

learningrate = 1
w = [0 for i in range(18)]
b = 0

g = [0 for i in range(18)]
gb = 0

# print(data[9])


def f(a): # calc y
    y = b
    # print(len(a))
    for i in range(0, len(a)):
        y += w[i]*a[i] + w[i+9]*(a[i]**2)
    return y

def LF(): # loss function
    tot1, tot2 = 0, 0
    for i in range(9, len(data[9])-1):
        tot1 += data[9][i]-f(data[9][i-9: i])
        tot2 += (data[9][i]-f(data[9][i-9: i]))**2
    return tot1, tot2

def getpart(): # get partial w_i,b
    global w, b, learningrate, g, gb
    parw = [0 for i in range(18)]
    parb = 0
    for i in range(9, len(data[9])-1):
        a = data[9][i-9: i]
        tmp = data[9][i]-f(a)
        # print('a = ', a, ' tmp = ', tmp, 'f(a) = ', f(a))
        parb += -2*(tmp)
        for j in range(9):
            parw[j] += -2*tmp*a[j]
            parw[j+9] += -2*tmp*a[j]*a[j]

    for i in range(18):
        g[i] += parw[i]**2
    gb += parb**2

    # print(parw, parb)
    for i in range(18):
        w[i]=w[i]-learningrate/np.sqrt(g[i])*parw[i]
    b = b-learningrate/np.sqrt(gb)*parb

# main loop


times = 10000
for i in range(times): 
    loss1, loss2 = LF()
    print(loss2)
    getpart()


print(w, b)
print('----------------------')

# read test data & output result
ff = open('test.csv', 'r', True, 'big5hkscs')
fo = open('ans.csv', 'w', True, 'utf-8')

dd = ff.readlines()

print('id,value', file=fo)

for i in dd:
    line = i.split(',')
    # print(line)
    if(line[1].startswith("PM2.5")):
        a = []
        for j in line[2:]:
            a.append(float(j))
        print(line[0], ',', f(a), file=fo, sep='')

加了三次方，感觉要Overfitting…
哦 结果变得肥肠爆炸…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
import numpy as np

# read data

ff = open('train.csv', 'r', True, 'big5hkscs');
data_init = ff.readlines()
total_line = len(data_init)-1
# total_line = 19
total_type = 18

data = [[] for i in range(total_type)]
# print(data)

for i in range(1, total_line+1): 
    # print(i, data_init[i], end=' ')
    for j in data_init[i].split(',')[3:]: 
        # print(j, end=' ')
        if(j.startswith("NR")):
            data[(i-1)%total_type].append(0.0)
        else:
            data[(i-1)%total_type].append(float(j))
    # print()
# print(data)

# for i in data:
#     print(len(i))

ff.close()

# init model without adagrad
# just consider one type

learningrate = 1

w = [0 for i in range(27)]
b = 0

g = [0 for i in range(27)]
gb = 0

# print(data[9])


def f(a): # calc y
    y = b
    # print(len(a))
    for i in range(9):
        y += w[i]*a[i] + w[i+9]*(a[i]**2) + w[i+18]*(a[i]**3)
    return y

def LF(): # loss function
    tot1, tot2 = 0, 0
    for i in range(9, len(data[9])-1):
        tot1 += data[9][i]-f(data[9][i-9: i])
        tot2 += (data[9][i]-f(data[9][i-9: i]))**2
    return tot1, tot2

def getpart(): # get partial w_i,b
    global w, b, learningrate, g, gb
    parw = [0 for i in range(27)]
    parb = 0
    for i in range(9, len(data[9])-1):
        a = data[9][i-9: i]
        tmp = data[9][i]-f(a)
        # print('a = ', a, ' tmp = ', tmp, 'f(a) = ', f(a))
        parb += -2*(tmp)
        for j in range(9):
            parw[j] += -2*tmp*a[j]
            parw[j+9] += -2*tmp*a[j]*a[j]
            parw[j+18] += -2*tmp*(a[j]**3)

    for i in range(27):
        g[i] += parw[i]**2
    gb += parb**2

    # print(parw, parb)
    for i in range(27):
        w[i]=w[i]-learningrate/np.sqrt(g[i])*parw[i]
    b = b-learningrate/np.sqrt(gb)*parb

# main loop


times = 10000
for i in range(times): 
    loss1, loss2 = LF()
    print(loss2)
    getpart()


print(w, b)
print('----------------------')

# read test data & output result
ff = open('test.csv', 'r', True, 'big5hkscs')
fo = open('ans.csv', 'w', True, 'utf-8')

dd = ff.readlines()

print('id,value', file=fo)

for i in dd:
    line = i.split(',')
    # print(line)
    if(line[1].startswith("PM2.5")):
        a = []
        for j in line[2:]:
            a.append(float(j))
        print(line[0], ',', f(a), file=fo, sep='')

hw2

https://www.kaggle.com/c/ml2019spring-hw2
作业说明

这就是一个Binary Classification，直接用X_train提出的特征就可以了

Probabilistic Generateive Model

假设是高斯分布来算的，所以不需要训练，秒出结果，得分也不太高private:0.76047 public:0.76707
也不知道是不是我写挂了…

没错是我写挂了，这全是0吧…

ps.手写矩阵运算好麻烦啊，虽然也不长，但是种类太多，感觉不如C++方便，可能我都写C++写惯了，抽时间看一下numpy好了

probabilistic_generateive_model.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
import numpy as np
from math import pi, log, exp

file_x = open('X_train', 'r')
file_y = open('Y_train', 'r')
file_xt = open('X_test', 'r')
file_yt = open('Y_test', 'w')

#x_train = y_train = x_test = []
#mu1 = mu2 = sigm = []

def deal_data(x):
    for i in range(0, len(x)):
        x[i]=x[i].split(',')
        for j in range(0, len(x[i])):
            x[i][j]=int(x[i][j])

def load_data():
    global x_train, y_train, x_test
    x_train = file_x.readlines()[1:]
    y_train = file_y.readlines()[1:]
    x_test = file_xt.readlines()[1:]
    
    deal_data(x_train)
    deal_data(y_train)
    deal_data(x_test)

def mult(a, b):
    c = []
    for i in range(len(a)):
        c.append([])
        for j in range(len(b[0])):
            c[i].append(0)
            
    for i in range(len(a)):
        for j in range(len(b)):
            for k in range(len(b[0])):
                c[i][k] += a[i][j]*b[j][k]
    return c

def mult_num(a, b):
    c = []
    for i in range(len(a)):
        c.append([])
        for j in range(len(a[i])):
            c[i].append(a[i][j]*b)
    return c

def add(a, b):
    c = []
    for i in range(len(a)):
        c.append([])
        for j in range(len(a[0])):
            c[i].append(0)
            
    for i in range(len(a)):
        for j in range(len(a[0])):
            c[i][j] = a[i][j]+b[i][j]
    return c

def transpose(a):
    c = []
    for i in range(len(a[0])):
        c.append([])
        for j in range(len(a)):
            c[i].append(a[j][i])
    return c 

    
def train():
    global x_train, y_train, x_test
    global n, n0, n1, dim, mu0, mu1, sigm, sigm0, sigm1, w, b
    n = len(y_train)
    n0 = n1 = 0
    dim = len(x_train[0])
    mu0 = []
    mu1 = []
    
    for i in range(dim):
        mu0.append(0)
        mu1.append(0)
    
    for i in range(n):
        if y_train[i][0] == 0:
            n0+=1
            for j in range(dim):
                mu0[j] += x_train[i][j]
        else:
            n1+=1
            for j in range(dim):
                mu1[j] += x_train[i][j]
    
    for i in range(dim):
        mu0[i] /= n0
        mu1[i] /= n1
    
    sigm0 = 0
    sigm1 = 0
#    for i in range(dim):
#        sigm0.append(0)
#        sigm1.append(0)
    
    for i in range(n):
        if y_train[i][0] == 0:
            for j in range(dim):
                sigm0+=(x_train[i][j]-mu0[j])**2
        else:
            for j in range(dim):
                sigm1+=(x_train[i][j]-mu1[j])**2
    
    sigm0 /= n0
    sigm1 /= n1
    sigm = n0/(n0+n1)*sigm0 + n1/(n0+n1)*sigm1
    
    #change to vector
        
    w = []
    for i in range(dim):
        w.append((mu0[i]-mu1[i])/sigm)
    
    for i in range(dim):
        mu0[i] = [mu0[i]]
        mu1[i] = [mu1[i]]
    
    b0 = mult_num(mult(transpose(mu0), mu0), -0.5/sigm)
    b1 = mult_num(mult(transpose(mu1), mu1), 0.5/sigm)
    b = add(b0, b1)
    b = b[0][0]+log(n0/n1)
    
#    print(b)

#    for i in range(dim):
#        sigm0[i] /= n0
#        sigm1[i] /= n1

def sigmoid(z):
    return 1/(1+exp(-z))
  
def get_probability(x):
    global w, b
    z=0
    for i in range(dim):
       z += w[i]*x[i]
    z += b
    return sigmoid(z)

def get_predict(x):
    if get_probability(x)>0.5:
        return 0
    else:
        return 1

def test():
    print('id,label', file=file_yt)
    for i in range(len(x_test)):
        print(i+1, get_predict(x_test[i]),sep=',', file=file_yt)

def main():    
    load_data()
    print('load_data() ok')
    train()
    print('train() ok')
    test()
    print('test() ok')
    

if __name__ == '__main__': 
    main()

Logistic Regression Model

设$s(x)=\frac{1}{1+e^{-x}}$，那么$s’(x)=s(x)(1-s(x))$，反过来证明很简单，正推求导还是有些麻烦的
这个1w数据，又有很多浮点运算…实在是难顶…好慢…

哎，写了个Logistic Regression，train来train去，0还是太少了，我佛啦

我发现它每个几个就会出现一次结果非常糟的情况…

参考了下别人的代码魔改了一发，就是加上了AdaGrad，然后求偏微分的时候除总数，再加上Regularization

哇！ 我懂了！
有个肥肠肥肠肥肠肥肠肥肠肥肠肥肠肥肠肥肠重要的数据处理就是把一些范围肥肠大的数据范围变小，就除均值就可以了，在这里，有第0,1,3,4,5有关年龄，收入支出的部分范围特别大，把他范围缩小，然后再train就不会出现之前那样每个几个就会出现一次结果非常糟的情况了，train 30次就能得到private:0.84363 public:0.84656

最高也就private: 0.85345 public:0.85356

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
import numpy as np

file_x = open('X_train', 'r')
file_y = open('Y_train', 'r')
file_xt = open('X_test', 'r')
file_yt = open('Y_test', 'w')

#x_train = y_train = x_test = []
#mu1 = mu2 = sigm = []

def deal_data(x):
    for i in range(0, len(x)):
        x[i]=x[i].split(',')
        for j in range(0, len(x[i])):
            x[i][j]=int(x[i][j])

def deal_col(x, a, xt):
    mean_x = 0
    for i in range(len(x)):
        mean_x += x[i][a]
    mean_x /= len(x)
    for i in range(len(x)):
        x[i][a] /= mean_x
    for i in range(len(xt)):
        xt[i][a] /= mean_x

def load_data():
    global x_train, y_train, x_test, x_val, y_val
    x_train = file_x.readlines()[1:]
    y_train = file_y.readlines()[1:]
    x_test = file_xt.readlines()[1:]
    
    deal_data(x_train)
    deal_data(y_train)
    deal_data(x_test)
    
    deal_col(x_train, 0, x_test)
    deal_col(x_train, 1, x_test)
    deal_col(x_train, 3, x_test)
    deal_col(x_train, 4, x_test)
    deal_col(x_train, 5, x_test)
    
    
    x = x_train
    y = y_train
    x_train, x_val = x[:20000], x[20000:]
    y_train, y_val = y[:20000], y[20000:]
    
def sigmoid(z):
    return 1/(1+np.exp(-z))

def get_loss(x):
    global w, b, dim
    for i in range(dim):
        pass
    
def train():
    global x_train, y_train, x_test, x_val, y_val
    global w, b, dim
    w = []
    dim = len(x_train[0])
    b = 0
    lr = 1
    y = []
    reg_rate = 0.001
    
    gradient_w_sum = []
    gradient_b_sum = 0
    
    for i in range(dim):
        w.append(1)
        gradient_w_sum.append(0)
    
    for t in range(200):
        print(t, ' testing...', end='')
        y = []
        err = 0
        for i in range(len(x_train)):
            y.append(b)
            for j in range(dim):
                y[i] += x_train[i][j]*w[j]
            y[i] = sigmoid(y[i])
            if (y[i]>0.5) != y_train[i][0]:
                err += 1
        
        print(' train acc:', 1-err/len(x_train), end='')
        
        err = 0
        for i in range(len(x_val)):
            yt = b
            for j in range(dim):
                yt += x_val[i][j]*w[j]
            yt = sigmoid(yt)
            if (yt>0.5) != y_val[i][0]:
                err += 1
        print(' val acc:', 1-err/len(x_val))
        
        
        for i in range(dim):
            gradient_w = 2*reg_rate*w[i]
            for j in range(len(x_train)):
                gradient_w += -(y_train[j][0]-y[j])*x_train[j][i]
            gradient_w = gradient_w/len(x_train)
            gradient_w_sum[i] += gradient_w**2
            w[i] = w[i] - lr/np.sqrt(gradient_w_sum[i])*gradient_w

        gradient_b = 0
        for i in range(len(x_train)):
            gradient_b += -(y_train[i][0]-y[i])
            
        gradient_b = gradient_b/len(x_train)
        gradient_b_sum += gradient_b**2
        b = b - lr/np.sqrt(gradient_b_sum)*gradient_b
    

def get_probability(x):
    global w, b, dim
    z = 0
    for i in range(dim):
       z += w[i]*x[i]
    z += b
    return sigmoid(z)

def get_predict(x):
    # print(get_probability(x))
    if get_probability(x)>0.5:
        return 1
    else:
        return 0

def test():
    print('id,label', file=file_yt)
    nn = 0
    for i in range(len(x_test)):
        if get_predict(x_test[i])==0:
            nn += 1
        print(i+1, get_predict(x_test[i]),sep=',', file=file_yt)
    print(nn, len(x_test))

def main():
    load_data()
    print('load_data() ok')
    train()
    print('train() ok')
    print(w, b)
    test()
    print('test() ok')

#main
if __name__ == '__main__':
    main()

我又修改了一下处理数据那部分，把范围大的变成$\frac{x-\min\{x\}}{\max\{x\}-\min\{x\}}$
结果稍差了一丢丢private:0.84903 public:0.85282
不管了…看后面的了

1
2
3
4
5
6
7
8
9
10
def deal_col(x, a, xt):
    mxx = x[0][a]
    mix = x[0][a]
    for i in range(len(x)):
        mxx = max(mxx, x[i][a])
        mix = min(mix, x[i][a])
    for i in range(len(x)):
        x[i][a] = (x[i][a]-mix)/(mxx-mix)
    for i in range(len(xt)):
        xt[i][a] = (xt[i][a]-mix)/(mxx-mix)

hw3

https://www.kaggle.com/c/ml2019spring-hw3
作业说明

Model

要做的是一个人脸表情的分类
数据处理的部分用了好长时间，主要是显式循环太致命了…我还不太会处理csv，所以直接暴力读文件处理了
这是最一开始写的，得到的output是这样

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Epoch 1/20
2019-07-30 23:58:40.386315: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-07-30 23:58:40.410366: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2494135000 Hz
2019-07-30 23:58:40.411071: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5616a3e35300 executing computations on platform Host. Devices:
2019-07-30 23:58:40.411112: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2019-07-30 23:58:40.587853: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
28709/28709 [==============================] - 21s 724us/step - loss: 1.6676 - acc: 0.3480
Epoch 2/20
28709/28709 [==============================] - 21s 715us/step - loss: 1.5076 - acc: 0.4226
Epoch 3/20
28709/28709 [==============================] - 21s 748us/step - loss: 1.3983 - acc: 0.4698
Epoch 4/20
28709/28709 [==============================] - 21s 737us/step - loss: 1.2961 - acc: 0.5130
Epoch 5/20
28709/28709 [==============================] - 21s 742us/step - loss: 1.1895 - acc: 0.5582
Epoch 6/20
28709/28709 [==============================] - 21s 739us/step - loss: 1.0732 - acc: 0.6067
Epoch 7/20
28709/28709 [==============================] - 22s 772us/step - loss: 0.9517 - acc: 0.6570
Epoch 8/20
28709/28709 [==============================] - 23s 797us/step - loss: 0.8311 - acc: 0.7044
Epoch 9/20
28709/28709 [==============================] - 23s 792us/step - loss: 0.7124 - acc: 0.7521
Epoch 10/20
28709/28709 [==============================] - 22s 769us/step - loss: 0.6000 - acc: 0.7962
Epoch 11/20
28709/28709 [==============================] - 22s 759us/step - loss: 0.5022 - acc: 0.8347
Epoch 12/20
28709/28709 [==============================] - 23s 793us/step - loss: 0.4063 - acc: 0.8699
Epoch 13/20
28709/28709 [==============================] - 23s 800us/step - loss: 0.3259 - acc: 0.9009
Epoch 14/20
28709/28709 [==============================] - 23s 793us/step - loss: 0.2617 - acc: 0.9235
Epoch 15/20
28709/28709 [==============================] - 23s 790us/step - loss: 0.1984 - acc: 0.9473
Epoch 16/20
28709/28709 [==============================] - 23s 801us/step - loss: 0.1556 - acc: 0.9639
Epoch 17/20
28709/28709 [==============================] - 23s 789us/step - loss: 0.1387 - acc: 0.9668
Epoch 18/20
28709/28709 [==============================] - 22s 768us/step - loss: 0.1521 - acc: 0.9577
Epoch 19/20
28709/28709 [==============================] - 23s 818us/step - loss: 0.0914 - acc: 0.9804
Epoch 20/20
28709/28709 [==============================] - 22s 755us/step - loss: 0.0693 - acc: 0.9884
28709/28709 [==============================] - 11s 392us/step

Train Acc: 0.9921975671163343

提交得到的分数很低private:0.47394 public:0.49456
我觉得是batch_size和epochs没设置好，明天调
有一个地方就是Conv2D的Dimension一开始实在是没搞懂，好像必须要3维才可以，然后就搞成我写的这样可以成功运行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
import numpy as np
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.layers import Conv2D, MaxPooling2D, Flatten
from keras.optimizers import SGD, Adam
from keras.utils import np_utils
def load_data():
    train_f = open('train.csv', 'r')
    test_f = open('test.csv', 'r')
    train_data = train_f.readlines()[1:]
    test_data = test_f.readlines()[1:]
    
    x_train = []
    y_train = []
    x_test = []
    
    for i in range(len(train_data)):
        train_data[i]=train_data[i].split(',')
        x_train.append(train_data[i][1].split(' '))
        for j in range(len(x_train[i])):
            x_train[i][j]=int(x_train[i][j])
        y_train.append(int(train_data[i][0]))
    
    for i in range(len(test_data)):
        test_data[i]=test_data[i].split(',')
        x_test.append(test_data[i][1].split(' '))
        for j in range(len(x_test[i])):
            x_test[i][j]=int(x_test[i][j])
    
    x = np.array(x_train)
    y = np.array(y_train)
    xx = np.array(x_test)
    
    x_train = x
    y_train = y
    x_test = xx
    
    # deal data
    total_data = x_train.shape[0]
    x_train = x_train.reshape(total_data, 48, 48, 1)
    x_test = x_test.reshape(x_test.shape[0], 48, 48, 1)
    x_train = x_train.astype('float32')
    x_test = x_test.astype('float32')
    x_train = x_train / 255
    x_test = x_test / 255
    
    y_train = np_utils.to_categorical(y_train, 7)
    
    return x_train, y_train, x_test
    

def train(x_train, y_train):
    model = Sequential()
    model.add(Conv2D(25, (6, 6), input_shape=(x_train.shape[1], x_train.shape[2], x_train.shape[3])))
    model.add(MaxPooling2D((2, 2)))
    model.add(Flatten())
    model.add(Dense(output_dim = 100))
    model.add(Activation('relu'))
    model.add(Dense(output_dim = 7))
    model.add(Activation('softmax'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    model.fit(x_train, y_train, batch_size = 100, epochs = 20)
    result = model.evaluate(x_train, y_train, batch_size=10)
    print('\nTrain Acc:', result[1])
    return model

def test(model, x_test):
    y_test = model.predict(x_test)
    test_f = open('y_test.csv', 'w')
    print('id,label', file=test_f)
    for i in range(len(y_test)):
        print(i, np.argmax(y_test[i]), sep=',', file=test_f)
    test_f.close()

def main():
    x_train, y_train, x_test = load_data()
    print('load_data() ok')
    model = train(x_train, y_train)
    print('train() ok')
    test(model, x_test)
    print('test() ok')

if __name__ == '__main__':
    main()

后来我改了一下，得分private: 0.45500 public:0.45583，还是不太行嗷，回头改一下Model，我先睡了zzz

1
2
3
4
5
6
7
8
9
10
11
12
13
14
def train(x_train, y_train):
    model = Sequential()
    model.add(Conv2D(25, (6, 6), input_shape=(x_train.shape[1], x_train.shape[2], x_train.shape[3])))
    model.add(MaxPooling2D((2, 2)))
    model.add(Flatten())
    model.add(Dense(output_dim = 100))
    model.add(Activation('relu'))
    model.add(Dense(output_dim = 7))
    model.add(Activation('softmax'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    model.fit(x_train, y_train, batch_size = 10, epochs = 20)
    result = model.evaluate(x_train, y_train, batch_size=1)
    print('\nTrain Acc:', result[1])
    return model

output

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Epoch 1/20
2019-07-31 00:08:55.510380: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-07-31 00:08:55.534347: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2494135000 Hz
2019-07-31 00:08:55.535015: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5639b5442300 executing computations on platform Host. Devices:
2019-07-31 00:08:55.535039: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2019-07-31 00:08:55.662054: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
28709/28709 [==============================] - 75s 3ms/step - loss: 1.6555 - acc: 0.3503
Epoch 2/20
28709/28709 [==============================] - 85s 3ms/step - loss: 1.5076 - acc: 0.4157
Epoch 3/20
28709/28709 [==============================] - 83s 3ms/step - loss: 1.4020 - acc: 0.4644
Epoch 4/20
28709/28709 [==============================] - 84s 3ms/step - loss: 1.3111 - acc: 0.5012
Epoch 5/20
28709/28709 [==============================] - 81s 3ms/step - loss: 1.2128 - acc: 0.5410
Epoch 6/20
28709/28709 [==============================] - 83s 3ms/step - loss: 1.1142 - acc: 0.5840
Epoch 7/20
28709/28709 [==============================] - 82s 3ms/step - loss: 1.0072 - acc: 0.6264
Epoch 8/20
28709/28709 [==============================] - 82s 3ms/step - loss: 0.9050 - acc: 0.6643
Epoch 9/20
28709/28709 [==============================] - 81s 3ms/step - loss: 0.8023 - acc: 0.7046
Epoch 10/20
28709/28709 [==============================] - 81s 3ms/step - loss: 0.7115 - acc: 0.7392
Epoch 11/20
28709/28709 [==============================] - 77s 3ms/step - loss: 0.6215 - acc: 0.7720
Epoch 12/20
28709/28709 [==============================] - 74s 3ms/step - loss: 0.5508 - acc: 0.7971
Epoch 13/20
28709/28709 [==============================] - 76s 3ms/step - loss: 0.4865 - acc: 0.8204
Epoch 14/20
28709/28709 [==============================] - 77s 3ms/step - loss: 0.4371 - acc: 0.8413
Epoch 15/20
28709/28709 [==============================] - 78s 3ms/step - loss: 0.3915 - acc: 0.8592
Epoch 16/20
28709/28709 [==============================] - 76s 3ms/step - loss: 0.3626 - acc: 0.8690
Epoch 17/20
28709/28709 [==============================] - 76s 3ms/step - loss: 0.3190 - acc: 0.8858
Epoch 18/20
28709/28709 [==============================] - 76s 3ms/step - loss: 0.3013 - acc: 0.8930
Epoch 19/20
28709/28709 [==============================] - 76s 3ms/step - loss: 0.2744 - acc: 0.9027
Epoch 20/20
28709/28709 [==============================] - 77s 3ms/step - loss: 0.2455 - acc: 0.9124
28709/28709 [==============================] - 11s 380us/step

Train Acc: 0.9182486253005799

batch_size差不多是多少个数据更新一次参数，过大就没啥效果，过小梯度可能就会出现问题

于是我又调了一下参数，顺便加了一层Conv2D

1
2
3
4
5
6
7
8
9
10
11
12
model = Sequential()
model.add(Conv2D(25, (6, 6), input_shape=(x_train.shape[1], x_train.shape[2], x_train.shape[3])))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(25, (6, 6)))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(output_dim = 100))
model.add(Activation('relu'))
model.add(Dense(output_dim = 7))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train, batch_size = 100, epochs = 20)

结果变好了一点private:0.49651 public:0.50766
但是在Training Data上面的准确率居然可以到0.94？？？是不是Overfitted了

我增加了验证集，确实是Overfitted了
既然the deeper the better，那么我把他改的比较Deep了,然后可能参数过多了…train了40+min，结果正确率没变…跟蒙的正确率差不多…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
def train(x_train, y_train, x_val, y_val):
    model = Sequential()
    model.add(Conv2D(32, (3, 3), input_shape=(x_train.shape[1], x_train.shape[2], x_train.shape[3])))
    model.add(Conv2D(32, (3, 3)))
    model.add(MaxPooling2D((2, 2)))
    model.add(Conv2D(64, (3, 3)))
    model.add(MaxPooling2D((2, 2)))
    model.add(Conv2D(64, (2, 2)))
    model.add(MaxPooling2D((2, 2)))
    model.add(Conv2D(64, (2, 2)))
    model.add(MaxPooling2D((2, 2)))
    model.add(Flatten())
    model.add(Dense(output_dim = 100))
    #model.add(Dropout(0.3))
    model.add(Activation('relu'))
    model.add(Dense(output_dim = 7))
    model.add(Activation('softmax'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    model.fit(x_train, y_train, batch_size = 200, epochs = 10)
    result = model.evaluate(x_train, y_train, batch_size=1)
    print('\nTrain Acc:', result[1])
    result = model.evaluate(x_val, y_val, batch_size=1)
    print('\nVal Acc:', result[1])
    return model

结果还是不理想
Train Acc: 0.4293785310734463 Val Acc: 0.38791915480018374.
private 0.38088 public:0.40011.

train一次真的好久…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
def train(x_train, y_train, x_val, y_val):
    model = Sequential()
    model.add(Conv2D(32, (3, 3), input_shape=(x_train.shape[1], x_train.shape[2], x_train.shape[3])))
    model.add(MaxPooling2D((2, 2)))
    model.add(Conv2D(64, (3, 3)))
    model.add(MaxPooling2D((2, 2)))
    model.add(Conv2D(64, (2, 2)))
    model.add(MaxPooling2D((2, 2)))
    #model.add(Conv2D(64, (2, 2)))
    #model.add(MaxPooling2D((2, 2)))
    model.add(Flatten())
    model.add(Dense(output_dim = 100))
    model.add(Dropout(0.3))
    model.add(Activation('relu'))
    model.add(Dense(output_dim = 7))
    model.add(Activation('softmax'))
    #model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.1), metrics=['accuracy'])
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    
    model.fit(x_train, y_train, batch_size = 200, epochs = 10)
    result = model.evaluate(x_train, y_train, batch_size=1)
    print('\nTrain Acc:', result[1])
    result = model.evaluate(x_val, y_val, batch_size=1)
    print('\nVal Acc:', result[1])
    return model

Train Acc: 0.5773211339433029Val Acc: 0.513435920992191.
private: 0.49512 public:0.51518.

把epochs改成30多Train一会
Train Acc: 0.9242537873106345 Val Acc: 0.5257234726688103得分private:0.51518 public:0.53998

我把之前的超复杂的model Train了100个epochs…结果没啥变化..噗…我可是train了5h 12-17 private: 0.51908 public:0.53413
Train Acc: 0.9301534923253837 Val Acc: 0.5213596692696371感觉又overfitted了？

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
def train(x_train, y_train, x_val, y_val):
    model = Sequential()
    model.add(Conv2D(64, (5, 5), input_shape=(x_train.shape[1], x_train.shape[2], x_train.shape[3])))
    model.add(MaxPooling2D((2, 2)))
    model.add(BatchNormalization(axis = -1))
    model.add(Dropout(0.3))

    model.add(Conv2D(128, (5, 5)))
    model.add(MaxPooling2D((2, 2)))
    model.add(BatchNormalization(axis = -1))
    model.add(Dropout(0.3))

    model.add(Conv2D(256, (5, 5)))
    model.add(MaxPooling2D((2, 2)))
    model.add(BatchNormalization(axis = -1))
    model.add(Dropout(0.3))
    #model.add(Conv2D(64, (2, 2)))
    #model.add(MaxPooling2D((2, 2)))
    model.add(Flatten())
    model.add(Dense(output_dim = 128))
    model.add(Dropout(0.5))
    model.add(Activation('relu'))
    model.add(Dense(output_dim = 7))
    model.add(Activation('softmax'))
    #model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.1), metrics=['accuracy'])
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model

吐了，train了一天，最好的结果0.55642 public:0.56505，改天来改改model再train8
—- Simple Baseline —-都没过orz

不管了，我一直train不起来，这是我最后train的一次model
主要是看这个改了一下
怎么感觉是toolkit的问题…可能我姿势不太对？

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
import numpy as np
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.layers import Convolution2D, MaxPooling2D, Flatten, BatchNormalization
from keras.optimizers import SGD, Adam
from keras.utils import np_utils
from keras.models import load_model

def load_data():
    train_f = open('train.csv', 'r')
    test_f = open('test.csv', 'r')
    train_data = train_f.readlines()[1:]
    test_data = test_f.readlines()[1:]
    
    x_train = []
    y_train = []
    x_test = []
    
    for i in range(len(train_data)):
        train_data[i]=train_data[i].split(',')
        x_train.append(train_data[i][1].split(' '))
        for j in range(len(x_train[i])):
            x_train[i][j]=int(x_train[i][j])
        y_train.append(int(train_data[i][0]))
    
    for i in range(len(test_data)):
        test_data[i]=test_data[i].split(',')
        x_test.append(test_data[i][1].split(' '))
        for j in range(len(x_test[i])):
            x_test[i][j]=int(x_test[i][j])
    
    x = np.array(x_train)
    y = np.array(y_train)
    xx = np.array(x_test)
    
    x_train = x
    y_train = y
    x_test = xx
    
    # deal data
    total_data = x_train.shape[0]
    x_train = x_train.reshape(total_data, 48, 48, 1)
    x_test = x_test.reshape(x_test.shape[0], 48, 48, 1)
    x_train = x_train.astype('float32')
    x_test = x_test.astype('float32')
    x_train = x_train / 255
    x_test = x_test / 255
    
    y_train = np_utils.to_categorical(y_train, 7)
    
    num = 26000
    x_val, y_val = x_train[num:, ], y_train[num:, ]
    
    x_train = x_train[:num, ]
    y_train = y_train[:num, ]
    
    return x_train, y_train, x_val, y_val, x_test
    

def train(x_train, y_train, x_val, y_val):
    model = Sequential()
    model.add(Convolution2D(32, 3, 3, input_shape=(x_train.shape[1], x_train.shape[2], x_train.shape[3])))
    model.add(Convolution2D(32, 3, 3, border_mode='same', activation='relu'))
    model.add(Convolution2D(32, 3, 3, border_mode='same', activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    
    model.add(Convolution2D(64, 3, 3, border_mode='same', activation='relu'))
    model.add(Convolution2D(64, 3, 3, border_mode='same', activation='relu'))
    model.add(Convolution2D(64, 3, 3, border_mode='same', activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    
    model.add(Convolution2D(128, 3, 3, border_mode='same', activation='relu'))
    model.add(Convolution2D(128, 3, 3, border_mode='same', activation='relu'))
    model.add(Convolution2D(128, 3, 3, border_mode='same', activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    
    model.add(Flatten())
    model.add(Dense(units = 64, activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(units = 64, activation='relu'))
    model.add(Dropout(0.5))
        
    model.add(Dense(units = 7, activation='softmax'))
    #model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.1), metrics=['accuracy'])
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    
    model.fit(x_train, y_train, nb_epoch=50, batch_size=128,
          validation_split=0.3, shuffle=True, verbose=1)
    model.save('my_model.h5')
    loss_and_metrics = model.evaluate(x_train, y_train, batch_size=128, verbose=1)
    print('Done!')
    print('Loss: ', loss_and_metrics[0])
    print(' Acc: ', loss_and_metrics[1])
    
    return model

def test(model, x_test):
    y_test = model.predict(x_test)
    test_f = open('y_test.csv', 'w')
    print('id,label', file=test_f)
    for i in range(len(y_test)):
        print(i, np.argmax(y_test[i]), sep=',', file=test_f)
    test_f.close()

def main():
    x_train, y_train, x_val, y_val, x_test = load_data()
    print('load_data() ok')
    model = train(x_train, y_train, x_val, y_val)
    
    print('train() ok')
    test(model, x_test)
    print('test() ok')

if __name__ == '__main__':
    main()

Data

最后来做一下，有趣的数据和Model的可视化

training data前十张图片

标签分别是
0 Angry 1 Angry 2 Fear 3 Sad 4 Neutral
5 Fear 6 Sad 7 Happy 8 Happy 9 Fear

test data前十张照片

我给出了y_test
0 Happy 1 Happy 2 Happy 3 Happy 4 Neutral
5 Surprise 6 Angry 7 Neutral 8 Fear 9 Neutral

感觉还挺对的啊233
相关代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import matplotlib.pyplot as plt

def show(x_train, y_train, x_val, y_val, x_test):
    emo = ['Angry', 'Disgust', 'Fear', 'Happy',
           'Sad', 'Surprise', 'Neutral']
    for i in range(10):
        plt.imshow(x_train[i])
        plt.axis('off')
        plt.savefig('pic'+str(i)+'.jpg')
        plt.show()
        print(i, emo[y_train[i]])
    
    for i in range(10):
        plt.imshow(x_test[i])
        plt.axis('off')
        plt.savefig('test_pic'+str(i)+'.jpg')
        plt.show()

下面就是有关Model的部分

这是train出最好结果的Model结构print(model.summary())

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 44, 44, 32)        832       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 22, 22, 32)        0         
_________________________________________________________________
batch_normalization_1 (Batch (None, 22, 22, 32)        128       
_________________________________________________________________
dropout_1 (Dropout)          (None, 22, 22, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 18, 18, 64)        51264     
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 9, 9, 64)          0         
_________________________________________________________________
batch_normalization_2 (Batch (None, 9, 9, 64)          256       
_________________________________________________________________
dropout_2 (Dropout)          (None, 9, 9, 64)          0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 7, 7, 128)         73856     
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 3, 3, 128)         0         
_________________________________________________________________
batch_normalization_3 (Batch (None, 3, 3, 128)         512       
_________________________________________________________________
dropout_3 (Dropout)          (None, 3, 3, 128)         0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 2, 2, 128)         65664     
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 1, 1, 128)         0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 128)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 128)               16512     
_________________________________________________________________
activation_1 (Activation)    (None, 128)               0         
_________________________________________________________________
dropout_4 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 64)                8256      
_________________________________________________________________
activation_2 (Activation)    (None, 64)                0         
_________________________________________________________________
dropout_5 (Dropout)          (None, 64)                0         
_________________________________________________________________
dense_3 (Dense)              (None, 7)                 455       
_________________________________________________________________
activation_3 (Activation)    (None, 7)                 0         
=================================================================
Total params: 217,735
Trainable params: 217,287
Non-trainable params: 448
_________________________________________________________________
None

第一个卷积层参数print(model.get_layer('conv2d_1').get_weights())

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
[array([[[[ 1.98697541e-02,  7.08070025e-02,  2.13078111e-02,
          -5.15683815e-02,  1.00081712e-01,  2.15147361e-02,
           5.15370332e-02, -5.25256731e-02, -7.89779201e-02,
           6.46693213e-03,  3.96410637e-02, -5.25688417e-02,
           1.44804746e-01, -7.12847039e-02,  1.06713027e-01,
           1.60096228e-01,  1.89897269e-01, -8.76477435e-02,
          -2.29081400e-02, -3.11734769e-02, -1.93705603e-01,
          -6.44565597e-02, -5.65660931e-03, -1.87695608e-01,
           1.21000633e-01, -6.11960106e-02, -4.44339998e-02,
           1.05157614e-01, -1.10467389e-01, -2.06037536e-01,
           2.08152682e-01,  1.53192570e-02]],

        [[-7.94749632e-02, -1.10797726e-01, -2.95411889e-02,
           1.05928695e-02, -9.10679698e-02, -1.85829073e-01,
           2.02179357e-01,  1.05242565e-01, -1.98338836e-01,
           6.08536191e-02, -3.61365639e-02,  5.57313822e-02,
          -1.43119842e-02,  3.56776677e-02, -6.43971115e-02,
           1.39809683e-01,  8.17038235e-04, -1.02193907e-01,
          -8.24483410e-02, -1.46627268e-02, -3.01948309e-01,
          -5.91862574e-02,  7.99618885e-02,  2.42080484e-02,
           9.46005136e-02,  2.90510654e-01,  4.47756145e-04,
          -1.72438115e-01, -1.18252495e-03, -5.75673319e-02,
           1.69455364e-01,  3.26001421e-02]],

        [[-4.84515466e-02, -9.15088356e-02, -5.86454906e-02,
          -1.31094322e-01, -6.93618506e-03, -2.83256590e-01,
          -1.69737684e-03, -1.20050490e-01,  2.41740972e-01,
           9.86552760e-02, -9.77197811e-02,  1.34425595e-01,
          -9.40313116e-02,  1.00749932e-01,  1.31006762e-02,
           9.30177867e-02, -1.66146964e-01,  5.57237025e-03,
           7.31411353e-02, -8.76804162e-03,  1.37711242e-01,
          -4.58096415e-02,  2.22220905e-02,  1.01391926e-01,
          -7.97675103e-02,  3.93110923e-02,  8.29865634e-02,
          -1.85488507e-01,  1.11202924e-02,  3.66388783e-02,
           1.33667052e-01,  1.18211597e-01]],

        [[-4.10749484e-03,  1.25970155e-01,  3.67107615e-02,
           4.58605774e-02, -8.73558037e-03, -1.11491427e-01,
          -2.09319130e-01, -3.44122830e-03,  5.52160554e-02,
           1.17073983e-01,  1.50324449e-01,  1.80855498e-01,
          -1.18359365e-01,  1.13875926e-01, -7.70208091e-02,
           5.23166358e-02, -2.35963296e-02,  4.03642021e-02,
           1.39719369e-02,  2.40484951e-03,  1.66882589e-01,
           2.26039141e-02, -2.48743184e-02, -1.94001179e-02,
           2.79328898e-02,  6.15491420e-02, -2.51877345e-02,
           6.85293749e-02, -5.04620709e-02,  2.18394846e-02,
          -7.98307136e-02,  1.57275852e-02]],

        [[-4.24592867e-02, -5.98774999e-02,  6.20517395e-02,
           6.69216141e-02,  1.39782215e-02,  1.39022190e-02,
          -8.40072632e-02, -9.28316936e-02, -3.81821878e-02,
           3.28571610e-02, -9.50155482e-02,  2.06548795e-01,
          -5.07548172e-03, -8.27670172e-02,  1.74515531e-03,
           7.49436691e-02,  5.22173010e-02, -1.54335704e-02,
           6.05179043e-03, -2.36486383e-02,  4.83768359e-02,
          -1.98077876e-02, -5.95450103e-02, -3.22199836e-02,
           1.26348529e-02, -9.37070101e-02, -1.34252422e-02,
           1.63472250e-01,  2.09844895e-02,  4.21280973e-02,
          -1.49431959e-01,  9.01276767e-02]]],


       [[[ 3.45688313e-02,  7.97107220e-02,  3.33844125e-02,
          -1.20801199e-02,  2.63867676e-02,  2.75757816e-02,
          -7.70123675e-02, -3.14535312e-02, -2.06373960e-01,
           1.05471522e-01,  4.95559983e-02, -1.16412774e-01,
          -6.32849857e-02,  2.04185806e-02, -9.10404772e-02,
           8.10925439e-02,  1.11957282e-01, -1.06745861e-01,
          -5.45982271e-02, -2.82833027e-03, -2.66196430e-01,
           1.64179057e-01,  1.23541646e-01, -1.65091053e-01,
           7.54139572e-02,  2.50726908e-01, -5.71109504e-02,
           1.48351267e-01, -3.64737064e-02,  1.51068226e-01,
          -4.74842303e-02,  1.05010234e-01]],

        [[ 4.32509072e-02, -1.08206309e-01, -5.32749258e-02,
           8.09469670e-02, -1.45653322e-01,  1.83570251e-01,
           4.24414650e-02,  1.59827664e-01,  2.48608693e-01,
           7.29027241e-02,  3.15706767e-02, -1.96331352e-01,
          -1.85681745e-01, -1.96331993e-01,  8.14330503e-02,
          -5.10441847e-02,  1.26650885e-01,  3.09712440e-01,
          -4.17465568e-02, -5.22688702e-02,  1.27634957e-01,
           1.45883352e-01,  8.01252499e-02,  3.01035553e-01,
          -4.15359400e-02,  3.02916348e-01, -2.24125832e-02,
          -5.01342490e-02,  1.41805753e-01,  2.88155973e-01,
          -6.53511472e-03,  8.58637840e-02]],

        [[-1.38286784e-01, -4.97966297e-02, -1.29996330e-01,
           4.16094959e-02,  1.82389528e-01, -2.68514678e-02,
           2.51998603e-01, -1.14475004e-01,  2.60868251e-01,
          -3.40055041e-02, -1.70233380e-02, -1.62914380e-01,
          -2.20064163e-01, -2.13668033e-01,  1.09006934e-01,
          -1.12826496e-01, -2.98617661e-01,  2.04329818e-01,
           1.35896914e-02,  6.32194150e-03,  1.94850698e-01,
           5.52918576e-02, -5.35391085e-02,  7.24675506e-02,
          -3.01813986e-02,  1.07929237e-01,  1.62357897e-01,
          -2.60545492e-01, -2.00226828e-02,  2.04357430e-01,
          -9.20708328e-02,  2.05012739e-01]],

        [[-5.31569403e-03,  1.90459922e-01, -1.82926297e-01,
          -1.36217758e-01,  6.76145703e-02, -2.20544443e-01,
          -2.50425451e-02,  1.04731329e-01, -2.30462104e-01,
          -2.48308837e-01,  1.51725367e-01, -2.57288784e-01,
          -8.46081376e-02,  1.50209665e-01,  6.37715831e-02,
           3.92922014e-02,  9.87009183e-02,  1.67022254e-02,
           6.77632987e-02, -1.95640363e-02, -2.88411397e-02,
          -1.35260761e-01, -8.90125260e-02, -1.68812513e-01,
           5.60835637e-02, -1.15505241e-01, -4.61194385e-03,
           1.25047281e-01, -1.84047252e-01,  9.05039459e-02,
           1.28791839e-01,  1.42685696e-01]],

        [[ 1.28499687e-01, -8.88020843e-02, -7.54232705e-02,
          -2.42703990e-03, -1.13100246e-01, -1.24508232e-01,
          -9.55692455e-02,  1.04941204e-01, -1.64641496e-02,
          -2.12375224e-01, -1.05855174e-01, -6.46060929e-02,
           1.16432726e-01,  3.79320197e-02, -7.94462860e-02,
           6.03967756e-02,  2.62370169e-01,  1.01605114e-02,
          -5.71687818e-02,  2.33785566e-02,  1.01750411e-01,
          -9.63896811e-02, -7.12030604e-02,  3.58088724e-02,
          -8.03487822e-02, -8.63136053e-02, -4.95648608e-02,
           7.65802786e-02, -1.39876679e-01, -3.15147713e-02,
          -7.21401721e-03,  1.37567632e-02]]],


       [[[ 8.77455696e-02,  5.97124249e-02, -1.89694241e-01,
           3.23649533e-02, -1.92350626e-01,  1.24370664e-01,
          -1.71399102e-01, -1.43710732e-01, -5.04495315e-02,
          -1.09340347e-01,  2.09975004e-01,  1.20991776e-02,
           4.99377362e-02,  4.60348763e-02, -2.89339144e-02,
           4.50112969e-02, -4.30244394e-03,  1.93148911e-01,
           3.53351355e-01,  4.69863154e-02, -9.06451568e-02,
          -9.48860496e-02,  2.56411016e-01,  1.34009928e-01,
          -1.25456363e-01,  7.16938078e-02, -9.46179405e-02,
           1.81610599e-01,  8.04117322e-02,  2.65546918e-01,
           1.72054153e-02, -5.89300804e-02]],

        [[ 1.96502239e-01, -1.39927343e-01, -1.33911312e-01,
           1.26991317e-01,  1.59659669e-01,  2.67307848e-01,
          -6.45235255e-02,  3.04940462e-01,  2.59633511e-01,
          -1.34681433e-01, -6.99868128e-02,  1.27351627e-01,
           2.96805538e-02,  2.23741010e-01, -2.36314595e-01,
           1.37561960e-02,  1.17661051e-01,  7.28028081e-03,
           3.55358928e-01,  1.30962580e-01,  2.41935819e-01,
           6.30789669e-03, -3.37278545e-02,  2.55262971e-01,
          -3.57140094e-01,  1.59507662e-01, -1.46307647e-01,
           7.23242164e-02,  1.81715831e-01,  1.53966248e-01,
           5.50780967e-02,  2.59742830e-02]],

        [[-4.03862726e-03, -8.40769988e-03,  4.09948453e-02,
          -1.06060386e-01,  2.83326656e-01,  3.28720927e-01,
           2.72140056e-01, -2.37294659e-01, -7.40716830e-02,
          -2.88817793e-01, -6.50088638e-02, -4.09717560e-02,
           2.44265974e-01, -2.33402833e-01, -1.10050410e-01,
           3.10584600e-03, -2.57611006e-01, -4.29998398e-01,
           2.15267524e-01,  1.15541726e-01, -8.29731598e-02,
           1.27682254e-01, -2.33266324e-01, -1.31910518e-01,
          -2.38840252e-01, -1.11221030e-01,  2.74591565e-01,
          -1.67183250e-01,  1.53959781e-01, -2.62739919e-02,
          -5.13264649e-02,  9.53476317e-03]],

        [[-2.91227967e-01,  1.92851841e-01,  6.34998009e-02,
          -4.25702594e-02, -1.84211820e-01,  2.67261922e-01,
           1.99890241e-01, -1.15273908e-01, -2.64678597e-01,
          -1.63764969e-01,  3.91174294e-02, -3.50707620e-02,
           2.68421412e-01, -1.73132405e-01,  2.80285906e-02,
          -2.66031735e-02, -1.72160462e-01, -2.76752502e-01,
           1.25095621e-01,  1.37536764e-01, -1.05138674e-01,
           1.68358728e-01, -1.84328616e-01, -2.26817891e-01,
          -2.03812003e-01, -8.87360126e-02,  4.47154827e-02,
          -5.68596832e-02,  3.41318071e-01, -1.64236873e-01,
           2.29175854e-02,  1.24349169e-01]],

        [[-1.50303990e-01, -1.24774635e-01, -8.22538063e-02,
           5.53404493e-03, -1.11148380e-01, -2.94404346e-02,
          -1.43259659e-03,  1.85924873e-01,  1.08227178e-01,
           1.17234342e-01, -4.91014384e-02, -2.44117349e-01,
           9.34721082e-02,  2.39500254e-01,  2.65940160e-01,
          -1.61937177e-02,  1.31614238e-01, -7.58006722e-02,
           1.05317712e-01, -4.97416444e-02,  1.06382117e-01,
           1.55792519e-01, -6.84151053e-02,  1.45529345e-01,
           2.03677062e-02, -8.97034109e-02, -2.06795484e-01,
           1.54043451e-01,  2.10126981e-01, -8.91636238e-02,
           1.28453285e-01,  9.36990604e-02]]],


       [[[-1.62867770e-01,  1.78554595e-01,  1.07262485e-01,
          -2.01251507e-01, -1.86686456e-01, -1.39267615e-03,
          -2.91764736e-01, -6.71359822e-02,  9.27731097e-02,
          -4.40010466e-02,  1.04116030e-01,  9.00273323e-02,
          -9.75889117e-02, -9.58028287e-02,  1.44259349e-01,
           1.72845557e-01, -7.35422820e-02, -5.54325320e-02,
          -5.81960455e-02, -9.40498710e-03,  2.49349073e-01,
           2.98932809e-02,  1.47391170e-01,  2.23646581e-01,
          -1.66808844e-01, -2.98017204e-01,  1.76479578e-01,
           9.54928771e-02, -1.94601282e-01,  3.18542421e-02,
          -1.29441142e-01, -4.51699793e-02]],

        [[ 1.38353303e-01, -1.54019877e-01,  3.47476661e-01,
           3.10183734e-01,  9.80487838e-02, -8.70270059e-02,
          -3.14944327e-01,  3.03581864e-01,  1.18549809e-01,
           4.82388325e-02, -4.62612629e-01,  1.56953588e-01,
           1.20078214e-01,  2.14997604e-01,  2.20677666e-02,
          -1.07790619e-01, -4.78487350e-02, -1.66399360e-01,
          -3.04439962e-01, -1.03561424e-01,  1.27254218e-01,
           2.68438198e-02, -3.16898555e-01, -1.08468547e-01,
           4.99541610e-02, -3.13619196e-01, -3.84023368e-01,
           1.85616702e-01, -1.11300424e-01, -2.30957553e-01,
          -8.71629417e-02, -2.18425304e-01]],

        [[ 5.16243160e-01, -1.59118176e-01,  2.56632596e-01,
           2.31499642e-01,  3.12861830e-01, -1.93787217e-01,
          -3.51059884e-02, -1.76955208e-01, -1.34864435e-01,
           1.62184462e-01,  1.46136060e-01,  2.63225555e-01,
           1.66042566e-01,  1.86691016e-01, -2.67370224e-01,
          -5.90280676e-03,  2.70183504e-01,  5.89993186e-02,
          -2.57559538e-01, -4.12272394e-01, -8.70700553e-02,
           1.34470537e-01, -1.67689323e-01, -3.41424406e-01,
           2.84074187e-01, -1.07147172e-01,  1.03664622e-01,
          -2.63156950e-01, -3.71470511e-01, -2.90910363e-01,
           1.30605936e-01, -4.24639434e-01]],

        [[ 4.88423556e-02,  3.35189223e-01,  2.53643900e-01,
          -2.06524417e-01, -3.78751934e-01,  1.34045901e-02,
           1.64140776e-01, -1.75927207e-01, -1.66527703e-01,
           3.91573310e-01,  2.06493884e-01,  2.74771124e-01,
           1.69764921e-01, -4.06905115e-01, -3.35692346e-01,
           8.99787545e-02, -5.60279638e-02,  3.24890286e-01,
          -6.52969703e-02, -3.02911997e-01, -1.91059798e-01,
           2.11907119e-01,  6.60512149e-02, -6.08844198e-02,
           1.97099358e-01, -4.50475607e-03,  3.00196797e-01,
          -1.27629504e-01, -2.09806144e-01, -1.09069586e-01,
           1.95026800e-01, -2.41192505e-01]],

        [[-4.21599858e-02, -1.33833423e-01,  1.73788846e-01,
          -8.74219313e-02,  1.79445803e-01,  2.18065772e-02,
           1.11098528e-01,  2.27415338e-01,  1.67024761e-01,
           1.86085999e-01,  6.45482726e-03,  1.15672685e-01,
           9.32103917e-02,  4.86121178e-02, -1.34105667e-01,
           2.61788070e-01, -1.33057624e-01,  2.08185226e-01,
           1.11163380e-02, -1.39638558e-01,  3.82541418e-02,
           2.05305278e-01,  2.74346590e-01,  1.07309587e-01,
           1.53153718e-01, -7.80853480e-02, -1.45936459e-01,
           1.15661368e-01,  2.84138117e-02,  2.60127039e-04,
           2.58475721e-01,  2.48304214e-02]]],


       [[[-9.96622518e-02,  1.37207210e-01,  6.49073049e-02,
          -2.56701887e-01, -1.92326424e-03, -1.43860981e-01,
           1.09956741e-01,  1.07560353e-02,  3.54471877e-02,
          -7.16849370e-03,  2.60522217e-01, -4.55740020e-02,
           6.92887977e-02, -2.51750741e-02,  1.67305902e-01,
           3.12767439e-02, -1.02101907e-01, -1.38990544e-02,
          -5.27952351e-02, -5.41742854e-02,  9.97792631e-02,
           1.18075432e-02, -9.80602354e-02, -7.14249611e-02,
          -2.99854353e-02, -3.63657847e-02,  2.59663343e-01,
           1.45671824e-02,  2.26698264e-01, -4.12496440e-02,
          -1.07028335e-01,  1.12535998e-01]],

        [[-3.71173561e-01, -3.47134918e-01, -1.25734866e-01,
          -1.04274608e-01, -1.17725596e-01, -4.32399511e-02,
          -1.25010118e-01,  1.28871411e-01, -1.16636446e-02,
           3.53644863e-02, -3.43158126e-01, -6.99633360e-02,
          -4.92055565e-02, -2.04492901e-02,  2.45652124e-01,
          -2.31047466e-01, -2.06009001e-01,  3.27508003e-02,
          -1.18006527e-01,  2.33523156e-02,  4.04132828e-02,
          -1.25319809e-01, -2.09883824e-01, -1.37493312e-01,
           2.33293012e-01, -1.44781396e-02, -2.18876049e-01,
           1.32711887e-01,  3.90832350e-02, -2.01453134e-01,
          -4.23350960e-01,  1.45124316e-01]],

        [[-5.36839813e-02,  1.58995256e-01, -1.83390737e-01,
           3.47549677e-01,  1.38828710e-01,  4.24572304e-02,
          -1.27515435e-01, -9.66101289e-02, -1.87338620e-01,
          -1.97422896e-02, -1.08366840e-01, -1.55322403e-01,
          -1.61157817e-01,  2.01927379e-01,  2.53322780e-01,
          -5.62116742e-01, -5.91066889e-02,  5.40728793e-02,
          -7.92302191e-02,  1.88086838e-01,  6.59958273e-03,
          -2.42399514e-01,  1.69646740e-01, -9.47589334e-03,
           1.11526139e-01,  6.88120276e-02, -9.78308246e-02,
          -1.60721883e-01,  6.95262775e-02, -6.26887083e-02,
          -2.41015360e-01, -2.02056006e-01]],

        [[ 1.95601895e-01,  1.46274611e-01, -2.14165911e-01,
           1.88677847e-01, -6.13280199e-03,  3.82143594e-02,
          -2.77706925e-02, -2.12541133e-01,  3.08963843e-02,
          -2.32768711e-02,  4.44432311e-02, -7.64122903e-02,
          -2.42977053e-01,  2.11474299e-02,  3.08647826e-02,
          -2.96457887e-01,  2.82384634e-01, -9.74976271e-02,
          -1.13737859e-01,  3.41809720e-01, -2.39258468e-01,
          -3.60288233e-01,  1.55391827e-01,  1.84775457e-01,
           7.04837516e-02,  1.14113115e-01,  1.53539777e-01,
          -1.83223680e-01,  8.52515474e-02,  4.64094169e-02,
          -2.26785466e-02, -3.02122593e-01]],

        [[ 4.71174717e-02, -1.33249089e-01, -7.07437098e-02,
          -1.64638892e-01,  5.91888167e-02,  6.15437292e-02,
           2.02578202e-01,  3.63162197e-02,  4.95729856e-02,
          -1.76949397e-01,  8.75761509e-02, -2.57797502e-02,
          -2.62792528e-01, -1.61450431e-01, -1.10014781e-01,
          -3.13372947e-02,  9.80259553e-02, -8.82068127e-02,
           4.45931666e-02,  1.94095060e-01, -1.57402858e-01,
          -1.39938518e-01,  2.33131334e-01,  3.16902958e-02,
           2.28416715e-02,  6.12144656e-02, -9.37361270e-03,
           1.28603980e-01, -7.55553991e-02,  1.83364391e-01,
           1.22262053e-01, -4.33576889e-02]]]], dtype=float32), array([ 0.06133625, -0.06743222,  0.12276675,  0.25595677,  0.01088508,
        0.16797405,  0.12758622, -0.3513619 , -0.08484235, -0.0949592 ,
        0.04725884, -0.17716943, -0.05790677,  0.04177701, -0.12494741,
        0.39589486, -0.0406141 , -0.11065134, -0.04198102, -0.10928375,
        0.06801659,  0.09649571,  0.34070975, -0.10038836, -0.04970933,
        0.01176555, -0.18670177, -0.0030286 ,  0.05685186, -0.18897383,
        0.16669714,  0.1053075 ], dtype=float32)]

每一块是32个，一共26块，前面是weight，后面是bias吧…

将training data第一张图丢进第一个卷积层中得到的结果



第32张为什么加载不出来？？？？？
最后一张是原图，可以看出来有些感觉就是变成惊讶或者悲伤的样子的，有些特别处理了眉毛，有些特别处理了眼睛的样子吧..
(纯属口胡，我觉得我该好好看看相关论文了

相关代码

1
2
3
4
5
6
7
8
9
10
11
def show(model, x_train, y_train):
    #print(model.get_layer('conv2d_1').get_weights())
    layer_model=Model(inputs=model.input,outputs=model.layers[0].output)
    t = layer_model.predict(x_train)
    print(t, t.shape)
    
    for i in range(32):
        plt.imshow(t[0, :, :, i])
        plt.axis('off')
        plt.savefig('showpic'+str(i)+'.jpg')
        plt.show()

Related Posts
Have fun.