machine learning - Low accuracy with change to TensorFlow Cifar10 example -

i trying modify network structure provided cifar10 in tensorflow. typically, added convolution layer (conv12) after first convolution layer (conv1). no matter how set filter (i tried 1x1, 3x3, 5x5) , whether using weight decay or not, having new layer decrease accuracy below 10%. equivalent random guess in cifar10 since there 10 classes.

the code structure following, don't modify other part of cifar except setting size of input image 48x48 (instead of 24x24). guess input size should not matter.

note conv12 depthwise convolution layer because want add linear layer after conv1 layer in order minimize change original code. doing expected accuracy should similar original version, decreases around 10%. (i tried normal convolution layer didn't work also.)

  tf.variable_scope('conv1') scope:     kernel1 = _variable_with_weight_decay('weights', shape=[5, 5, 3, 64],                                          stddev=1e-4, wd=0.0)     conv_1 = tf.nn.conv2d(images, kernel1, [1, 1, 1, 1], padding='same')     biases1 = _variable_on_cpu('biases', [64], tf.constant_initializer(0.0))     bias1 = tf.nn.bias_add(conv_1, biases1)     conv1 = tf.nn.relu(bias1, name=scope.name)     _activation_summary(conv1)     tf.variable_scope('conv12') scope:     kernel12 = _variable_with_weight_decay('weights', shape=[1, 1, 64, 1],                                          stddev=1e-4, wd=0.0)     #conv_12 = tf.nn.conv2d(conv1, kernel12, [1, 1, 1, 1], padding='same')     conv_12 = tf.nn.depthwise_conv2d(conv1, kernel12, [1, 1, 1, 1], padding='same')     biases12 = _variable_on_cpu('biases', [64], tf.constant_initializer(0.0))     bias12 = tf.nn.bias_add(conv_12, biases12)             conv12 = tf.nn.relu(bias12)     _activation_summary(conv12)    pool1 = tf.nn.max_pool(conv12, ksize=[1, 3, 3, 1], strides=[1, 2, 2, 1],                      padding='same', name='pool1') ..... 

could please tell me wrong code?

your second convolution:

kernel12 = _variable_with_weight_decay('weights', shape=[1, 1, 64, 1] 

is taking depth-64 output of previous layer , squeezing down depth-1 output. doesn't seem match whichever code have following (if it's conv2 cifar example tensorflow, isn't going work well, because 1 expects depth-64 input.)

perhaps wanted shape=[1, 1, 64, 64], add "inception-style" 1x1 convolutional layer model?