Vfrdtyky

I have a multi-output problem (multi-label, multi-classification). In brief, the problem regards instrument recognition in polyphonic music, therefore my model needs to be able to predict the instruments (which can be multiple) in a song.

I need to use a CNN for this. The first block of the network model has been omitted since it is composed only by the feature extractor, with all the convolution, pooling and dropout. I am using the high level API of Tensorflow, Estimators.

    # first part omitted, this is last dropout from a fully connected layer

    dropout = tf.layers.dropout(inputs=dense, rate=0.5, training=mode == tf.estimator.ModeKeys.TRAIN)



    print('Shape Dropout', dropout.shape)



    ###########################################################################



    # Logits Layer

    logits = tf.layers.dense(inputs=dropout, units=labels.shape[1])



    print('Shape Logits:', logits.shape)



    predictions = tf.round(tf.sigmoid(logits))



    if mode == tf.estimator.ModeKeys.PREDICT:

        return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions)





    loss = tf.nn.weighted_cross_entropy_with_logits(targets=tf.cast(labels, tf.float32), logits=logits, pos_weight=3, name=None)



    loss = tf.reduce_mean(tf.reduce_sum(loss, axis=1))



    accuracy = tf.reduce_mean(tf.cast(tf.equal(predictions, tf.round(tf.cast(labels, tf.float32))), tf.float32))



    # calculate f1_score, precision and recall for multilabel problem

    f1s = [0, 0, 0]



    labels = tf.cast(labels, tf.float64)

    predictions = tf.cast(predictions, tf.float64)



    for i, axis in enumerate([None, 0]):

        TP = tf.count_nonzero(predictions * labels, axis=axis)

        FP = tf.count_nonzero(predictions * (labels - 1), axis=axis)

        FN = tf.count_nonzero((predictions - 1) * labels, axis=axis)



        precision = tf.truediv(TP, (TP + FP))

        recall = tf.truediv(TP, (TP + FN))

        f1 = 2 * precision * recall / (precision + recall)



        f1s[i] = tf.reduce_mean(f1)



    weights = tf.reduce_sum(labels, axis=0)

    weights /= tf.reduce_sum(weights)



    f1s[2] = tf.reduce_sum(tf.cast(f1, tf.float64) * weights)



    micro, macro, weighted = f1s





    tf.summary.scalar("micro_mLabels", micro)

    tf.summary.scalar("macro_mLabels", macro)

    tf.summary.scalar("weighted_mLabels", weighted)





    tf.summary.scalar("precision_sLabel", tf.metrics.precision(labels=labels, predictions=tf.cast(predictions, tf.int32)))

    tf.summary.scalar("recall_sLabel",tf.metrics.recall(labels=labels, predictions=tf.cast(predictions, tf.int32)))





    if mode == tf.estimator.ModeKeys.TRAIN:

        optimizer = tf.train.AdamOptimizer(learning_rate=0.001)

        train_op = optimizer.minimize(

                                  loss=loss,

                                  global_step=tf.train.get_global_step())

        logging_hook = tf.train.LoggingTensorHook({"loss": loss, "accuracy": accuracy}, every_n_iter=10)

        return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op, training_hooks = [logging_hook])







    eval_metric_ops = {

        "accuracy_sLabel": tf.metrics.accuracy(

        labels=labels,

        predictions=predictions),

        "precision_sLabel": tf.metrics.precision(

        labels=labels,

        predictions=tf.cast(predictions, tf.int32)),

        "recall_sLabel": tf.metrics.recall(

        labels=labels,

        predictions=tf.cast(predictions, tf.int32)),

    }



    return tf.estimator.EstimatorSpec(mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)

I recalculate precision and recall internally to check whether was a Tensorflow bug.

Basically, the problem I am having is that the accuracy is really high from the beginning, like in 20 iterations is already at 90% (very weird, should take few epochs), precision and recall differently sometimes are 0 and sometimes not, depending from what hyperparameters I set.

I understand I need to follow a Sigmoid approach for multi-label but I think I am doing things in a wrong way.

What I believe is that the way in which I store the predictions from the logits and in how I calculate the loss is wrong. However, I am not quite sure about the rest of the code, that's why I added the full block.

Really appreciate some help,

Thank you in advance.