softmax_loss 함수 : 루프를 행렬로 변환

Question

이제 스탠포드 cs231n 과정을 배우고 있습니다. softmax_loss 함수를 완료 할 때 전체 벡터화 된 형식으로 쓰는 것이 쉽지 않음을 발견했습니다. 특히 dw 용어를 다루는 것이 쉽지 않습니다. 아래는 제 코드입니다. 누군가 코드를 최적화 할 수 있습니까? 감사하겠습니다.

def softmax_loss_vectorized(W, X, y, reg): 

    loss = 0.0 
    dW = np.zeros_like(W) 


    num_train = X.shape[0] 
    num_classes = W.shape[1] 

    scores = X.dot(W) 
    scores -= np.max(scores, axis = 1)[:, np.newaxis] 
    exp_scores = np.exp(scores) 
    sum_exp_scores = np.sum(exp_scores, axis = 1) 
    correct_class_score = scores[range(num_train), y] 

    loss = np.sum(np.log(sum_exp_scores)) - np.sum(correct_class_score) 

    exp_scores = exp_scores/sum_exp_scores[:,np.newaxis] 

    # **maybe here can be rewroten into matrix operations** 
    for i in xrange(num_train): 
    dW += exp_scores[i] * X[i][:,np.newaxis] 
    dW[:, y[i]] -= X[i] 

    loss /= num_train 
    loss += 0.5 * reg * np.sum(W*W) 
    dW /= num_train 
    dW += reg * W 


    return loss, dW 

Answer 1

다음은 벡터화 된 구현 예입니다. 그러나 나는 당신이 조금 더 많은 시간을 보내고 스스로 해결책을 찾으려고 노력할 것을 제안합니다. 모든 softmax 값을 가진 행렬을 만들고 올바른 원소에서 -1을 뺀다.

def softmax_loss_vectorized(W, X, y, reg): 
    num_train = X.shape[0] 

    scores = X.dot(W) 
    scores -= np.max(scores) 
    correct_scores = scores[np.arange(num_train), y] 

    # Compute the softmax per correct scores in bulk, and sum over its logs. 
    exponents = np.exp(scores) 
    sums_per_row = np.sum(exponents, axis=1) 
    softmax_array = np.exp(correct_scores)/sums_per_row 
    information_array = -np.log(softmax_array) 
    loss = np.mean(information_array) 

    # Compute the softmax per whole scores matrix, which gives the matrix for X rows coefficients. 
    # Their linear combination is algebraically dot product X transpose. 
    all_softmax_matrix = (exponents.T/sums_per_row).T 
    grad_coeff = np.zeros_like(scores) 
    grad_coeff[np.arange(num_train), y] = -1 
    grad_coeff += all_softmax_matrix 
    dW = np.dot(X.T, grad_coeff)/num_train 

    # Regularization 
    loss += 0.5 * reg * np.sum(W * W) 
    dW += reg * W 

    return loss, dW