math - Multilayer Perceptron replaced with Single Layer Perceptron -
i got problem in understending difference between mlp , slp.
i know in first case mlp has more 1 layer (the hidden layers) , neurons got non linear activation function, logistic function (needed gradient descent). have read that:
"if neurons in mlp had linear activation function, mlp replaced single layer of perceptrons, can solve linearly separable problems"
i don't understand why in specific case of xor, not linearly separable, equivalent mlp 2 layer network, every neurons got linear activation function, step function. understand need 2 line separation, in case cannot apply rule of previous statment (the replacement of mlp slp).
mlp xor:
http://s17.postimg.org/c7hwv0s8f/xor.png
in linked image neurons b , c have linear activation function (like step function)
a linear function f(x) = x + b
. if take linear function g(z) = c z + d
, , apply g(f(x)) (which equivalent of feeding output of 1 linear layer input next linear layer) g(f(x)) = c (a x + b) + d = ac x + cb + d = (ac) x + (cb + d)
in linear function.
the step function not linear function - cannot write a x + b
. that's why mlp using step function strictly more expressive single layer perceptron using step function.
Comments
Post a Comment