math - Multilayer Perceptron replaced with Single Layer Perceptron -


i got problem in understending difference between mlp , slp.

i know in first case mlp has more 1 layer (the hidden layers) , neurons got non linear activation function, logistic function (needed gradient descent). have read that:

"if neurons in mlp had linear activation function, mlp replaced single layer of perceptrons, can solve linearly separable problems"

i don't understand why in specific case of xor, not linearly separable, equivalent mlp 2 layer network, every neurons got linear activation function, step function. understand need 2 line separation, in case cannot apply rule of previous statment (the replacement of mlp slp).

mlp xor:

http://s17.postimg.org/c7hwv0s8f/xor.png

in linked image neurons b , c have linear activation function (like step function)

xor: http://s17.postimg.org/n77pkd81b/xor1.png

a linear function f(x) = x + b. if take linear function g(z) = c z + d, , apply g(f(x)) (which equivalent of feeding output of 1 linear layer input next linear layer) g(f(x)) = c (a x + b) + d = ac x + cb + d = (ac) x + (cb + d) in linear function.

the step function not linear function - cannot write a x + b. that's why mlp using step function strictly more expressive single layer perceptron using step function.


Comments

Popular posts from this blog

angularjs - ADAL JS Angular- WebAPI add a new role claim to the token -

php - CakePHP HttpSockets send array of paramms -

node.js - Using Node without global install -