Python-Theano Tutorials: Bare bones introduction to machine learning from linear regression to convolutional neural networks using Theano.

Theano-Tutorials

Bare bones introduction to machine learning from linear regression to convolutional neural networks using Theano.

Dataset It's worth noting that this library assumes that the reader has access to the mnist dataset. This dataset is freely available and is accessible through Yann LeCun's personal website.

If you want to automate the download of the dataset, there is an included file that will do this for you. Simply run the following: sudo ./download_mnist.sh

Known Issues Library not loaded: /usr/local/opt/openssl/lib/libssl.1.0.0.dylib This results from a broken openssl installation on mac. It can be fixed by uninstalling and reinstalling openssl: sudo brew remove openssl brew install openssl

Comments

  • Add missing w_o as model argument
    Add missing w_o as model argument

    Nov 6, 2015

                                                                                                                                                                                                           
    Reply
  • Is the code in 2_logistic_regression.py for softmax regression or logistic regression?
    Is the code in 2_logistic_regression.py for softmax regression or logistic regression?

    Nov 20, 2015

    I don't see any logistic function code in 2_logistic_regression.py, is this really code for logistic regression?

    Reply
  • remove two lines
    remove two lines

    Dec 6, 2015

    you import something has not used.

    Reply
  • load folder
    load folder

    Dec 8, 2015

    Hello, Im new in learning and in theano. So this tutorial is perfect and it helps me a lot. Please, how can i get the fonction mnist(), and the load folder.

    Thank you a lot

    Reply
  • Can't run final version of code
    Can't run final version of code

    Dec 23, 2015

    Error occur when running final version of code.

    python 5_convolutional_net.py
    

    Traceback

    Traceback (most recent call last):
      File "5_convolutional_net.py", line 78, in <module>
        noise_l1, noise_l2, noise_l3, noise_l4, noise_py_x = model(X, w, w2, w3, w4, 0.2, 0.5)
      File "5_convolutional_net.py", line 44, in model
        r = conv2d(X, w, border_mode='full')
      File "/home/fuxi/.virtualenvs/deep/local/lib/python2.7/site-packages/theano/tensor/nnet/conv.py", line 151, in conv2d
        return op(input, filters)
      File "/home/fuxi/.virtualenvs/deep/local/lib/python2.7/site-packages/theano/gof/op.py", line 507, in __call__
        node = self.make_node(*inputs, **kwargs)
      File "/home/fuxi/.virtualenvs/deep/local/lib/python2.7/site-packages/theano/tensor/nnet/conv.py", line 628, in make_node
        "inputs(%s), kerns(%s)" % (_inputs.dtype, _kerns.dtype))
    NotImplementedError: The image and the kernel must have the same type.inputs(float32), kerns(float64)
    
    Reply
  • use theano.nnet.conv2d instead of theano.nnet.conv.conv2d
    use theano.nnet.conv2d instead of theano.nnet.conv.conv2d

    Feb 19, 2016

    Had a friend play with the tutorial and ran into the issue described here: https://github.com/Theano/Theano/issues/3992

    It might be friendlier for new people who want to play around with the code use theano.nnet.conv2d. If there's interest , I'd happily make a PR (though feel free to make the one line change without me) if you feel like it's the right thing to do.

    Reply
  • 128 examples on 2_logistic_regression.py
    128 examples on 2_logistic_regression.py

    Apr 5, 2016

    Hi Alec, would you be kind to explain how you reached 128 examples for mini-batch training?

    Reply
  • Any documentation
    Any documentation

    Jun 8, 2016

    Thanks for putting up the code here! It would be great if you have some documentation to describe the code, or even some inline comments would be helpful too.

    Reply
  • How to load mnist?
    How to load mnist?

    Aug 14, 2016

    In your logistic regression ,from load import mnist?

    Reply
  • Weights are not updated in 3_net.py
    Weights are not updated in 3_net.py

    Oct 12, 2016

    In file 3_net.py, everything seems to go fine but the problem is, the weights w_h are not getting updated. They have the same random values which were initialized at the beginning in all iterations of training.

    In the first iteration, (only a subset of w_h and w_o values are printed )

    Iteration: 1 Cost: 2.29556935362 w_h: [ 0.026779 -0.00800639 -0.01096162 -0.00354766 0.00137482] w_o: [-0.01249968 -0.00815679 0.01787239 -0.00080373 -0.00373115]

    and in the 25th iteration Iteration: 25 Cost: 2.24883255277 w_h: [ 0.026779 -0.00800639 -0.01096162 -0.00354766 0.00137482] w_o: [-0.0131051 -0.00619329 0.01693202 -0.00053586 -0.00538656]

    This was not even the problem of sigmoid activation unit due to vanishing gradients because same problem was faced even when tanh and ReLU activation units were used.

    Reply
  • save weights
    save weights

    Aug 11, 2016

    hey, how save weights in 5 example (cnn)?

    Reply
  • Issue regarding pickling a convolutional neural network
    Issue regarding pickling a convolutional neural network

    Mar 30, 2016

    I recently ran into an issue while working on training a convolutional neural network for a CIFAR-10 dataset using theano . The whole code is running well except for pickling the best cnn model. I am attaching a document of the code and also the screenshot of the error I am getting. Thank you. covolutional (1).txt logistic_sgd.txt capture

    Reply
  • with dropout, accuracy is lower than without dropout
    with dropout, accuracy is lower than without dropout

    Jul 13, 2015

    I run 5_convolutional_net.py with two case(drop-out, no drop-out) when I run conv_net with drop-out, I get the low accuracy score In order to enable drop out, at the code line below l1, l2, l3, l4, py_x = model(X, w, w2, w3, w4, 0., 0.) replace 0., 0. with 0.5, 0.5

    The reult is below.

    | Epoch | Drop out | NO Drop out | | --- | --- | --- | | 1 | 0.8646 | 0.8984 | | 2 | 0.9007 | 0.9727 | | 3 | 0.9398 | 0.9838 | | 3 | 0.9409 | 0.9878 | | 4 | 0.9436 | 0.9884 | | 5 | 0.9501 | 0.9893 | | 6 | 0.9473 | 0.9903 | | 7 | 0.9525 | 0.9914 |

    with other data (cifar-10), result is same. Can you say me why the accuracy with drop-out is low??

    Reply
  • Added ability to download mnist and updated README accordingly
    Added ability to download mnist and updated README accordingly

    Jan 23, 2015

    In response to the comment on the youtube video by brhgvishal.

    This should give people a more straightforward option to access the needed data.

    @Newmu

    Reply
  • Is the RMSprop func right?
    Is the RMSprop func right?

    Jun 14, 2015

    The function create a new shared variable named 'acc' when it is called. Will the values be accumulated in 'acc'?

    Reply
  • Convnet giving error in line 44?
    Convnet giving error in line 44?

    Jul 4, 2015

    Hi Newmu,

    First off, thanks for putting up this tutorials, they are really nice for first time learner. I am not sure about the others but I am running the convnet example on my Mac and it gives error: NotImplementedError: The image and the kernel must have the same type.inputs(float32), kerns(float64)

    I am not sure why that is the case. Any help would be appreciated, thanks!

    Reply