Python-Neural style: Torch implementation of neural style algorithm

neural-style

This is a torch implementation of the paper A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge.

The paper presents an algorithm for combining the content of one image with the style of another image using convolutional neural networks. Here's an example that maps the artistic style of The Starry Night onto a night-time photograph of the Stanford campus:

Applying the style of different images to the same content image gives interesting results. Here we reproduce Figure 2 from the paper, which renders a photograph of the Tubingen in Germany in a variety of styles:

Here are the results of applying the style of various pieces of artwork to this photograph of the golden gate bridge:

Content / Style Tradeoff

The algorithm allows the user to trade-off the relative weight of the style and content reconstruction terms, as shown in this example where we port the style of Picasso's 1907 self-portrait onto Brad Pitt:

Style Scale

By resizing the style image before extracting style features, we can control the types of artistic features that are transfered from the style image; you can control this behavior with the -style_scale flag. Below we see three examples of rendering the Golden Gate Bridge in the style of The Starry Night. From left to right, -style_scale is 2.0, 1.0, and 0.5.

Multiple Style Images

You can use more than one style image to blend multiple artistic styles.

Clockwise from upper left: "The Starry Night" + "The Scream", "The Scream" + "Composition VII", "Seated Nude" + "Composition VII", and "Seated Nude" + "The Starry Night"

Style Interpolation

When using multiple style images, you can control the degree to which they are blended:

Transfer style but not color

If you add the flag -original_colors 1 then the output image will retain the colors of the original image; this is similar to the recent blog post by deepart.io.

Setup:

Dependencies:

Optional dependencies:

After installing dependencies, you'll need to run the following script to download the VGG model:

sh models/download_models.sh

This will download the original VGG-19 model. Leon Gatys has graciously provided the modified version of the VGG-19 model that was used in their paper; this will also be downloaded. By default the original VGG-19 model is used.

If you have a smaller memory GPU then using NIN Imagenet model will be better and gives slightly worse yet comparable results. You can get the details on the model from BVLC Caffe ModelZoo and can download the files from NIN-Imagenet Download Link

You can find detailed installation instructions for Ubuntu in the installation guide.

Usage

Basic usage:

th neural_style.lua -style_image <image.jpg> -content_image <image.jpg>

OpenCL usage with NIN Model (This requires you download the NIN Imagenet model files as described above):

th neural_style.lua -style_image examples/inputs/picasso_selfport1907.jpg -content_image examples/inputs/brad_pitt.jpg -output_image profile.png -model_file models/nin_imagenet_conv.caffemodel -proto_file models/train_val.prototxt -gpu 0 -backend clnn -num_iterations 1000 -seed 123 -content_layers relu0,relu3,relu7,relu12 -style_layers relu0,relu3,relu7,relu12 -content_weight 10 -style_weight 1000 -image_size 512 -optimizer adam

OpenCL NIN Model Picasso Brad Pitt

To use multiple style images, pass a comma-separated list like this:

-style_image starry_night.jpg,the_scream.jpg.

Note that paths to images should not contain the ~ character to represent your home directory; you should instead use a relative path or a full absolute path.

Options:

  • -image_size: Maximum side length (in pixels) of of the generated image. Default is 512.
  • -style_blend_weights: The weight for blending the style of multiple style images, as a comma-separated list, such as -style_blend_weights 3,7. By default all style images are equally weighted.
  • -gpu: Zero-indexed ID of the GPU to use; for CPU mode set -gpu to -1.

Optimization options:

  • -content_weight: How much to weight the content reconstruction term. Default is 5e0.
  • -style_weight: How much to weight the style reconstruction term. Default is 1e2.
  • -tv_weight: Weight of total-variation (TV) regularization; this helps to smooth the image. Default is 1e-3. Set to 0 to disable TV regularization.
  • -num_iterations: Default is 1000.
  • -init: Method for generating the generated image; one of random or image. Default is random which uses a noise initialization as in the paper; image initializes with the content image.
  • -optimizer: The optimization algorithm to use; either lbfgs or adam; default is lbfgs. L-BFGS tends to give better results, but uses more memory. Switching to ADAM will reduce memory usage; when using ADAM you will probably need to play with other parameters to get good results, especially the style weight, content weight, and learning rate; you may also want to normalize gradients when using ADAM.
  • -learning_rate: Learning rate to use with the ADAM optimizer. Default is 1e1.
  • -normalize_gradients: If this flag is present, style and content gradients from each layer will be L1 normalized. Idea from andersbll/neural_artistic_style.

Output options:

  • -output_image: Name of the output image. Default is out.png.
  • -print_iter: Print progress every print_iter iterations. Set to 0 to disable printing.
  • -save_iter: Save the image every save_iter iterations. Set to 0 to disable saving intermediate results.

Layer options:

  • -content_layers: Comma-separated list of layer names to use for content reconstruction. Default is relu4_2.
  • -style_layers: Comma-separated list of layer names to use for style reconstruction. Default is relu1_1,relu2_1,relu3_1,relu4_1,relu5_1.

Other options:

  • -style_scale: Scale at which to extract features from the style image. Default is 1.0.
  • -original_colors: If you set this to 1, then the output image will keep the colors of the content image.
  • -proto_file: Path to the deploy.txt file for the VGG Caffe model.
  • -model_file: Path to the .caffemodel file for the VGG Caffe model. Default is the original VGG-19 model; you can also try the normalized VGG-19 model used in the paper.
  • -pooling: The type of pooling layers to use; one of max or avg. Default is max. The VGG-19 models uses max pooling layers, but the paper mentions that replacing these layers with average pooling layers can improve the results. I haven't been able to get good results using average pooling, but the option is here.
  • -backend: nn, cudnn, or clnn. Default is nn. cudnn requires cudnn.torch and may reduce memory usage. clnn requires cltorch and clnn
  • -cudnn_autotune: When using the cuDNN backend, pass this flag to use the built-in cuDNN autotuner to select the best convolution algorithms for your architecture. This will make the first iteration a bit slower and can take a bit more memory, but may significantly speed up the cuDNN backend.

Frequently Asked Questions

Problem: Generated image has saturation artifacts:

Solution: Update the image packge to the latest version: luarocks install image

Problem: Running without a GPU gives an error message complaining about cutorch not found

Solution: Pass the flag -gpu -1 when running in CPU-only mode

Problem: The program runs out of memory and dies

Solution: Try reducing the image size: -image_size 256 (or lower). Note that different image sizes will likely require non-default values for -style_weight and -content_weight for optimal results. If you are running on a GPU, you can also try running with -backend cudnn to reduce memory usage.

Problem: Get the following error message:

models/VGG_ILSVRC_19_layers_deploy.prototxt.cpu.lua:7: attempt to call method 'ceil' (a nil value)

Solution: Update nn package to the latest version: luarocks install nn

Problem: Get an error message complaining about paths.extname

Solution: Update torch.paths package to the latest version: luarocks install paths

Problem: NIN Imagenet model is not giving good results.

Solution: Make sure the correct -proto_file is selected. Also make sure the correct parameters for -content_layers and -style_layers are set. (See OpenCL usage example above.)

Problem: -backend cudnn is slower than default NN backend

Solution: Add the flag -cudnn_autotune; this will use the built-in cuDNN autotuner to select the best convolution algorithms.

Memory Usage

By default, neural-style uses the nn backend for convolutions and L-BFGS for optimization. These give good results, but can both use a lot of memory. You can reduce memory usage with the following:

  • Use cuDNN: Add the flag -backend cudnn to use the cuDNN backend. This will only work in GPU mode.
  • Use ADAM: Add the flag -optimizer adam to use ADAM instead of L-BFGS. This should significantly reduce memory usage, but may require tuning of other parameters for good results; in particular you should play with the learning rate, content weight, style weight, and also consider using gradient normalization. This should work in both CPU and GPU modes.
  • Reduce image size: If the above tricks are not enough, you can reduce the size of the generated image; pass the flag -image_size 256 to generate an image at half the default size.

With the default settings, neural-style uses about 3.5GB of GPU memory on my system; switching to ADAM and cuDNN reduces the GPU memory footprint to about 1GB.

Speed

Speed can vary a lot depending on the backend and the optimizer. Here are some times for running 500 iterations with -image_size=512 on a Maxwell Titan X with different settings:

  • -backend nn -optimizer lbfgs: 62 seconds
  • -backend nn -optimizer adam: 49 seconds
  • -backend cudnn -optimizer lbfgs: 79 seconds
  • -backend cudnn -cudnn_autotune -optimizer lbfgs: 58 seconds
  • -backend cudnn -cudnn_autotune -optimizer adam: 44 seconds
  • -backend clnn -optimizer lbfgs: 169 seconds
  • -backend clnn -optimizer adam: 106 seconds

Here are the same benchmarks on a Pascal Titan X with cuDNN 5.0 on CUDA 8.0 RC:

  • -backend nn -optimizer lbfgs: 43 seconds
  • -backend nn -optimizer adam: 36 seconds
  • -backend cudnn -optimizer lbfgs: 45 seconds
  • -backend cudnn -cudnn_autotune -optimizer lbfgs: 30 seconds
  • -backend cudnn -cudnn_autotune -optimizer adam: 22 seconds

Multi-GPU scaling

You can use multiple GPUs to process images at higher resolutions; different layers of the network will be computed on different GPUs. You can control which GPUs are used with the -gpu flag, and you can control how to split layers across GPUs using the -multigpu_strategy flag.

For example in a server with four GPUs, you can give the flag -gpu 0,1,2,3 to process on GPUs 0, 1, 2, and 3 in that order; by also giving the flag -multigpu_strategy 3,6,12 you indicate that the first two layers should be computed on GPU 0, layers 3 to 5 should be computed on GPU 1, layers 6 to 11 should be computed on GPU 2, and the remaining layers should be computed on GPU 3. You will need to tune the -multigpu_strategy for your setup in order to achieve maximal resolution.

We can achieve very high quality results at high resolution by combining multi-GPU processing with multiscale generation as described in the paper Controlling Perceptual Factors in Neural Style Transfer by Leon A. Gatys, Alexander S. Ecker, Matthias Bethge, Aaron Hertzmann and Eli Shechtman.

Here is a 3620 x 1905 image generated on a server with four Pascal Titan X GPUs:

The script used to generate this image can be found here.

Implementation details

Images are initialized with white noise and optimized using L-BFGS.

We perform style reconstructions using the conv1_1, conv2_1, conv3_1, conv4_1, and conv5_1 layers and content reconstructions using the conv4_2 layer. As in the paper, the five style reconstruction losses have equal weights.

Citation

If you find this code useful for your research, please cite:

@misc{Johnson2015,
  author = {Johnson, Justin},
  title = {neural-style},
  year = {2015},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/jcjohnson/neural-style}},
}

Comments

  • The NIN model download links have been taken down/changed.
    The NIN model download links have been taken down/changed.

    Sep 15, 2019

    The Google drive link is down, but the files can still be downloaded from their new locations here:

    https://gist.github.com/mavenlin/d802a5849de39225bcc6

    Reply
  • Training loss coefficients
    Training loss coefficients

    Oct 29, 2019

    Instead of taking pre-defined style loss coefficients (0.2/layer, I think), did you try learning these coefficients using a different network? There should be some optimal values for them I believe

    Reply
  • Styling a Transparent Image
    Styling a Transparent Image

    Dec 1, 2019

    Is anyone familiar with a way to style a transparent image? I've tried passing the .png content and doing a .png output, but the transparent background gets styled.

    Reply
  • Coremltools issue
    Coremltools issue

    Mar 10, 2020

    I created .py file according to Apple instruction Apple Instuction and Documentation

    import coremltools
    
    # Convert a Caffe model to a classifier in Core ML
    coreml_model = coremltools.converters.caffe.convert(
        ('nin_imagenet_conv.caffemodel', 'train_val.prototxt'), predicted_feature_name='class_labels.txt'
    )
    
    # Now save the model
    coreml_model.save('nin_imagenet_conv.mlmodel')
    

    Then in Terminal I run python convert.py

    But then I get a error:

    ================= Starting Conversion from Caffe to CoreML ======================
    Layer 0: Type: 'Data', Name: 'data'. Output(s): 'data', 'label'.
    WARNING: Skipping Data Layer 'data' of type 'Data'. It is recommended to use Input layer for deployment.
    Layer 1: Type: 'Data', Name: 'data'. Output(s): 'data', 'label'.
    WARNING: Skipping Data Layer 'data' of type 'Data'. It is recommended to use Input layer for deployment.
    Layer 2: Type: 'Convolution', Name: 'conv1'. Input(s): 'data'. Output(s): 'conv1'.
    Layer 3: Type: 'ReLU', Name: 'relu0'. Input(s): 'conv1'. Output(s): 'conv1'.
    Layer 4: Type: 'Convolution', Name: 'cccp1'. Input(s): 'conv1'. Output(s): 'cccp1'.
    Layer 5: Type: 'ReLU', Name: 'relu1'. Input(s): 'cccp1'. Output(s): 'cccp1'.
    Layer 6: Type: 'Convolution', Name: 'cccp2'. Input(s): 'cccp1'. Output(s): 'cccp2'.
    Layer 7: Type: 'ReLU', Name: 'relu2'. Input(s): 'cccp2'. Output(s): 'cccp2'.
    Layer 8: Type: 'Pooling', Name: 'pool0'. Input(s): 'cccp2'. Output(s): 'pool0'.
    Layer 9: Type: 'Convolution', Name: 'conv2'. Input(s): 'pool0'. Output(s): 'conv2'.
    Layer 10: Type: 'ReLU', Name: 'relu3'. Input(s): 'conv2'. Output(s): 'conv2'.
    Layer 11: Type: 'Convolution', Name: 'cccp3'. Input(s): 'conv2'. Output(s): 'cccp3'.
    Layer 12: Type: 'ReLU', Name: 'relu5'. Input(s): 'cccp3'. Output(s): 'cccp3'.
    Layer 13: Type: 'Convolution', Name: 'cccp4'. Input(s): 'cccp3'. Output(s): 'cccp4'.
    Layer 14: Type: 'ReLU', Name: 'relu6'. Input(s): 'cccp4'. Output(s): 'cccp4'.
    Layer 15: Type: 'Pooling', Name: 'pool2'. Input(s): 'cccp4'. Output(s): 'pool2'.
    Layer 16: Type: 'Convolution', Name: 'conv3'. Input(s): 'pool2'. Output(s): 'conv3'.
    Layer 17: Type: 'ReLU', Name: 'relu7'. Input(s): 'conv3'. Output(s): 'conv3'.
    Layer 18: Type: 'Convolution', Name: 'cccp5'. Input(s): 'conv3'. Output(s): 'cccp5'.
    Layer 19: Type: 'ReLU', Name: 'relu8'. Input(s): 'cccp5'. Output(s): 'cccp5'.
    Layer 20: Type: 'Convolution', Name: 'cccp6'. Input(s): 'cccp5'. Output(s): 'cccp6'.
    Layer 21: Type: 'ReLU', Name: 'relu9'. Input(s): 'cccp6'. Output(s): 'cccp6'.
    Layer 22: Type: 'Pooling', Name: 'pool3'. Input(s): 'cccp6'. Output(s): 'pool3'.
    Layer 23: Type: 'Dropout', Name: 'drop'. Input(s): 'pool3'. Output(s): 'pool3'.
    WARNING: Skipping training related layer 'drop' of type 'Dropout'.
    Layer 24: Type: 'Convolution', Name: 'conv4-1024'. Input(s): 'pool3'. Output(s): 'conv4'.
    Layer 25: Type: 'ReLU', Name: 'relu10'. Input(s): 'conv4'. Output(s): 'conv4'.
    Layer 26: Type: 'Convolution', Name: 'cccp7-1024'. Input(s): 'conv4'. Output(s): 'cccp7'.
    Layer 27: Type: 'ReLU', Name: 'relu11'. Input(s): 'cccp7'. Output(s): 'cccp7'.
    Layer 28: Type: 'Convolution', Name: 'cccp8-1024'. Input(s): 'cccp7'. Output(s): 'cccp8'.
    Layer 29: Type: 'ReLU', Name: 'relu12'. Input(s): 'cccp8'. Output(s): 'cccp8'.
    Layer 30: Type: 'Pooling', Name: 'pool4'. Input(s): 'cccp8'. Output(s): 'pool4'.
    Layer 31: Type: 'Accuracy', Name: 'accuracy'. Input(s): 'pool4', 'label'. Output(s): 'accuracy'.
    WARNING: Skipping training related layer 'accuracy' of type 'Accuracy'.
    Layer 32: Type: 'SoftmaxWithLoss', Name: 'loss'. Input(s): 'pool4', 'label'. WARNING: Skipping training related layer 'loss' of type 'SoftmaxWithLoss'.
    
    ================= Summary of the conversion: ===================================
    Traceback (most recent call last):
      File "convert.py", line 5, in <module>
        ('nin_imagenet_conv.caffemodel', 'train_val.prototxt'), predicted_feature_name='class_labels.txt'
      File "/Users/pavel.tarasevich/Library/Python/2.7/lib/python/site-packages/coremltools/converters/caffe/_caffe_converter.py", line 192, in convert
        predicted_feature_name)
      File "/Users/pavel.tarasevich/Library/Python/2.7/lib/python/site-packages/coremltools/converters/caffe/_caffe_converter.py", line 260, in _export
        predicted_feature_name)
    RuntimeError: Unable to infer input name and dimensions. Please provide a .prototxt file with 'Input' layer and dimensions defined.
    
    Reply
  • issues with cltorch backend
    issues with cltorch backend

    Apr 28, 2020

    I have an AMD gpu so cuda is off the table. I have cltorch installed, however when i try to run neural-style.lua in torch I get this error:

    (base) [email protected]:~/torch/install/neural-style# th neural_style.lua /root/torch-cl/install/bin/luajit: /root/torch-cl/install/share/lua/5.1/trepl/init.lua:384: module 'cutorch' not found:No LuaRocks module found for cutorch no field package.preload['cutorch'] no file '/root/.luarocks/share/lua/5.1/cutorch.lua' no file '/root/.luarocks/share/lua/5.1/cutorch/init.lua' no file '/root/torch-cl/install/share/lua/5.1/cutorch.lua' no file '/root/torch-cl/install/share/lua/5.1/cutorch/init.lua' no file '/root/torch/install/share/lua/5.1/cutorch.lua' no file '/root/torch/install/share/lua/5.1/cutorch/init.lua' no file './cutorch.lua' no file '/root/torch/install/share/luajit-2.1.0-beta1/cutorch.lua' no file '/usr/local/share/lua/5.1/cutorch.lua' no file '/usr/local/share/lua/5.1/cutorch/init.lua' no file '/root/.luarocks/lib/lua/5.1/cutorch.so' no file '/root/torch-cl/install/lib/lua/5.1/cutorch.so' no file '/root/torch-cl/install/lib/cutorch.so' no file '/root/torch/install/lib/cutorch.so' no file '/root/torch/install/lib/lua/5.1/cutorch.so' no file './cutorch.so' no file '/usr/local/lib/lua/5.1/cutorch.so' no file '/usr/local/lib/lua/5.1/loadall.so' stack traceback: [C]: in function 'error' /root/torch-cl/install/share/lua/5.1/trepl/init.lua:384: in function 'require' neural_style.lua:327: in function 'setup_gpu' neural_style.lua:53: in function 'main' neural_style.lua:601: in main chunk [C]: in function 'dofile' ...t/torch-cl/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x5574787f2f50 (base) [email protected]:~/torch/install/neural-style#

    Again I can't use cuda or cutorch because i have an amd processor. In the tutorial it says that cltorch is supported, but again it wants to reference files that are impossible for me to install. I've tried installing cuda and cutorch anway, but cutorch immediately recognized my invalid hardware. I'm new to linux and everything torch, so this is like all a little giberish! thank you for your time.

    Reply
  • Torch: not enough memory: you tried to allocate 0GB. Buy new RAM!
    Torch: not enough memory: you tried to allocate 0GB. Buy new RAM!

    Oct 31, 2021

    Screenshot_1

    Reply
  • Flexible gradients normalization
    Flexible gradients normalization

    Feb 2, 2017

    Since results with regular and normalized gradients are quite different, maybe it would be useful to apply floating point normalization factor instead of boolean "on/off" option.

    Difference – smooth choice between ordinary and normalized gradients (above are the old option values, below are the new ones):

    flexible_gradients_normalization_full_range

    Disadvantages: new syntax is incompatible with old option (need to use "-normalize_gradients 1.0").

    New "normalize_gradients" value "0.0" (or skipped option) corresponds to old value "false" (normalization skipped completely), and new value "1.0" corresponds to old value "true" (old code is used unchanged).

    *Update: added normalization auto-disabling when content losses are low – it helps to begin stylization faster. With this, option values above 1.0 always make noisy garbage, therefore its range should be limited to 1.

    Reply
  • 16-layer VGG Face Model
    16-layer VGG Face Model

    Jan 4, 2016

    I thought it would be interesting to see what the 16-layer VGG face model does:

    http://www.robots.ox.ac.uk/~vgg/software/vgg_face/

    I tried it with a couple different 16-layer prototxt model files I found (links below):

    th neural_style.lua -proto_file models/VGG_ILSVRC_16_layers_deploy.prototxt -model_file VGG_ILSVRC_16_layers.caffemodel  -gpu 0 -backend cudnn
    

    Either way I get this error:

    Couldn't load VGG_ILSVRC_16_layers.caffemodel
    /home/prime/torch/install/bin/luajit: neural_style.lua:79: attempt to index a nil value
    stack traceback:
        neural_style.lua:79: in function 'main'
        neural_style.lua:490: in main chunk
        [C]: in function 'dofile'
        ...rime/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
        [C]: at 0x00406670
    

    Do I need to generate .lua files like the other prototxt files have? I couldn't quite figure out how to write one by hand at first glance.

    I looked and all the same content and style layer names used in the default 19 layer model appear to be present.

    https://github.com/samim23/NeuralTalkAnimator/blob/master/python_features/VGG_ILSVRC_16_layers_deploy.prototxt.txt https://gist.githubusercontent.com/ksimonyan/211839e770f7b538e2d8/raw/0067c9b32f60362c74f4c445a080beed06b07eb3/VGG_ILSVRC_16_layers_deploy.prototxt

    Reply
  • Getting OpenCl to run with new distrocl
    Getting OpenCl to run with new distrocl

    Jun 11, 2016

    I can run neural-style fine with cpu but trying to run opencl fails. Using a 8gb amd card. I'm fairly new to Ubuntu, but my best guess is that the new way of installing cltorch and clnn through distro-cl is not getting picked up correctly. I passed all the tests without errors running install tests for cltorch source ~/torch-cl/install/bin/torch-activate luajit -l torch -e 'torch.test()' luajit -l nn -e 'nn.test()' luajit -l cltorch -e 'cltorch.test()' luajit -l clnn -e 'clnn.test()

    Here is what i get when I run the neural-style opencl example

    [email protected]:~/neural-style$ th neural_style.lua -style_image examples/inputs/picasso_selfport1907.jpg -content_image examples/inputs/brad_pitt.jpg -output_image profile.png -model_file models/nin_imagenet_conv.caffemodel -proto_file models/train_val.prototxt -gpu 0 -backend clnn -num_iterations 1000 -seed 123 -content_layers relu0,relu3,relu7,relu12 -style_layers relu0,relu3,relu7,relu12 -content_weight 10 -style_weight 1000 -image_size 512 -optimizer adam libthclnn_searchpath /home/zed/torch-cl/install/lib/lua/5.1/libTHCLNN.so Successfully loaded models/nin_imagenet_conv.caffemodel MODULE data UNDEFINED warning: module 'data [type 5]' not found conv1: 96 3 11 11 cccp1: 96 96 1 1 cccp2: 96 96 1 1 conv2: 256 96 5 5 cccp3: 256 256 1 1 cccp4: 256 256 1 1 conv3: 384 256 3 3 cccp5: 384 384 1 1 cccp6: 384 384 1 1 conv4-1024: 1024 384 3 3 cccp7-1024: 1024 1024 1 1 cccp8-1024: 1000 1024 1 1 Using Advanced Micro Devices, Inc. , OpenCL platform: AMD Accelerated Parallel Processing Using OpenCL device: Hawaii Setting up content layer 2 : relu0
    Setting up style layer 2 : relu0
    WARNING: Skipping content loss
    Setting up content layer 9 : relu3
    THClReduceAll.cl build log: "/tmp/OCL24627T10.cl", line 51: warning: function "IndexToOffset_999_get" was declared but never referenced static inline unsigned int IndexToOffset_999_get(unsigned int linearId, global const TensorInfoCl *info) { ^

    "/tmp/OCL24627T10.cl", line 66: warning: function "getLinearBlockId" was declared but never referenced static inline unsigned int getLinearBlockId() { ^

    THClReduceAll.cl build log: "/tmp/OCL24627T11.cl", line 9: warning: variable "in1" was declared but never referenced float *in1 = &_in1; ^

    "/tmp/OCL24627T11.cl", line 10: warning: variable "out" was declared but never referenced float *out = &_out; ^

    "/tmp/OCL24627T11.cl", line 51: warning: function "IndexToOffset_999_get" was declared but never referenced static inline unsigned int IndexToOffset_999_get(unsigned int linearId, global const TensorInfoCl *info) { ^

    "/tmp/OCL24627T11.cl", line 66: warning: function "getLinearBlockId" was declared but never referenced static inline unsigned int getLinearBlockId() { ^

    Setting up style layer 9 : relu3
    WARNING: Skipping content loss
    Setting up content layer 16 : relu7
    Setting up style layer 16 : relu7
    WARNING: Skipping content loss
    Setting up content layer 28 : relu12
    Setting up style layer 28 : relu12
    WARNING: Skipping content loss
    Running optimization with ADAM
    /home/zed/torch/install/bin/luajit: /home/zed/torch/install/share/lua/5.1/nn/Container.lua:67: In 34 module of nn.Sequential: /home/zed/torch/install/share/lua/5.1/nn/THNN.lua:109: bad argument #8 to 'v' (cannot convert 'number' to 'struct THClTensor *') stack traceback: [C]: in function 'v' /home/zed/torch/install/share/lua/5.1/nn/THNN.lua:109: in function 'SpatialConvolutionMM_updateGradInput' ...ed/torch/install/share/lua/5.1/nn/SpatialConvolution.lua:133: in function <...ed/torch/install/share/lua/5.1/nn/SpatialConvolution.lua:127> [C]: in function 'xpcall' /home/zed/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors' /home/zed/torch/install/share/lua/5.1/nn/Sequential.lua:55: in function 'updateGradInput' neural_style.lua:320: in function 'opfunc' /home/zed/torch/install/share/lua/5.1/optim/adam.lua:33: in function 'adam' neural_style.lua:343: in function 'main' neural_style.lua:500: in main chunk [C]: in function 'dofile' .../zed/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00406670

    WARNING: If you see a stack trace below, it doesn't point to the place where this error occured. Please use only the one above. stack traceback: [C]: in function 'error' /home/zed/torch/install/share/lua/5.1/nn/Container.lua:67: in function 'rethrowErrors' /home/zed/torch/install/share/lua/5.1/nn/Sequential.lua:55: in function 'updateGradInput' neural_style.lua:320: in function 'opfunc' /home/zed/torch/install/share/lua/5.1/optim/adam.lua:33: in function 'adam' neural_style.lua:343: in function 'main' neural_style.lua:500: in main chunk [C]: in function 'dofile' .../zed/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00406670

    Since the new cltorch install uses its own distribution of torch there might be an issue there but might just be my install. Any help would be appreciated.

    Reply
  • Error when running without CUDA: No LuaRocks module found for cutorch
    Error when running without CUDA: No LuaRocks module found for cutorch

    Sep 3, 2015

    Trying to run with the default parameters. Getting this error:

    /Users/x/torch/install/bin/luajit: /Users/x/torch/install/share/lua/5.1/trepl/init.lua:363: module 'cutorch' not found:No LuaRocks module found for cutorch
        no field package.preload['cutorch']
        no file '/Users/x/.luarocks/share/lua/5.1/cutorch.lua'
        no file '/Users/x/.luarocks/share/lua/5.1/cutorch/init.lua'
        no file '/Users/x/torch/install/share/lua/5.1/cutorch.lua'
        no file '/Users/x/torch/install/share/lua/5.1/cutorch/init.lua'
        no file './cutorch.lua'
        no file '/Users/x/torch/install/share/luajit-2.1.0-alpha/cutorch.lua'
        no file '/usr/local/share/lua/5.1/cutorch.lua'
        no file '/usr/local/share/lua/5.1/cutorch/init.lua'
        no file '/Users/x/.luarocks/lib/lua/5.1/cutorch.so'
        no file '/Users/x/torch/install/lib/lua/5.1/cutorch.so'
        no file './cutorch.so'
        no file '/usr/local/lib/lua/5.1/cutorch.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
    stack traceback:
        [C]: in function 'error'
        /Users/x/torch/install/share/lua/5.1/trepl/init.lua:363: in function 'require'
        neural_style.lua:42: in function 'main'
        neural_style.lua:370: in main chunk
        [C]: in function 'dofile'
        .../x/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:131: in main chunk
        [C]: at 0x01028fc7b0
    

    I tried explicitly using the nn engine. Also to install cutorch, which fails due to Specify CUDA_TOOLKIT_ROOT_DIR.

    Note, this is OSX.

    Thanks!

    Reply
  • Hey, sorry to bother you guys again... Reinstalling stuff and hit a weird error code
    Hey, sorry to bother you guys again... Reinstalling stuff and hit a weird error code

    Mar 21, 2018

    I'm finally making good on my promise to make a tutorial for artists on how to use neural style, but.... well I'm reinstalling and I ran into this error and was wondering what I might doing wrong? Following the ubuntu install instructions.

    image

    Reply
  • OpenCL support
    OpenCL support

    Sep 17, 2015

    I tried implementing OpenCL support and the code is at: https://github.com/napsternxg/neural-style/tree/opencl

    However I get the following error when running the code:

    $ $ th neural_style_opencl.lua -style_image examples/inputs/picasso_selfport1907.jpg -content_image examples/inputs/brad_pitt.jpg -gpu 0 -backend 'clnn' -output_image profile.png
    [libprotobuf WARNING google/protobuf/io/coded_stream.cc:505] Reading dangerously large protocol message.  If the message turns out to be larger than 1073741824 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
    [libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 574671192
    Successfully loaded models/VGG_ILSVRC_19_layers.caffemodel
    conv1_1: 64 3 3 3
    conv1_2: 64 64 3 3
    conv2_1: 128 64 3 3
    conv2_2: 128 128 3 3
    conv3_1: 256 128 3 3
    conv3_2: 256 256 3 3
    conv3_3: 256 256 3 3
    conv3_4: 256 256 3 3
    conv4_1: 512 256 3 3
    conv4_2: 512 512 3 3
    conv4_3: 512 512 3 3
    conv4_4: 512 512 3 3
    conv5_1: 512 512 3 3
    conv5_2: 512 512 3 3
    conv5_3: 512 512 3 3
    conv5_4: 512 512 3 3
    fc6: 1 1 25088 4096
    fc7: 1 1 4096 4096
    fc8: 1 1 4096 1000
    Using Advanced Micro Devices, Inc. , OpenCL platform: AMD Accelerated Parallel Processing
    Using OpenCL device: Turks
    /home/torch/install/bin/luajit: C++ exception
    

    I believe the issue is because of the SpatialConvolutionMM which is implemented in ccn2 module.

    Reply