Skip to content
This repository was archived by the owner on Apr 8, 2026. It is now read-only.
This repository was archived by the owner on Apr 8, 2026. It is now read-only.

RuntimeError: curand error generating random normals 102 #5

Description

@yxyuni

Using gpu device 0: GeForce GTX 980 (CNMeM is enabled with initial size: 80.0% of memory, cuDNN 5105)
/home/vismod/anaconda2/envs/tensorflow/lib/python2.7/site-packages/theano/sandbox/cuda/init.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.
warnings.warn(warn)
[graphy] floatX = float32
Logpath: /media/vismod/148fd670-6cd2-4eda-9e3c-339a23098e8d/vismod/iaf-master/log
WARNING - Deep VAE - No observers have been added to this run
INFO - Deep VAE - Running command 'train'
INFO - Deep VAE - Started
CVAE1 with {'depths': [2, 2, 2], 'nl': u'elu', 'n_h2': 64, 'n_z': 32, 'shape_x': [3, 32, 32], 'optim': u'adamax', 'weightsharing': False, 'px': u'logistic', 'kernel_x': [5, 5], 'n_h1': 64, 'prior': u'diag', 'posterior': u'down_iaf2_nl', 'pad_x': 0, 'beta2': 0.001, 'beta1': 0.1, 'depth_ar': 1, 'alpha': 0.002, 'kl_min': 0.25, 'downsample_type': u'nn', 'kernel_h': [3, 3]}
ERROR - Deep VAE - Failed after 0:00:01!
Traceback (most recent calls WITHOUT Sacred internals):
File "/media/vismod/148fd670-6cd2-4eda-9e3c-339a23098e8d/vismod/iaf-master/train.py", line 185, in train
model = construct_model(data_init)
File "/media/vismod/148fd670-6cd2-4eda-9e3c-339a23098e8d/vismod/iaf-master/train.py", line 128, in construct_model
model = models.cvae1(**margs)
File "/media/vismod/148fd670-6cd2-4eda-9e3c-339a23098e8d/vismod/iaf-master/models.py", line 543, in cvae1
f_encode_decode(w)
File "/media/vismod/148fd670-6cd2-4eda-9e3c-339a23098e8d/vismod/iaf-master/models.py", line 446, in f_encode_decode
h = layers[i][j].up(h, w)
File "/media/vismod/148fd670-6cd2-4eda-9e3c-339a23098e8d/vismod/iaf-master/models.py", line 144, in up
qz[0] = N.rand.gaussian_diag(qz_mean, 2*qz_logsd)
File "/media/vismod/148fd670-6cd2-4eda-9e3c-339a23098e8d/vismod/iaf-master/graphy/nodes/rand.py", line 81, in gaussian_diag
eps = G.rng_curand.normal(size=mean.shape)
File "/home/vismod/anaconda2/envs/tensorflow/lib/python2.7/site-packages/theano/sandbox/cuda/rng_curand.py", line 368, in normal
self.next_seed())
File "/home/vismod/anaconda2/envs/tensorflow/lib/python2.7/site-packages/theano/sandbox/cuda/rng_curand.py", line 108, in new_auto_update
o_gen, sample = self(generator, cast(v_size, 'int32'))
File "/home/vismod/anaconda2/envs/tensorflow/lib/python2.7/site-packages/theano/gof/op.py", line 668, in call
required = thunk()
File "/home/vismod/anaconda2/envs/tensorflow/lib/python2.7/site-packages/theano/gof/op.py", line 883, in rval
fill_storage()
File "/home/vismod/anaconda2/envs/tensorflow/lib/python2.7/site-packages/theano/gof/cc.py", line 1707, in call
reraise(exc_type, exc_value, exc_trace)
File "", line 2, in reraise
RuntimeError: curand error generating random normals 102

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions