Add greenmask for randomized database imports

[danlamanna]

This adds https://docs.greenmask.io/latest as a development utility which makes it easy to get a small subset of real, redacted, production data.

@annehaley can you give dev/greenmask-dump.sh a try locally and let me know how it works? You may want to run the server with export DJANGO_ISIC_FAKE_STORAGE=placeholder since there won't be any real images.

[annehaley]

I was able to run the script and checked the results in the client. I see this in the images list view:

I'm assuming that these blank images are expected for this fake data (though maybe the "Showing 0 results" should be fixed?). Each "image" has associated info and metadata.

[brianhelba]

Can this be put into the Dockerfile?

[danlamanna]

It could, but it depends on features e.g. heroku and psql. Ugh, no it doesn't. What is the heuristic for what should be in the image vs somewhere else? If there were a greenmask feature I assume we'd use that like we do with others.

[danlamanna]

I'm assuming that these blank images are expected for this fake data

Yeah there's 2 options, either export DJANGO_ISIC_FAKE_STORAGE=placeholder to get placeholder images or export DJANGO_ISIC_FAKE_STORAGE=proxy to get the real images at the cost of slower load times. The latter requires auth info I'll send to you offline.

maybe the "Showing 0 results" should be fixed?

We compute most of our counts with elasticsearch so this is unfortunately expected. ./manage.py populate_elasticsearch should fix it.

[annehaley]

Thanks @danlamanna for helping me resolve those issues. I can now use the placeholder or proxy mode for images, and running the populate_elasticsearch command worked to fill in that "0 results" label with the real number.