Add conversion backend

[alecandido]

As we have different MC available as back-end (at the moment mg5 and yadism), we should add a conversion back-end powered by pineappl conversion scripts.

Indeed, we are not able to produce all of the grids needed (and we won't be for quite some time), as some of them are the result of MC runs, with some non-publicly available MC.
In these cases we're gently gifted the runcards, so we should download them from somewhere else (or have the user running rr downloading them), and then convert to pineappl.

[cschwan]

What exactly should we convert? Can you give an example?

[alecandido]

I'm not so familiar with our APPLgrids/fastNLO grids, but for example (if I remember correctly) we should have grid from NNLOjet, for which we don't have the code, and we won't have soon (if ever) another code to produce them.

Even though, I guess most of the things are actually K-factors...
(but we agreed to drop K-factors, didn't we?)

[cschwan]

Do you mean those: https://ploughshare.web.cern.ch/ploughshare/? For them we have appl2pine and fnlo2pine, of course.

[alecandido]

I was not even aware of the website, and I wonder if there are others, since I remember that at the time of 4.0 publication NNPDF contacted people in order to check if it was fine to make grids public (that to me meant some grids were not public yet, but given directly to NNPDF).

[alecandido]

In any case, these or others, since those grids are not PineAPPL grids and we need to convert, I was thinking about make appl2pine and fnlo2pine part of the runcards runner, such that you have a uniform way (a single command) to generate them, even if they are coming by conversion.

Maybe, it would be appropriate to warn explicitly in case of conversion, but this can always be done.

[felixhekhorn]

Maybe, it would be appropriate to warn explicitly in case of conversion, but this can always be done.

Why do we need a warning? it is just generated by some specific program, i.e. wget more or less

[felixhekhorn]

@cschwan to give you a bit more of context: yesterday we were brainstorming a bit about the new "theory layout" (theory in Emanuele sense) and we figured it should be sufficient to have a list of PineAPPL grids together with a file which spells out how to combine the grids together to match to the experimental datasets - the specific format of that file is to be discussed in https://github.com/NNPDF/fktables/issues/12

[alecandido]

The specific program will be appl2pine (for example), and thus pineappl, and wget-like.

In any case, I thought that by default a computation from scratch is expected, since this should run (based on the runcards). If a conversion of a former run is happening behind the scenes, better to warn, or not?

[felixhekhorn]

Ok, I see - but actually it's a bit more complex: the user only specifies the theory card, (which, as you said, would be ignored since we have no other choice), but the "MC runcard" is always in the (runcards) repo and not explicit (so is by chance implicitly the one the NNLOjet people used)

[alecandido]

I guess we have no access to the NNLOjet runcard, since we have even no access to NNLOjet.

So I believe in the corresponding runcards folder, there will be only metadata, and the actual runcard will be replaced with information needed to retrieve the grid...

[scarlehoff]

I think NNLOjet has only been used for K-factors?

[alecandido]

Yeah, but the idea was to get rid of K-factors, and burn them into PineAPPL grids, if I'm not wrong...

[scarlehoff]

Really? In any case, what information do you need from NNLOJET (or any other program?). The K-factors are a bit of a "god given" number.

I think the idea is (or should be) getting rid of K-factors and using NNLO grids.

[alecandido]

But how do you generate NNLO grids? And in particular, how do you generate NNLO grids in the next few weeks/months?

[scarlehoff]

You don't. That's why I'm not entirely sure why is this relevant. In the next few weeks/months the k-factors are "god-given" numbers already in NNPDF and that are applied on top of the NLO grids.