pyhf: add staterror config to spec to have faithful roundtrip

Description

we should add staterrorconfig as a channel property. Currently most analyses use Poisson as the config, which is not the default and thus needs to be specificed in the XML.

Converting XML -> JSON -> XML loses this information since we never record this non-default value in the JSON and thus cannot reproduce the original XML

during parsing, the staterrorconfig should become a value in the paramsets used by the staterror modifier.

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Comments: 15 (11 by maintainers)

Most upvoted comments

hi, i appreciate the discussion here. we do generally have limited time and I do think this is prioritized enough that we want to make sure it gets into the upcoming major release. What I propose we’ll do (I have not discussed this with the core developers) is to support a user-friendly interface through pyhf in our next major release as this will change the API minimally, but allows for that support.

That said, it is possible to make this functional today in your existing code like so:

import pyhf

import json
ws = pyhf.Workspace(json.load(open('3b_tag21.2.27-1_RW_ExpSyst_79800_multibin_excl_Gtt_2400_5000_800.json')))

pdf = ws.model()
print(pdf.config.param_set('staterror_SR1L_Inj_Lmeff_cuts'))

def to_poisson(func):
    def wrapper(*args, **kwargs):
      result = func(*args, **kwargs)
      result['paramset_type'] = pyhf.parameters.constrained_by_poisson
      return result
    return wrapper

pyhf.modifiers.staterror.required_parset = to_poisson(pyhf.modifiers.staterror.required_parset)

pdf = ws.model()
print(pdf.config.param_set('staterror_SR1L_Inj_Lmeff_cuts'))

and this seems to work in being able to change things on a global-level (which I propose that pyhf supports primarily):

$ python do_poisson.py
<pyhf.parameters.paramsets.constrained_by_normal object at 0x1314b41f0>
<pyhf.parameters.paramsets.constrained_by_poisson object at 0x1317a0310>

This does also allow you to change constraints for other modifier types in a global way in a similar fashion …

To summarize

  • pyhf should support a global configuration for staterror constraint type (at least functionally in the code) which is an API change
  • a future minor release can expand the v1.0.0 HiFa JSON specification to allow for this to be serializable
  • the default staterror constraint should switch to Poisson to match ROOT 6.22+

Maybe to be practical, I think there are multiple aspects to this:

  • change the default from Gaussian to Poisson,
  • make the constraint term configurable,
  • include the constraint term choice in the model specification.

Changing the default without implementing a way to switch behavior might not be ideal, as it will change the results of people who are used to the Gaussian setup. I think it might be possible to make this configurable though without already involving the JSON spec, and only do so in a separate step (assuming that this is maybe easier to make some partial progress). The natural way to configure this to me would be pyhf.Workspace.model via the modifier_settings kwarg, which already takes the interpolation codes for normsys and histosys, and could take another staterror key with something like {"constraint": "poisson"} or {"constraint": "gaussian"} as possible values.