Skip Ribbon Commands
Skip to main content

art-stakeholders

Mar 10
Stakeholder meeting notes - 2016/03/10
Stakeholder meeting notes - 2016/03/10
======================================

Attendees:
----------

Christopher Backhouse, Tim Doody, Lynn Garren, Chris Jones,
Tom Junk, Kyle Knoepfel, Jim Kowalkowski, Rob Kutschke,
Marc Paterno, Brian Rebel, Paul Russo

Announcements
-------------

Power outage killed our best development machine, has taken
some time to recover.


New Issues
----------

In the Event Summary report, in the "Modules in End-Path"
section, the "Visited" column does not represent the results
of SelectEvents, it is actually the number of events input to
the SelectEvents logic.  We propose to insert a "Selected"
column to represent how many events passed the SelectEvents
clause.  A discussion followed about the meanings of many of
the counters in the summary.

Resolved Issues
---------------

#11797 sam_metadata_dumper seg faults and hangs on non-art
       root files

This problem was fixed and pushed to root 6 branch.

#11913 Invalid JSON syntax for "runs" array when multiple
       subruns are processed.

This problem was fixed and pushed to the root 6 branch.

Do we need to backport these fixes to the root 5 branch?
Consensus of those present is that this is not needed, but
MicroBooNE may need this.  A mail will be sent out soliciting
answers from those not present.

Mar 10
Stakeholder meeting notes - 2016/03/03
Stakeholder meeting notes - 2016/03/03
======================================

Attendees:
----------

Kurt Biery, Lynn Garren, Robert Hatcher, Chris Jones,
Kyle Knoepfel, Marc Paterno, Brian Rebel, Paul Russo


Announcements
-------------

Marc Paterno discussed a change to Run and SubRun product
handling: art will now store a set of event ranges per
product to support merging of products which represent
summary information derived from sets of events.

Acturus Issues
--------------

Flexible output file handling development is continuing.
Work on avoiding autoparsing in Root 6 is in progress.

New Issues
----------

#11860 fhiclcpp-types tests fail for GCC5/Clang due to inline
       namespace in demangled symbols

This is a problem only for compilers we currently do not
support.  GCC 5 has an ABI change that Root 6 currently does
not support.

#11119 Allow an EDFilter to not produce all products if it's
       returning false

We are currently soliciting votes.  Kurt Biery for artdaq
voted for no change in behavior.  Nova was not present to vote.

Mar 08
Stakeholder meeting notes - 2016/02/25
Attendees:

Tim Doody, Herb Greenlee, Chris Jones, Tom Junk, Rob Kutschke, Brian Rebel

artists: Lynn Garren, Kyle Knoepfel, Marc Paterno

art announcements

Nothing significant to report since most of the work is going into Arcturus preparation.

Potential renaming of canvas

Due to various issues, the new product "canvas" may be named something else.  Discussions are being had at various levels regarding this. 

Discussion of issue #11119

There was discussion as to whether a product should be required to be put on an event for a filter, even if that filter returns 'false'.  For all stakeholders assembled, the answer was 'yes'.  However, since not all experiments were represented, an email will be sent to the stakeholders for a more complete tally.

Better documentation for art

Tom brought up how it would be helpful to have a reference manual for art.  Marc mentioned that various technologies are under discussion (e.g. Doxygen).  There was also mention of a summer student assisting in the art workbook—Rob said this could be helpful if the scope of the student's work were limited to the more mechanical issues (producing scripts that automatically run tests upon code updates, etc.).  Rob has a meeting next week with interested parties to determine how much benefit there is to this idea.
Feb 25
Stakeholder meeting notes - 2016/02/18
Stakeholder meeting notes - 2016/02/18
======================================

Attendees:
----------

Christopher Backhouse, Tim Doody, Lynn Garren, Chris Green,
Ken Herner, Alex Himmel, Chris Jones, Kyle Knoepfel,
Rob Kutschke, Marc Paterno, Brian Rebel, Paul Russo,
Erica Snider


Announcements
-------------

There was a discussion of which projects and experiments are
going to move to Root 6 and when.

It was noted that the LArIAT online and offline are
independent of one another, so one of them can move to Root 6
without the other having to move as well.

Nova will not switch to Root 6 until July at the earliest.

Rob Kutschke noted that Mu2e would be interested in switching
after their next review.

Mark Paterno noted that Muon g-2 is probably interested in
switching earlier than other experiments might be, but
Adam Lyon will have to speak to that.

The LArSoft ups products will each be split at the end of March
into two products in a similar way to the art split:

The products:

     larcore
     lardata
     larsim

will each become:

     larcore --> larcore, larcorelite,
     lardata --> lardata, lardatalite,
     larsim  --> larsim,  larsimlite.

The lite products will depend only on canvas, and not on art.

Brian Rebel asked for a description of the art/canvas split
and Marc Paterno explained.


Acturus Issues
--------------

The implementation of run and subrun fragments, product
aggregation, and flexible output file handling is continuing.

Work on suppressing root 6 autoparsing is in progress.

An event list handling feature is also being implemented.

Work on the canvas root 5 functionality is nearly complete.
It is expected that we will not need to create any
distributions of the root 5 version of canvas, but if it
becomes necessary this can be done.  For now the purpose is
simply to ease maintenance effort.


New Issues
----------

It was reported that if a job runs the SAM metadata dumper
on a non-art root file the job hangs instead of issuing an
error message and terminating.


Feb 17
Stakeholder meeting notes - 2016/02/11
Stakeholder meeting notes - 2016/02/11
======================================

Attendees:
----------

Christopher Backhouse, Kurt Biery, Lynn Garren,
Patrick Gartung, Chris Green, Tom Junk, Kyle Knoepfel,
Rob Kutschke, Marc Paterno, Brian Rebel, Paul Russo


Acturus Issues
--------------

The run/subrun fragements and flexible output file handling
feature is steadily making progress.

Work is in progress on minimizing root 6 autoparsing.

Work is in progress on an event list handling feature.


New Issues
----------

#11646 - Build Geant4 v10.2 with the USolids library
         (but with USOLIDS_VECGEOM=OFF)

This work was requested by Adam Lyon of muon g-2.

Rob Kutschke, speaking for mu2e, thinks this will be fine so
long as it is a separate release.

LArSoft would like to move to this eventually, but first
needs to move to Geant4 v10.1.

The difficulty is with the Geant4 dependence on CLHEP.
Geant4 v10.1 can use either CLHEP v2.2 or CLHEP v2.3, and
right now our builds use v2.2.  When moving to Geant4 v10.2
we must move up to CLHEP v2.3 which uses c++11 features.

There was some discussion of building art with CLHEP v2.3.

We discussed the possibility of globally disabling root 6
autoparsing at art startup.  There was general agreement that
this will probably be ok.  One thing to note at this will not
prevent user code from re-enabling it whenever necessary.

Dave Adams of Dune has expressed an interest in non-singleton
services.  Marc Paterno explained that this will probably
happen, but it will not be called a service.  Discussions are
ongoing with Amir.

Feb 11
Stakeholder meeting notes - 2016/02/04
Stakeholder meeting notes - 2016/02/04
======================================

Attendees:
----------

Christopher Backhouse, Tim Doody, Lynn Garren, Patrick Gartung,
Chris Jones, Kyle Knoepfel, Rob Kutschke, Marc Paterno,
Brian Rebel, Paul Russo


Announcements
-------------

Marc Paterno announced that the art ups product is being
divided into two new ups products, one of which will still be
named art, and the other of which will be named "canvas".

The intent is to move the data product i/o handling into the
canvas product so that functionality may be used by the
"gallery" analysis framework.

It will be necessary for developers to change their code as
the header files and libraries are going to move around.
Translation scripts will be provided to make the change as
painless as possible.

There will be a release numbering change to clearly separate
the old and new art products:

Root 5: art v1_17_x --> v1_17_100
Root 6: art v1_18_x --> v1_19_x

That is the first release of the "new" art root 5 series will
start with v1_17_100, and the root 6 series will start
numbering at v1_19_00.

A request was made for a repetition of the operating system
survey report from last meeting, so we went over the results
again.  There were 47 answers received, Ubuntu was the most
popular, and there is some interest in Mac OSX, although
El Capitan (the latest release) is still a problem for us
due to the changes in the way that DYLD_LIBRARY_PATH is
handled.  We can build software releases for Ubuntu, and
we are, but we must take special precautions with the build
machines since Ubuntu is not a supported OS at Fermilab.


Acturus Issues
--------------

The implementation of run and subrun fragments, product
aggregation, and flexible file handling is well underway.

An event list handling feature is also being implemented.

Feb 04
Stakeholder meeting notes - 2016/01/28
Stakeholder meeting notes - 2016/01/28
======================================

Attendees:
----------

Tim Doody, John Freeman, Lynn Garren, Kyle Knoepfel,
Rob Kutschke, Marc Paterno, Paul Russo, Erica Snider

Releases
--------

Art 1.17.07
Art 1.18.05

The most important change is to fix #11485, build time
increase and library size increase.  Patrick Gartung has
reorganized the code to restrict the number of places that
regex templates are expanded.

Another change worth noting is all art classes with non-trivial
move constructors have been changed to mark those constructors
noexcept so that the standard library will use move semantics
for those classes in the standard containers.

Acturus Issues
--------------

Flexible file handling work is continuing, progress is being
made.

Resolved Issues.
----------------

* #11495 FindMany/One interface that works with art::Ptr

Rejected as a duplicate of #2536.  Solution is to use the
C++14 initializer list syntax.

Example of using a single Ptr as the reference collection
object:

FindOne<B> myBs{{myAPtr}, e, tag};

* #11177 find_ups_product fails if product name is all
         uppercase

Fixed, only affected people using TRACE_VERSION.

New Issues
----------

* #11118 SelectEvents setting should respect filters by default

The new vote is complete, with only NOvA voting for a change
in the current behavior, so the consensus is for no change.

* Extending OS support in our distributions.

There was some discussion of the results of the OS survey, with
the result that Ubuntu is the clear favorite for extending
support, although there is interest in others.

The problem is that Ubuntu is not a supported OS at Fermilab,
and any build system using it would have to be carefully
handled to not violate lab policy.  Further work will be needed.

Jan 20
Stakeholder meeting notes - 2016/01/14
Stakeholder meeting notes - 2016/01/14
======================================

Attendees:
----------

Christopher Backhouse, Tim Doody, John Freeman, Lynn Garren,
Patrick Gartung, Ken Herner, Chris Jones, Kyle Knoepfel,
Rob Kutschke, Gianluca Petrillo, Paul Russo, Erica Snider


Releases
--------

1.17.06
1.18.04

New features:

* SAM metadata inheritance
* Memory & TimeTracker db bug fixed (nfs).


Acturus Issues
--------------

* Flexible file output handling scheme

The requirements gathering phase is finished, and development
work is about to begin.


New Issues
----------

* #11118 SelectEvents setting should respect filters by default

Survey complete. Choice of 0, 1, or 2(a,b).

 0) No change.  Default to always write all events.
 1) Required if there is a filter in any trigger path.
2a) Default changes to "only write events that pass all trigger
    paths."
2b) Default changes to "only write events that pass one or more
    trigger paths."

nova: 1
muon g-2: 1

lariat: 0 or 1
larsoft: 0 or 1
dune: 0 or 1

uboone: 0
mu2e: 0
darkside: 0
artdaq: 0

Marc Paterno pointed out that there is still a lot of confusion
about what filters and output modules do.  He will send out an
email with a pointer to the art/LArSoft course material on the
topic, and a wiki page with documentation on the subject has
been created.

It was decided that email will be sent out with a more detailed
explanation and another poll will be taken with the choices
limited to options 0 and 1.

* Drop "noex" from SelectEvents syntax

A new wiki page has been created with SelectEvents
documentation.

Disallowing the "noex" spelling for "noexception" was proposed
for safety and there were no objections.

* Ubuntu builds of software

Builds of everything we distribute are available for the
Ubuntu long term support release 14.04 LTS, and in the future,
16.04 LTS.  There is one caveat, ifdhc uses the OSG client
tools which are only available for Scientific Linux, so this
product will not work as expected unless and until the OSG
client tools are available for Ubuntu.  It was however noted
that ifdhc will work on all platforms when an xrootd path is
used.

Jan 11
Stakeholder meeting notes - 2016/01/07
Stakeholder meeting notes - 2016/01/07
======================================

Attendees:
----------

Christopher Backhouse, Kurt Biery, Tim Doody, Lynn Garren,
Chris Jones, Tom Junk, Kyle Knoepfel, Rob Kutschke, Adam Lyon,
Brian Rebel, Paul Russo, Erica Snyder


Resolved Issues.
----------------

* 9830 Inherit sam metadata

Kyle Knoepfel described how to configure the new feature
from the command line and from fhicl files.

Marc Paterno indicated this will make it into the next release.

* 11340 MemoryTracker and TimeTracker file sqlite db crash

Fixed.

Marc Paterno pointed out that the fix required us to drop
database locking semantics, so no other process should
attempt to access the database while the art job is
in progress.

Marc Paterno enquired if this fix needs to be released as
soon as possible, and Herb Greenlee responded that
MemoryTracker and TimeTracker are useless until this is
released because the users must work on a nfs-mounted area in
the gpvm systems.  Lynn Garren indicated that the next build
for LArSoft is next week, so we could do it then.


Acturus Issues
--------------

The new flexible output file handling scheme discussions are
continuing and progress is being made.

* 11118 SelectEvents setting should respect filters by default

This is a request from Christopher Backhouse.

There was a long discussion about the pros and cons of
retaining the current behavior or of creating various kinds
of new behavior.

Marc Paterno asked Christopher Backhouse to write up a proposal
describing the new behavior he seeks.  Marc will then send that
proposal out to the stakeholders for vote.

* 11119 Allow an EDFilter to not produce all products if it's
  returning false

Marc Paterno says we will wait until #11118 is settled before
considering this one.


New Issues
----------

Herb Greenlee asked if we can inherit sam data_stream, event
though it is set by RootOuput and not by the file catalog.
Kyle Knoepfel thinks this can be done.

Rob Kutschke notes a new problem with MemoryTracker or
TimeTracker in that certain information is printed as all
zeroes on output even though the data in the database is
correct.  He will create a new issue for this one.

Jan 04
Stakeholder meeting notes - 2015/12/17
Stakeholder meeting notes - 2015/12/17
======================================

Attendees:
----------

Christopher Backhouse, Kurt Biery, Lynn Garren, Chris Jones,
Kyle Knoepfel, Rob Kutschke, Adam Lyon, Brian Rebel, Paul Russo

Resolved Issues.
----------------
* #11039 What do we need to do to make --print-available-modules
  and --module-description work?

Releases
--------

Art v1_17_05 released.

The immediate reason for the release is a request by
Gianluca Petrillo to update fhiclcpp to v3_12_07 to get
new features.

fhiclcpp enhancements:

* TupleAs template
* conditional config

fhiclcpp breaking changes:

1. The order of constructor arguments for Atom, Sequence, and
   Tuple has changed, the Comment must preceed any default
   value now.

2. The constructor for Sequence and Tuple that takes default
   values has changed the type of the default value, it must
   now either be a brace-initializer or a function-style cast
   to the return value of the config parameter operator().

3. The Sequence<T>::make_empty() member function has
   been removed.

Options to the art program have changed:

art --module_description
art --service-description

consolidated to:

art --print-description <arg>

art --print-available={source, plugin}

The EmptyEventTimestampPlugin now needs Boost Filesystem and
System libraries.

Lynn Garren notes that Ubuntu v14.04 LTS builds will be
available for this version, and Marc Paterno adds that this
build may work on other Debian-based releases.  Some discussion
of sites that use Ubuntu was had.

New Issues
----------

* Known bug in art v1_17_05 reported by NOvA: readng run/subrun
  data products in endRun/endSubRun can crash on file switch.

* Mac OSX El Capitan has a new security feature that ensures
  the DYLD_LIBRARY_PATH environment variable is not copied down
  to subshells.  Turning this feature off requires making
  changes to the system configuration from single-user mode.
  Fermilab may be able to disable this feature, the possibility
  is being investigated.  We may need to change the build
  system to compensate.

* #11118 SelectEvents setting should respect filters by default

  Marc Paterno suggested rescheduling this topic for the Jan. 7
  meeting due to light attendence, and this was agreed upon.

1 - 10Next
About this blog
No, this isn't actually my picture. I just haven't gotten around to updating this section. It's good to know that someone is reading every last word though. Thanks!