Meeting 2020-06-03


  • Brian
  • Derek
  • John
  • Soundar
  • Tommy Shearer

Presentation from Tommy

  • Dashboard design focused on use-cases
    • Site administrator dashboard giving overview
    • Identifying throughput and packet loss
  • Start with a sample use-case dashboard
    • Show examples to guide discussion
  • How to discover relevant dashboards?


ESnet data

  • Finished checkpoint file
  • Working on back-pressure to avoid overloading the bus

XRootD TCP statistics

  • Derek wrote draft TCP statistics plugin
  • Next step is to update XRootD collector

perfSONAR to bus

  • Need to have another focus meeting
  • Determine if we want to modify python code, or switch to Logstash
  • Use RabbitMQ JWT authorization backend?
    • Will contact provider to configure JWT public key

Ongoing topics

ESnet data

  • Collector on github
  • Uses Netbeam API
  • Have access to SNMP (over 9022 interfaces)
  • Data: traffic (2 years); errors, discards (1 month)
  • Size: About 1GB/mo
  • Working on reviewing schema and moving into production
  • Don’t send to tape now. Wait until we’re ready to move to production.

Sending HTCondor data to both ES instances

  • Chicago ES is newer, and plans to use newer features (ILM)
  • This incompatible with ES on the older GRACC instance
  • Should the data go to tape? Est. around couple hundred MB/day
    • Consensus is yes!

Case studies

  • U Waterloo writeup is complete
  • Purdue/FNAL, Madison, and throughput writeup is in progress
  • Make recorded sessions
  • Convert into webpage, add to SAND website (John)

HTCondor monitoring and emails

  • Derek will send weekly emails to SAND team list

Tape replay

  • Now sending Mar 2019
  • Add a messages per second graph