[AN-8382] Add script to save Hadoop counters after workflow completion
This script grabs all available job counters from the HistoryServer once a workflow finishes, and transforms them according to configuration, making them suitable to then ship to a local statsd endpoint. The script will collect all available counters - default and custom - and convert their Java-esque names to a simpler format. It also allows multiple "templates" to be defined, which allows generating multiple output metrics (which allow dimensions to be added, removed, for aggregation, etc) from a single input metric.
Showing
scripts/collect-hadoop-metrics.py
0 → 100644
This diff is collapsed.
Click to expand it.
scripts/collect-hadoop-metrics.yaml.example
0 → 100644
Please
register
or
sign in
to comment