Enable customizable logger levels for Spark apps
Spark apps default to the INFO logging level, which often creates excessive noise. Critical logs can be obscured by Spark's verbose output.
This change allows a custom Log4j properties file to be specified and hosted in the refinery. Users can control logging verbosity for quieter logs.
It only overrides the Spark driver log configuration as Skein is only retrieving those.
https://gerrit.wikimedia.org/r/c/analytics/refinery/+/1100394
Bug: T381074