onOtherEvent(event: SparkListenerEvent): Unit
SQLHistoryListener
SQLHistoryListener
is a custom SQLListener for History Server. It attaches SQL tab to History Server’s web UI only when the first SparkListenerSQLExecutionStart arrives and shuts onExecutorMetricsUpdate off. It also handles ends of tasks in a slightly different way.
Note
|
Support for SQL UI in History Server was added in SPARK-11206 Support SQL UI on the history server. |
Caution
|
FIXME Add the link to the JIRA. |
onOtherEvent
When SparkListenerSQLExecutionStart
event comes, onOtherEvent
attaches SQL tab to web UI and passes the call to the parent SQLListener.
onTaskEnd
Caution
|
FIXME |
Creating SQLHistoryListener Instance
SQLHistoryListener
is created using a (private[sql]
) SQLHistoryListenerFactory
class (which is SparkHistoryListenerFactory
).
The SQLHistoryListenerFactory
class is registered when SparkUI
creates a web UI for History Server as a Java service in META-INF/services/org.apache.spark.scheduler.SparkHistoryListenerFactory
:
org.apache.spark.sql.execution.ui.SQLHistoryListenerFactory
Note
|
Loading the service uses Java’s ServiceLoader.load method. |