RunnableCommand — Generic Logical Command with Side Effects

RunnableCommand is the generic logical command that is executed for its side effects.

RunnableCommand defines one abstract method run that computes a collection of Row records with the side effect, i.e. the result of executing a command.

run(sparkSession: SparkSession): Seq[Row]
Note
 RunnableCommand logical operator is translated to ExecutedCommandExec physical operator in BasicOperators strategy.
Note

run is executed when:
Table 1. Available RunnableCommands (in alphabetical order)
RunnableCommand Description

AddFileCommand

AddJarCommand

AlterDatabasePropertiesCommand

AlterTableAddPartitionCommand

AlterTableChangeColumnCommand

AlterTableDropPartitionCommand

AlterTableRecoverPartitionsCommand

AlterTableRenameCommand

AlterTableRenamePartitionCommand

AlterTableSerDePropertiesCommand

AlterTableSetLocationCommand

AlterTableSetPropertiesCommand

AlterTableUnsetPropertiesCommand

AlterViewAsCommand

AnalyzeColumnCommand

AnalyzeTableCommand

CacheTableCommand

When executed, CacheTableCommand creates a DataFrame followed by registering a temporary view for the optional query.

CACHE LAZY? TABLE [table] (AS? [query])?

CacheTableCommand requests the session-specific Catalog to cache the table.

Note
 CacheTableCommand uses SparkSession to access the Catalog.

If the caching is not LAZY (which is not by default), CacheTableCommand creates a DataFrame for the table and counts the rows (that will trigger the caching).

Note
 CacheTableCommand uses a Spark SQL pattern to trigger DataFrame caching by executing count operation. 
val q = "CACHE TABLE ids AS SELECT * from range(5)"
scala> println(sql(q).queryExecution.logical.numberedTreeString)
00 CacheTableCommand `ids`, false
01    +- 'Project [*]
02       +- 'UnresolvedTableValuedFunction range, [5]

val q2 = "CACHE LAZY TABLE ids"
scala> println(sql(q2).queryExecution.logical.numberedTreeString)
17/05/17 06:16:39 WARN CacheManager: Asked to cache already cached data.
00 CacheTableCommand `ids`, true

ClearCacheCommand

CreateDatabaseCommand

CreateDataSourceTableAsSelectCommand

When executed, …​FIXME

Used exclusively when DataSourceAnalysis evaluation rule resolves CreateTable logical operator with queries using non-Hive table providers (which is when DataFrameWriter saves a DataFrame to a non-Hive table or for Create Table As Select SQL statements)

CreateDataSourceTableCommand

CreateFunctionCommand

CreateHiveTableAsSelectCommand

CreateTableCommand

CreateTableLikeCommand

CreateTempViewUsing

CreateViewCommand

DescribeDatabaseCommand

DescribeFunctionCommand

DescribeTableCommand

DropDatabaseCommand

DropFunctionCommand

DropTableCommand

ExplainCommand

InsertIntoDataSourceCommand

InsertIntoHadoopFsRelationCommand

InsertIntoHiveTable

ListFilesCommand

ListJarsCommand

LoadDataCommand

RefreshResource

RefreshTable

ResetCommand

SaveIntoDataSourceCommand

When executed, requests DataSource to write a DataFrame to a data source per save mode.

Used exclusively when DataFrameWriter is requested to save a DataFrame to a data source.

SetCommand

SetDatabaseCommand

ShowColumnsCommand

ShowCreateTableCommand

ShowDatabasesCommand

ShowFunctionsCommand

ShowPartitionsCommand

ShowTablePropertiesCommand

ShowTablesCommand

StreamingExplainCommand

TruncateTableCommand

UncacheTableCommand

results matching ""

    No results matching ""