SessionCatalog — Metastore of Session-Specific Relational Entities

SessionCatalog is a catalog (aka registry or metastore) of session-specific relational entities.

You can access a session-specific SessionCatalog through SessionState.

val spark: SparkSession = ...
spark.sessionState.catalog

SessionCatalog is created when SessionState sets catalog (lazily).

Table 1. SessionCatalog’s Internal Registries and Counters (in alphabetical order)
Name	Description
`tempTables`	FIXME Used when…FIXME
`currentDb`	FIXME Used when…FIXME
`functionResourceLoader`	FIXME Used when…FIXME
`tableRelationCache`	A cache of fully-qualified table names to table relation plans (i.e. `LogicalPlan`). Used when `SessionCatalog` refreshes a table

`functionExists` Method

Caution

FIXME

Note	`functionExists` is used in: LookupFunctions logical evaluation rule (to make sure that `UnresolvedFunction` can be resolved, i.e. is registered with `SessionCatalog`) `CatalogImpl` to check if a function exists in a database …

`listFunctions` Method

Caution

FIXME

`refreshTable` Method

Caution

FIXME

`createTempFunction` Method

Caution

FIXME

`loadFunctionResources` Method

Caution

FIXME

`alterTempViewDefinition` Method

alterTempViewDefinition(name: TableIdentifier, viewDefinition: LogicalPlan): Boolean

alterTempViewDefinition alters the temporary view by updating an in-memory temporary table (when a database is not specified and the table has already been registered) or a global temporary table (when a database is specified and it is for global temporary tables).

Note	"Temporary table" and "temporary view" are synonyms.

alterTempViewDefinition returns true when an update could be executed and finished successfully.

`createTempView` Method

Caution

FIXME

`createGlobalTempView` Method

Caution

FIXME

`createTable` Method

Caution

FIXME

`alterTable` Method

Caution

FIXME

Creating SessionCatalog Instance

SessionCatalog takes the following when created:

ExternalCatalog
GlobalTempViewManager
FunctionResourceLoader
FunctionRegistry
CatalystConf
Hadoop’s Configuration
ParserInterface

SessionCatalog initializes the internal registries and counters.

Finding Function by Name (Using FunctionRegistry) — `lookupFunction` Method

lookupFunction(
  name: FunctionIdentifier,
  children: Seq[Expression]): Expression

lookupFunction finds a function by name.

For a function with no database defined that exists in FunctionRegistry, lookupFunction requests FunctionRegistry to find the function (by its unqualified name, i.e. with no database).

If the name function has the database defined or does not exist in FunctionRegistry, lookupFunction uses the fully-qualified function name to check if the function exists in FunctionRegistry (by its fully-qualified name, i.e. with a database).

For other cases, lookupFunction requests ExternalCatalog to find the function and loads its resources. It then creates a corresponding temporary function and looks up the function again.

Note	`lookupFunction` is used exclusively when `Analyzer` resolves functions.

SessionCatalog — Metastore of Session-Specific Relational Entities