ExternalCatalog — System Catalog of Permanent Entities

ExternalCatalog is the contract for system catalog of permanent entities, i.e. databases, tables, partitions, and functions.

There are currently two implementations of ExternalCatalog.

Table 1. ExternalCatalog Implementations
Catalog Alias	Catalog Class	Description
`in-memory`	`org.apache.spark.sql.catalyst.catalog.InMemoryCatalog`	An in-memory (ephemeral) system catalog
`hive`	`org.apache.spark.sql.hive.HiveExternalCatalog`

spark.sql.catalogImplementation property sets the current ExternalCatalog implementation (with in-memory being the default).

ExternalCatalog contract assumes that implementations offer the following features:

Table 2. ExternalCatalog Features per Entity
Feature	Function	Partitions	Tables	Databases
Create	X	X	X	X
Drop	X	X	X	X
Rename	X	X	X
Get	X	X	X
Check Existence	X		X	X
List	X	X	X
Alter		X	X	X
Load		X	X	X
Set				X

results matching ""