0
点赞
收藏
分享

微信扫一扫

dremio 的InformationSchemaCatalog 服务一

InformationSchemaCatalog 服务实现了模仿information_schema的能力,让我们可以更好的在bi 工具以及其他系统使用

接口定义

/**

* Facet of the catalog service that provides metadata with an information_schema like API.

* <p>

* The metadata provided may not satisfy the configured metadata policy requirement of various sources; the reads

* are directly from the KVStore without an inline refresh of the metadata.

*/

interface InformationSchemaCatalog {

 

/**

* List catalogs that satisfy the given search query.

*

* @param searchQuery        search query, may be null

* @return iterator of catalogs

*/

Iterator<Catalog> listCatalogs(SearchQuery searchQuery);

 

/**

* List schemata that satisfy the given search query.

*

* @param searchQuery        search query, may be null

* @return iterator of schemata

*/

Iterator<Schema> listSchemata(SearchQuery searchQuery);

 

/**

* List tables (datasets) that satisfy the given search query.

*

* @param searchQuery        search query, may be null

* @return iterator of tables

*/

Iterator<Table> listTables(SearchQuery searchQuery);

 

/**

* List views that satisfy the given search query.

*

* @param searchQuery        search query, may be null

* @return iterator of views

*/

Iterator<View> listViews(SearchQuery searchQuery);

 

/**

* List columns of all datasets that satisfy the given search query.

*

* @param searchQuery        search query, may be null

* @return iterator of columns

*/

Iterator<TableSchema> listTableSchemata(SearchQuery searchQuery);

 

}

子类

dremio 的InformationSchemaCatalog 服务一_ide

 

 

从上图可以看出InformationSchemaCatalog 的直接实现是没有包含PrivilegeCatalog,catalog 的子类需要实现此接口,具体的实现InformationSchemaCatalogImpl
核心依赖NamespaceService(此服务比较重要,dremio 利用了key,value 存储解决数据状态管理)
比如数据表获取的

 

@Override

public Iterator<Table> listTables(SearchQuery searchQuery) {

// 基于NamespaceService进行处理

final Iterable<Map.Entry<NamespaceKey, NameSpaceContainer>> searchResults =

userNamespace.find(getCondition(searchQuery));

 

return StreamSupport.stream(searchResults.spliterator(), false)

filter(IS_NOT_INTERNAL)

filter(IS_DATASET)

map(input -> {

final String sourceName = input.getKey().getRoot();

 

final TableType tableType;

if (input.getValue().getDataset().getType() == DatasetType.VIRTUAL_DATASET) {

tableType = TableType.VIEW;

else if ("sys".equals(sourceName) || "INFORMATION_SCHEMA".equals(sourceName)) {

// 包含了sys 以及INFORMATION_SCHEMA 的处理,sys 数据dremio 内部的,INFORMATION_SCHEMA 属于标准的INFORMATION_SCHEMA

tableType = TableType.SYSTEM_TABLE;

else {

tableType = TableType.TABLE;

}

 

return Table.newBuilder()

setCatalogName(DEFAULT_CATALOG_NAME)

setSchemaName(input.getKey().getParent().toUnescapedString())

setTableName(input.getKey().getName())

setTableType(tableType)

build();

})

iterator();

}

说明

InformationSchemaCatalog 的实现,完善了dremio 对于bi 以及jdbc 周边工具的集成,实现上不难,主要是利用了NamespaceService 服务

参考资料

sabot/kernel/src/main/java/com/dremio/exec/catalog/InformationSchemaCatalog.java
​​​https://docs.dremio.com/software/sql-reference/information-schema/tables/​​

举报

相关推荐

0 条评论