1. Application Development
  2. Table API & SQL

Table API & SQL

Apache Flink features two relational APIs - the Table API and SQL - for unified stream and batch processing. The Table API is a language-integrated query API for Scala and Java that allows the composition of queries from relational operators such as selection, filter, and join in a very intuitive way. Flink’s SQL support is based on Apache Calcite which implements the SQL standard. Queries specified in either interface have the same semantics and specify the same result regardless whether the input is a batch input (DataSet) or a stream input (DataStream).

The Table API and the SQL interfaces are tightly integrated with each other as well as Flink’s DataStream and DataSet APIs. You can easily switch between all APIs and libraries which build upon the APIs. For instance, you can extract patterns from a DataStream using the CEP library and later use the Table API to analyze the patterns, or you might scan, filter, and aggregate a batch table using a SQL query before running a Gelly graph algorithm on the preprocessed data.

Please note that the Table API and SQL are not yet feature complete and are being actively developed. Not all operations are supported by every combination of [Table API, SQL] and [stream, batch] input.

Setup

The Table API and SQL are bundled in the flink-table Maven artifact. The following dependency must be added to your project in order to use the Table API and SQL:

<dependency>
  <groupId>com.alibaba.blink</groupId>
  <artifactId>flink-table_2.11</artifactId>
  <version>1.5.1</version>
</dependency>

In addition, you need to add a dependency for either Flink’s Scala batch or streaming API. For a batch query you need to add:

<dependency>
  <groupId>com.alibaba.blink</groupId>
  <artifactId>flink-scala_2.11</artifactId>
  <version>1.5.1</version>
</dependency>

For a streaming query you need to add:

<dependency>
  <groupId>com.alibaba.blink</groupId>
  <artifactId>flink-streaming-scala_2.11</artifactId>
  <version>1.5.1</version>
</dependency>

Note: Due to an issue in Apache Calcite, which prevents the user classloaders from being garbage-collected, we do not recommend building a fat-jar that includes the flink-table dependency. Instead, we recommend configuring Flink to include the flink-table dependency in the system classloader. This can be done by copying the flink-table.jar file from the ./opt folder to the ./lib folder. See these instructions for further details.

Back to top

Where to go next?

Back to top