Flink connector

Notifications

User guide:

Source codes: starrocks-connector-for-apache-flink

Naming format of the JAR file:

Flink 1.15 and later: flink-connector-starrocks-${connector_version}_flink-${flink_version}.jar
Prior to Flink 1.15: flink-connector-starrocks-${connector_version}_flink-${flink_version}_${scala_version}.jar

Methods to obtain the JAR file:

Directly download the the Flink connector JAR file from the Maven Central Repository.
Add the Flink connector as a dependency in your Maven project's pom.xml file and download it. For specific instructions, see user guide.
Compile the source codes into Flink connector JAR file. For specific instructions, see user guide.

Version requirements:

Connector	Flink	StarRocks	Java	Scala
1.2.8	1.13,1.14,1.15,1.16,1.17	2.1 and later	8	2.11,2.12
1.2.7	1.11,1.12,1.13,1.14,1.15	2.1 and later	8	2.11,2.12

NOTICE

In general, the latest version of the Flink connector only maintains compatibility with the three most recent versions of Flink.

1.2.8

This release includes some improvements and bug fixes. The notable changes are as follows:

Support Flink 1.16 and 1.17.
Recommend to set sink.label-prefix when the sink is configured to guarantee the exactly-once semantics. For the specific instructions, see Exactly Once.

Improvements

Support to configure whether to use Stream Load transaction interface to guarantee at-least-once. #228
Add retry metrics for sink V1. #229
No need to getLabelState when EXISTING_JOB_STATUS is FINISHED. #231
Remove useless stack trace log for sink V1. #232
[Refactor] Move StarRocksSinkManagerV2 to stream-load-sdk. #233
Automatically detect partial updates according to a Flink table's schema instead of the sink.properties.columns parameter explicitly specified by users. #235
[Refactor] Move probeTransactionStreamLoad to stream-load-sdk. #240
Add git-commit-id-plugin for stream-load-sdk. #242
Use info log for DefaultStreamLoader#close. #243
Support to generate stream-load-sdk JAR file without dependencies. #245
Replace fastjson with jackson in stream-load-sdk. #247
Support to process update_before record. #250
Add the Apache license into files. #251
Support to get the exception in stream-load-sdk. #252
Enable strip_outer_array and ignore_json_size by default. #259
Try to cleanup lingering transactions when a Flink job restores and the sink semantics is exactly-once. #271
Return the first exception after the retrying fails. #279

Bug Fixes

Tests

Doc