dataengref errors

Home / Flink / Connectors and classpath

Flink SQL: Could not find any factory for identifier 'kafka'

org.apache.flink.table.api.ValidationExceptionCould not find any factory for identifier 'kafka' that implements 'org.apache.flink.table.factories.DynamicTableFactory' in the classpath.

The 'connector' or 'format' option in a CREATE TABLE statement names a factory that is not on the classpath. Flink discovers connectors with Java SPI, so the connector JAR is missing, the wrong version, or its META-INF/services file was lost when building a fat JAR.

Causes · Fixes · Source

Common causes

  1. The connector JAR (for example flink-sql-connector-kafka) is not in lib/ or not passed to the SQL client.How to confirm: The message lists the available factory identifiers; the one you need is missing from that list.
  2. A fat JAR was built without merging META-INF/services/org.apache.flink.table.factories.Factory, so only one module's factories survived.How to confirm: Unzip the JAR and open that services file; it should list every connector you bundled.
  3. The connector version does not match the Flink version, so the factory class fails to load.How to confirm: Compare the connector release with the Flink version on the connector download page.
  4. A typo in the identifier, for example 'kafak' or 'upsert_kafka' instead of 'upsert-kafka'.How to confirm: Compare the identifier with the list printed in the message.

How to fix it

Add the SQL connector JAR that matches your Flink version to lib/ (or with ADD JAR in the SQL client) and restart the cluster or session.

When shading with Maven, add the ServicesResourceTransformer so service files are merged instead of overwritten.

<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>

Where the message comes from

Message template in the source: Could not find any factory for identifier '%s' that implements '%s' in the classpath.

References

Last checked: