MERGE INTO TABLE is not supported temporarily.

See original GitHub issue

i’m trying to execute a Merge operation between 2 iceberg tables and it seems i get the following error: image

The setup is the following:

        <scala.version>2.12.0</scala.version>
        <spark.version>3.0.2</spark.version>
        <iceberg.version>0.11.1</iceberg.version>
        <hadoop.version>3.1.1</hadoop.version>
        <druid.version>1.1.21</druid.version>
        <kafka.version>2.0.0</kafka.version>
        <hive.version>3.1.1</hive.version>
    val spark = SparkSession
      .builder()
      .master("local[2]")
      .appName("IcebergAPI")
      .config("spark.sql.extensions", "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions")
      .config("spark.sql.catalog.spark_catalog", "org.apache.iceberg.spark.SparkSessionCatalog")
      .config("spark.sql.catalog.spark_catalog.type", "hive")
      .config("spark.sql.catalog.hive_prod", "org.apache.iceberg.spark.SparkCatalog")
      .config("spark.sql.catalog.hive_prod.type", "hive")
      .config("spark.sql.catalog.hive_prod.warehouse", "hdfs://ambari5:8020/warehouse/tablespace/managed/hive")
      .config("spark.sql.catalog.hive_prod.uri", "thrift://ambari6:9083")
      .config("iceberg.engine.hive.enabled", "true")
      .enableHiveSupport()
      .getOrCreate()

now i can execute following sql susseccful only merge failed

spark.sql("create database if not exists hive_prod.ods2")
spark.sql("CREATE TABLE IF NOT EXISTS hive_prod.ods.table4 (id bigint, data string) USING iceberg")
spark.sql("INSERT INTO hive_prod.ods.table4 VALUES (1, 'aa'), (2, 'bb'), (3, 'cc')")
spark.sql("select * from  hive_prod.ods.table4").show()

Do you have any idea why it’s acting like this?

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
RussellSpitzercommented, May 18, 2021

Ha no problem

0reactions
liuzx8888commented, May 18, 2021

@RussellSpitzer Thanks Russell, that was it ! Now it can run normally, thank you very, very, very much, you are such a great person

Read more comments on GitHub >

github_iconTop Results From Across the Web

MERGE INTO TABLE is not supported temporarily on Spark3 ...
yes, I am using Spark3.2.0, Scala2.12, Iceberg0.13.1. And it is running well when I execute some other statements like insert, create, etc. [2] ......
Read more >
Re: getting MERGE INTO TABLE is not supported temporarily.
UnsupportedOperationException : MERGE > INTO TABLE is not supported temporarily. > at > org.apache.spark.sql.execution.
Read more >
Implementing MERGE INTO sql in pyspark - Stack Overflow
Then I tried using MERGE INTO statement on those two temporary views. But it is failing. The reason can be MERGE is not...
Read more >
What's new in Apache Spark 3.0 - delete, update and merge ...
case _: MergeIntoTable => throw new UnsupportedOperationException(s"MERGE INTO TABLE is not supported temporarily.") And as a proof, ...
Read more >
MERGE INTO | Databricks on AWS
Merges a set of updates, insertions, and deletions based on a source table into a target Delta table. This statement is supported only...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found